2024-09-14T06:01:48.526 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-14T06:01:48.542 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T06:01:48.618 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900364 branch: main description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_copy_on_read} email: ceph-qa@ceph.com first_in_suite: false job_id: '7900364' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: main ceph: conf: client: rbd cache: true rbd clone copy on read: true global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 0d61375b8ab53be4c233f5b93e463edfdff9316a owner: scheduled_teuthology@teuthology priority: 950 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 5839 sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 sleep_before_teardown: 0 subset: 99/128 suite: rbd suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 0d61375b8ab53be4c233f5b93e463edfdff9316a targets: smithi039.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIAEO7C4Kjm91cr+Ms+0e8C1iVA2b5X93iIILPW4ZZM99aYEvjXv/95k/fhT5bpE9yyK3msPDvwpzxgE76SUTdY= smithi087.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCuSgZ9zpMJxTBT0CKGFS/T1G6vXMeITTyGGw6bcFJyMHd717i2wRejZOl0clEbUZpGCvL6vmlzQSB/b9XhGG+g= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_20:16:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-14T06:01:48.618 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa; will attempt to use it 2024-09-14T06:01:48.619 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa/tasks 2024-09-14T06:01:48.619 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-14T06:01:48.621 INFO:teuthology.task.internal:Checking packages... 2024-09-14T06:01:48.640 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '7f24e589328ae9e639c0009d93340e5b235d0a47' 2024-09-14T06:01:48.641 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-14T06:01:48.641 INFO:teuthology.packaging:ref: None 2024-09-14T06:01:48.641 INFO:teuthology.packaging:tag: None 2024-09-14T06:01:48.641 INFO:teuthology.packaging:branch: main 2024-09-14T06:01:48.641 INFO:teuthology.packaging:sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T06:01:48.641 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=main 2024-09-14T06:01:48.867 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4907-ga2782315-1jammy 2024-09-14T06:01:48.869 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-14T06:01:48.879 INFO:teuthology.task.internal:no buildpackages task found 2024-09-14T06:01:48.879 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-14T06:01:48.892 INFO:teuthology.task.internal:Saving configuration 2024-09-14T06:01:48.905 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-14T06:01:48.912 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-14T06:01:48.931 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900364', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 05:55:08.411070', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPd6tIWhVVicpq2GQ1iG9g9Bc2mnB7MjfammzwL+ox7b'} 2024-09-14T06:01:48.953 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900364', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 05:55:08.412454', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKACACuOmGY5T0TMprYAwhDZsQy/a5GQOlqS/HFp7F6K'} 2024-09-14T06:01:48.953 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-14T06:01:48.960 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-14T06:01:48.960 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-14T06:01:48.960 INFO:teuthology.run_tasks:Running task console_log... 2024-09-14T06:01:49.017 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe2eda77400>, signals=[15]) 2024-09-14T06:01:49.017 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-14T06:01:49.028 INFO:teuthology.task.internal:Opening connections... 2024-09-14T06:01:49.028 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi039.front.sepia.ceph.com 2024-09-14T06:01:49.030 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:01:49.122 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi087.front.sepia.ceph.com 2024-09-14T06:01:49.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:01:49.199 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-14T06:01:49.205 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2024-09-14T06:01:49.210 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2024-09-14T06:01:49.211 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2024-09-14T06:01:49.256 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-14T06:01:49.256 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="Ubuntu" 2024-09-14T06:01:49.256 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="22.04" 2024-09-14T06:01:49.256 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-14T06:01:49.256 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_CODENAME=jammy 2024-09-14T06:01:49.256 INFO:teuthology.orchestra.run.smithi039.stdout:ID=ubuntu 2024-09-14T06:01:49.256 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE=debian 2024-09-14T06:01:49.256 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-14T06:01:49.257 INFO:teuthology.orchestra.run.smithi039.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-14T06:01:49.257 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-14T06:01:49.257 INFO:teuthology.orchestra.run.smithi039.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-14T06:01:49.257 INFO:teuthology.orchestra.run.smithi039.stdout:UBUNTU_CODENAME=jammy 2024-09-14T06:01:49.257 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2024-09-14T06:01:49.279 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2024-09-14T06:01:49.284 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2024-09-14T06:01:49.284 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2024-09-14T06:01:49.332 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-14T06:01:49.332 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="Ubuntu" 2024-09-14T06:01:49.332 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="22.04" 2024-09-14T06:01:49.332 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-14T06:01:49.332 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_CODENAME=jammy 2024-09-14T06:01:49.332 INFO:teuthology.orchestra.run.smithi087.stdout:ID=ubuntu 2024-09-14T06:01:49.332 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE=debian 2024-09-14T06:01:49.332 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-14T06:01:49.332 INFO:teuthology.orchestra.run.smithi087.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-14T06:01:49.333 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-14T06:01:49.333 INFO:teuthology.orchestra.run.smithi087.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-14T06:01:49.333 INFO:teuthology.orchestra.run.smithi087.stdout:UBUNTU_CODENAME=jammy 2024-09-14T06:01:49.333 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2024-09-14T06:01:49.353 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-14T06:01:49.362 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-14T06:01:49.368 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-14T06:01:49.368 DEBUG:teuthology.orchestra.run.smithi039:> test '!' -e /home/ubuntu/cephtest 2024-09-14T06:01:49.371 DEBUG:teuthology.orchestra.run.smithi087:> test '!' -e /home/ubuntu/cephtest 2024-09-14T06:01:49.379 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-14T06:01:49.385 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-14T06:01:49.385 DEBUG:teuthology.orchestra.run.smithi039:> test -z $(ls -A /var/lib/ceph) 2024-09-14T06:01:49.417 DEBUG:teuthology.orchestra.run.smithi087:> test -z $(ls -A /var/lib/ceph) 2024-09-14T06:01:49.426 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-14T06:01:49.476 INFO:teuthology.run_tasks:Running task kernel... 2024-09-14T06:01:49.490 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-14T06:01:49.490 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-14T06:01:49.491 DEBUG:teuthology.orchestra.run.smithi039:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T06:01:49.491 DEBUG:teuthology.orchestra.run.smithi087:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T06:01:49.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:01:49.496 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-09-14T06:01:49.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:01:49.497 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-09-14T06:01:49.544 INFO:teuthology.orchestra.run.smithi039.stdout:5.15.0-56-generic 2024-09-14T06:01:49.544 INFO:teuthology.task.kernel:Running kernel on smithi039: 5.15.0-56-generic 2024-09-14T06:01:49.544 DEBUG:teuthology.orchestra.run.smithi039:> sudo apt-get clean 2024-09-14T06:01:49.545 INFO:teuthology.orchestra.run.smithi087.stdout:5.15.0-56-generic 2024-09-14T06:01:49.545 INFO:teuthology.task.kernel:Running kernel on smithi087: 5.15.0-56-generic 2024-09-14T06:01:49.545 DEBUG:teuthology.orchestra.run.smithi087:> sudo apt-get clean 2024-09-14T06:01:49.675 DEBUG:teuthology.orchestra.run.smithi087:> sudo apt-get update 2024-09-14T06:01:49.684 DEBUG:teuthology.orchestra.run.smithi039:> sudo apt-get update 2024-09-14T06:01:49.817 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T06:01:49.830 INFO:teuthology.orchestra.run.smithi087.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-14T06:01:49.842 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T06:01:49.848 INFO:teuthology.orchestra.run.smithi039.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-14T06:01:49.849 INFO:teuthology.orchestra.run.smithi039.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-14T06:01:49.899 INFO:teuthology.orchestra.run.smithi087.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-14T06:01:49.911 INFO:teuthology.orchestra.run.smithi039.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-14T06:01:49.953 INFO:teuthology.orchestra.run.smithi087.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-14T06:01:50.049 INFO:teuthology.orchestra.run.smithi087.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-14T06:01:50.107 INFO:teuthology.orchestra.run.smithi039.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-14T06:01:50.116 INFO:teuthology.orchestra.run.smithi087.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-14T06:01:50.152 INFO:teuthology.orchestra.run.smithi087.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-14T06:01:50.155 INFO:teuthology.orchestra.run.smithi087.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-14T06:01:50.155 INFO:teuthology.orchestra.run.smithi087.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-14T06:01:50.171 INFO:teuthology.orchestra.run.smithi087.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-14T06:01:50.172 INFO:teuthology.orchestra.run.smithi087.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-14T06:01:50.175 INFO:teuthology.orchestra.run.smithi087.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-14T06:01:50.175 INFO:teuthology.orchestra.run.smithi087.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-14T06:01:50.176 INFO:teuthology.orchestra.run.smithi039.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-14T06:01:50.179 INFO:teuthology.orchestra.run.smithi087.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-14T06:01:50.186 INFO:teuthology.orchestra.run.smithi087.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-14T06:01:50.187 INFO:teuthology.orchestra.run.smithi039.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-14T06:01:50.188 INFO:teuthology.orchestra.run.smithi087.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-14T06:01:50.188 INFO:teuthology.orchestra.run.smithi087.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-14T06:01:50.200 INFO:teuthology.orchestra.run.smithi087.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-14T06:01:50.205 INFO:teuthology.orchestra.run.smithi087.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-14T06:01:50.205 INFO:teuthology.orchestra.run.smithi087.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-14T06:01:50.205 INFO:teuthology.orchestra.run.smithi087.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-14T06:01:50.206 INFO:teuthology.orchestra.run.smithi087.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-14T06:01:50.206 INFO:teuthology.orchestra.run.smithi087.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-14T06:01:50.214 INFO:teuthology.orchestra.run.smithi039.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-14T06:01:50.217 INFO:teuthology.orchestra.run.smithi039.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-14T06:01:50.217 INFO:teuthology.orchestra.run.smithi039.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-14T06:01:50.222 INFO:teuthology.orchestra.run.smithi087.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-14T06:01:50.222 INFO:teuthology.orchestra.run.smithi087.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-14T06:01:50.223 INFO:teuthology.orchestra.run.smithi087.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-14T06:01:50.223 INFO:teuthology.orchestra.run.smithi087.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-14T06:01:50.223 INFO:teuthology.orchestra.run.smithi087.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-14T06:01:50.237 INFO:teuthology.orchestra.run.smithi039.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-14T06:01:50.237 INFO:teuthology.orchestra.run.smithi039.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-14T06:01:50.242 INFO:teuthology.orchestra.run.smithi039.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-14T06:01:50.243 INFO:teuthology.orchestra.run.smithi039.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-14T06:01:50.247 INFO:teuthology.orchestra.run.smithi039.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-14T06:01:50.251 INFO:teuthology.orchestra.run.smithi039.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-14T06:01:50.252 INFO:teuthology.orchestra.run.smithi039.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-14T06:01:50.252 INFO:teuthology.orchestra.run.smithi039.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-14T06:01:50.253 INFO:teuthology.orchestra.run.smithi039.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-14T06:01:50.288 INFO:teuthology.orchestra.run.smithi039.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-14T06:01:50.288 INFO:teuthology.orchestra.run.smithi039.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-14T06:01:50.288 INFO:teuthology.orchestra.run.smithi039.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-14T06:01:50.292 INFO:teuthology.orchestra.run.smithi039.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-14T06:01:50.295 INFO:teuthology.orchestra.run.smithi039.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-14T06:01:50.295 INFO:teuthology.orchestra.run.smithi039.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-14T06:01:50.316 INFO:teuthology.orchestra.run.smithi039.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-14T06:01:50.316 INFO:teuthology.orchestra.run.smithi039.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-14T06:01:50.319 INFO:teuthology.orchestra.run.smithi039.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-14T06:01:50.319 INFO:teuthology.orchestra.run.smithi039.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-14T06:01:50.327 INFO:teuthology.orchestra.run.smithi039.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-14T06:01:50.334 INFO:teuthology.orchestra.run.smithi039.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-14T06:01:50.337 INFO:teuthology.orchestra.run.smithi039.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-14T06:01:50.337 INFO:teuthology.orchestra.run.smithi039.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-14T06:01:50.337 INFO:teuthology.orchestra.run.smithi039.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-14T06:01:50.346 INFO:teuthology.orchestra.run.smithi039.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-14T06:01:50.346 INFO:teuthology.orchestra.run.smithi039.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-14T06:01:50.347 INFO:teuthology.orchestra.run.smithi039.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-14T06:01:50.347 INFO:teuthology.orchestra.run.smithi039.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-14T06:01:50.348 INFO:teuthology.orchestra.run.smithi039.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-14T06:01:50.363 INFO:teuthology.orchestra.run.smithi039.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-14T06:01:50.363 INFO:teuthology.orchestra.run.smithi039.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-14T06:01:50.364 INFO:teuthology.orchestra.run.smithi039.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-14T06:01:50.364 INFO:teuthology.orchestra.run.smithi039.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-14T06:01:50.364 INFO:teuthology.orchestra.run.smithi039.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-14T06:01:50.392 INFO:teuthology.orchestra.run.smithi087.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-14T06:01:50.813 INFO:teuthology.orchestra.run.smithi087.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-14T06:01:50.829 INFO:teuthology.orchestra.run.smithi087.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-14T06:01:50.838 INFO:teuthology.orchestra.run.smithi087.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-14T06:01:50.838 INFO:teuthology.orchestra.run.smithi087.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-14T06:01:50.840 INFO:teuthology.orchestra.run.smithi087.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-14T06:01:50.936 INFO:teuthology.orchestra.run.smithi087.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-14T06:01:50.947 INFO:teuthology.orchestra.run.smithi087.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-14T06:01:50.947 INFO:teuthology.orchestra.run.smithi087.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-14T06:01:50.963 INFO:teuthology.orchestra.run.smithi087.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-14T06:01:50.993 INFO:teuthology.orchestra.run.smithi087.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-14T06:01:50.997 INFO:teuthology.orchestra.run.smithi087.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-14T06:01:50.998 INFO:teuthology.orchestra.run.smithi087.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-14T06:01:50.999 INFO:teuthology.orchestra.run.smithi087.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-14T06:01:50.999 INFO:teuthology.orchestra.run.smithi087.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-14T06:01:51.076 INFO:teuthology.orchestra.run.smithi087.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-14T06:01:56.448 INFO:teuthology.orchestra.run.smithi087.stdout:Fetched 16.0 MB in 2s (7,534 kB/s) 2024-09-14T06:01:56.868 INFO:teuthology.orchestra.run.smithi039.stdout:Fetched 16.0 MB in 2s (7,026 kB/s) 2024-09-14T06:01:57.881 INFO:teuthology.orchestra.run.smithi087.stdout:Reading package lists... 2024-09-14T06:01:57.899 DEBUG:teuthology.orchestra.run.smithi087:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T06:01:57.993 INFO:teuthology.orchestra.run.smithi087.stdout:Reading package lists... 2024-09-14T06:01:58.178 INFO:teuthology.orchestra.run.smithi087.stdout:Building dependency tree... 2024-09-14T06:01:58.179 INFO:teuthology.orchestra.run.smithi087.stdout:Reading state information... 2024-09-14T06:01:58.343 INFO:teuthology.orchestra.run.smithi087.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T06:01:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T06:01:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T06:01:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T06:01:58.344 INFO:teuthology.orchestra.run.smithi087.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-14T06:01:58.344 INFO:teuthology.orchestra.run.smithi087.stdout: python2.7-minimal 2024-09-14T06:01:58.344 INFO:teuthology.orchestra.run.smithi087.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T06:01:58.345 INFO:teuthology.orchestra.run.smithi087.stdout:The following additional packages will be installed: 2024-09-14T06:01:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T06:01:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-14T06:01:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-14T06:01:58.347 INFO:teuthology.orchestra.run.smithi087.stdout:Suggested packages: 2024-09-14T06:01:58.347 INFO:teuthology.orchestra.run.smithi087.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-14T06:01:58.347 INFO:teuthology.orchestra.run.smithi087.stdout:Recommended packages: 2024-09-14T06:01:58.347 INFO:teuthology.orchestra.run.smithi087.stdout: thermald 2024-09-14T06:01:58.359 INFO:teuthology.orchestra.run.smithi039.stdout:Reading package lists... 2024-09-14T06:01:58.378 DEBUG:teuthology.orchestra.run.smithi039:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T06:01:58.395 INFO:teuthology.orchestra.run.smithi087.stdout:The following NEW packages will be installed: 2024-09-14T06:01:58.395 INFO:teuthology.orchestra.run.smithi087.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T06:01:58.395 INFO:teuthology.orchestra.run.smithi087.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-14T06:01:58.396 INFO:teuthology.orchestra.run.smithi087.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-14T06:01:58.397 INFO:teuthology.orchestra.run.smithi087.stdout:The following packages will be upgraded: 2024-09-14T06:01:58.398 INFO:teuthology.orchestra.run.smithi087.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-14T06:01:58.445 INFO:teuthology.orchestra.run.smithi039.stdout:Reading package lists... 2024-09-14T06:01:58.577 INFO:teuthology.orchestra.run.smithi087.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T06:01:58.577 INFO:teuthology.orchestra.run.smithi087.stdout:Need to get 113 MB of archives. 2024-09-14T06:01:58.578 INFO:teuthology.orchestra.run.smithi087.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-14T06:01:58.578 INFO:teuthology.orchestra.run.smithi087.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-14T06:01:58.638 INFO:teuthology.orchestra.run.smithi039.stdout:Building dependency tree... 2024-09-14T06:01:58.638 INFO:teuthology.orchestra.run.smithi039.stdout:Reading state information... 2024-09-14T06:01:58.810 INFO:teuthology.orchestra.run.smithi039.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T06:01:58.810 INFO:teuthology.orchestra.run.smithi039.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T06:01:58.810 INFO:teuthology.orchestra.run.smithi039.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T06:01:58.811 INFO:teuthology.orchestra.run.smithi039.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T06:01:58.811 INFO:teuthology.orchestra.run.smithi039.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-14T06:01:58.811 INFO:teuthology.orchestra.run.smithi039.stdout: python2.7-minimal 2024-09-14T06:01:58.811 INFO:teuthology.orchestra.run.smithi039.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T06:01:58.812 INFO:teuthology.orchestra.run.smithi039.stdout:The following additional packages will be installed: 2024-09-14T06:01:58.812 INFO:teuthology.orchestra.run.smithi039.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T06:01:58.812 INFO:teuthology.orchestra.run.smithi039.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-14T06:01:58.813 INFO:teuthology.orchestra.run.smithi039.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-14T06:01:58.814 INFO:teuthology.orchestra.run.smithi039.stdout:Suggested packages: 2024-09-14T06:01:58.814 INFO:teuthology.orchestra.run.smithi039.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-14T06:01:58.814 INFO:teuthology.orchestra.run.smithi039.stdout:Recommended packages: 2024-09-14T06:01:58.814 INFO:teuthology.orchestra.run.smithi039.stdout: thermald 2024-09-14T06:01:58.876 INFO:teuthology.orchestra.run.smithi039.stdout:The following NEW packages will be installed: 2024-09-14T06:01:58.876 INFO:teuthology.orchestra.run.smithi039.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T06:01:58.876 INFO:teuthology.orchestra.run.smithi039.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-14T06:01:58.877 INFO:teuthology.orchestra.run.smithi039.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-14T06:01:58.878 INFO:teuthology.orchestra.run.smithi039.stdout:The following packages will be upgraded: 2024-09-14T06:01:58.878 INFO:teuthology.orchestra.run.smithi039.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-14T06:01:59.058 INFO:teuthology.orchestra.run.smithi039.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T06:01:59.058 INFO:teuthology.orchestra.run.smithi039.stdout:Need to get 113 MB of archives. 2024-09-14T06:01:59.058 INFO:teuthology.orchestra.run.smithi039.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-14T06:01:59.059 INFO:teuthology.orchestra.run.smithi039.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-14T06:02:00.052 INFO:teuthology.orchestra.run.smithi087.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-14T06:02:00.415 INFO:teuthology.orchestra.run.smithi087.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-14T06:02:00.443 INFO:teuthology.orchestra.run.smithi039.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-14T06:02:00.798 INFO:teuthology.orchestra.run.smithi039.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-14T06:02:02.636 INFO:teuthology.orchestra.run.smithi087.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-14T06:02:02.636 INFO:teuthology.orchestra.run.smithi087.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-14T06:02:02.636 INFO:teuthology.orchestra.run.smithi087.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-14T06:02:02.991 INFO:teuthology.orchestra.run.smithi087.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-14T06:02:03.074 INFO:teuthology.orchestra.run.smithi087.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-14T06:02:03.149 INFO:teuthology.orchestra.run.smithi039.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-14T06:02:03.150 INFO:teuthology.orchestra.run.smithi039.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-14T06:02:03.150 INFO:teuthology.orchestra.run.smithi039.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-14T06:02:03.473 INFO:teuthology.orchestra.run.smithi087.stdout:Fetched 113 MB in 5s (24.3 MB/s) 2024-09-14T06:02:03.513 INFO:teuthology.orchestra.run.smithi039.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-14T06:02:03.595 INFO:teuthology.orchestra.run.smithi039.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-14T06:02:03.621 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-14T06:02:04.060 INFO:teuthology.orchestra.run.smithi039.stdout:Fetched 113 MB in 5s (24.1 MB/s) 2024-09-14T06:02:04.205 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-14T06:02:05.606 INFO:teuthology.orchestra.run.smithi087.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-14T06:02:05.611 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:02:05.692 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:06.145 INFO:teuthology.orchestra.run.smithi039.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-14T06:02:06.149 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:02:06.210 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:08.117 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-14T06:02:08.139 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:02:08.228 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:08.269 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-14T06:02:08.290 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:02:08.388 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:08.603 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-14T06:02:08.625 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:02:08.653 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:08.830 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-14T06:02:08.851 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:02:08.888 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:14.390 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T06:02:14.534 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T06:02:14.839 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T06:02:14.979 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T06:02:15.266 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-14T06:02:15.288 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-14T06:02:15.332 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T06:02:15.345 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T06:02:15.471 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T06:02:15.812 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T06:02:15.916 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T06:02:16.211 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-14T06:02:16.233 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-14T06:02:16.277 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T06:02:21.013 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-14T06:02:21.049 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:02:21.087 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:21.932 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-14T06:02:21.970 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:02:22.007 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:23.623 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T06:02:23.749 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T06:02:24.048 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T06:02:24.145 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:24.271 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-14T06:02:24.399 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:24.731 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T06:02:24.853 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T06:02:25.188 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T06:02:25.837 INFO:teuthology.orchestra.run.smithi087.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-14T06:02:25.837 INFO:teuthology.orchestra.run.smithi087.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-14T06:02:25.837 INFO:teuthology.orchestra.run.smithi087.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-14T06:02:25.837 INFO:teuthology.orchestra.run.smithi087.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-14T06:02:26.176 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:26.177 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:26.302 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-14T06:02:26.405 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:27.561 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:27.751 INFO:teuthology.orchestra.run.smithi039.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-14T06:02:27.751 INFO:teuthology.orchestra.run.smithi039.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-14T06:02:27.751 INFO:teuthology.orchestra.run.smithi039.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-14T06:02:27.751 INFO:teuthology.orchestra.run.smithi039.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-14T06:02:27.926 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:28.904 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-14T06:02:29.030 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-14T06:02:29.173 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:29.234 INFO:teuthology.orchestra.run.smithi087.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-14T06:02:29.234 INFO:teuthology.orchestra.run.smithi087.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-14T06:02:29.368 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:30.786 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-14T06:02:30.896 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-14T06:02:31.022 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:02:31.091 INFO:teuthology.orchestra.run.smithi039.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-14T06:02:31.091 INFO:teuthology.orchestra.run.smithi039.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-14T06:02:42.972 INFO:teuthology.orchestra.run.smithi087.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-14T06:02:42.972 INFO:teuthology.orchestra.run.smithi087.stdout:Sourcing file `/etc/default/grub' 2024-09-14T06:02:42.992 INFO:teuthology.orchestra.run.smithi087.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T06:02:43.020 INFO:teuthology.orchestra.run.smithi087.stdout:Generating grub configuration file ... 2024-09-14T06:02:43.274 INFO:teuthology.orchestra.run.smithi087.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T06:02:43.303 INFO:teuthology.orchestra.run.smithi087.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T06:02:43.606 INFO:teuthology.orchestra.run.smithi087.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T06:02:43.612 INFO:teuthology.orchestra.run.smithi087.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T06:02:43.681 INFO:teuthology.orchestra.run.smithi087.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T06:02:43.686 INFO:teuthology.orchestra.run.smithi087.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T06:02:43.838 INFO:teuthology.orchestra.run.smithi087.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T06:02:43.838 INFO:teuthology.orchestra.run.smithi087.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T06:02:43.838 INFO:teuthology.orchestra.run.smithi087.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T06:02:43.879 INFO:teuthology.orchestra.run.smithi087.stdout:done 2024-09-14T06:02:44.345 DEBUG:teuthology.orchestra.run.smithi087:> dpkg -s linux-image-generic 2024-09-14T06:02:44.365 INFO:teuthology.orchestra.run.smithi087.stdout:Package: linux-image-generic 2024-09-14T06:02:44.365 INFO:teuthology.orchestra.run.smithi087.stdout:Status: install ok installed 2024-09-14T06:02:44.365 INFO:teuthology.orchestra.run.smithi087.stdout:Priority: optional 2024-09-14T06:02:44.366 INFO:teuthology.orchestra.run.smithi087.stdout:Section: kernel 2024-09-14T06:02:44.366 INFO:teuthology.orchestra.run.smithi087.stdout:Installed-Size: 21 2024-09-14T06:02:44.366 INFO:teuthology.orchestra.run.smithi087.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T06:02:44.366 INFO:teuthology.orchestra.run.smithi087.stdout:Architecture: amd64 2024-09-14T06:02:44.366 INFO:teuthology.orchestra.run.smithi087.stdout:Source: linux-meta 2024-09-14T06:02:44.366 INFO:teuthology.orchestra.run.smithi087.stdout:Version: 5.15.0.121.121 2024-09-14T06:02:44.366 INFO:teuthology.orchestra.run.smithi087.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T06:02:44.366 INFO:teuthology.orchestra.run.smithi087.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T06:02:44.366 INFO:teuthology.orchestra.run.smithi087.stdout:Recommends: thermald 2024-09-14T06:02:44.366 INFO:teuthology.orchestra.run.smithi087.stdout:Description: Generic Linux kernel image 2024-09-14T06:02:44.366 INFO:teuthology.orchestra.run.smithi087.stdout: This package will always depend on the latest generic kernel image 2024-09-14T06:02:44.366 INFO:teuthology.orchestra.run.smithi087.stdout: available. 2024-09-14T06:02:44.367 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-14T06:02:44.367 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T06:02:44.368 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T06:02:44.368 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-14T06:02:44.368 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi087.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T06:02:44.368 DEBUG:teuthology.orchestra.run.smithi087:> sudo apt-get clean 2024-09-14T06:02:44.511 DEBUG:teuthology.orchestra.run.smithi087:> sudo apt-get update 2024-09-14T06:02:44.625 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T06:02:44.747 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T06:02:44.838 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T06:02:44.918 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T06:02:45.320 INFO:teuthology.orchestra.run.smithi039.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-14T06:02:45.320 INFO:teuthology.orchestra.run.smithi039.stdout:Sourcing file `/etc/default/grub' 2024-09-14T06:02:45.331 INFO:teuthology.orchestra.run.smithi039.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T06:02:45.347 INFO:teuthology.orchestra.run.smithi039.stdout:Generating grub configuration file ... 2024-09-14T06:02:45.641 INFO:teuthology.orchestra.run.smithi039.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T06:02:45.667 INFO:teuthology.orchestra.run.smithi039.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T06:02:45.955 INFO:teuthology.orchestra.run.smithi039.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T06:02:45.960 INFO:teuthology.orchestra.run.smithi039.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T06:02:46.030 INFO:teuthology.orchestra.run.smithi087.stdout:Reading package lists... 2024-09-14T06:02:46.033 INFO:teuthology.orchestra.run.smithi039.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T06:02:46.038 INFO:teuthology.orchestra.run.smithi039.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T06:02:46.048 DEBUG:teuthology.orchestra.run.smithi087:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T06:02:46.110 INFO:teuthology.orchestra.run.smithi087.stdout:Reading package lists... 2024-09-14T06:02:46.194 INFO:teuthology.orchestra.run.smithi039.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T06:02:46.194 INFO:teuthology.orchestra.run.smithi039.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T06:02:46.194 INFO:teuthology.orchestra.run.smithi039.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T06:02:46.247 INFO:teuthology.orchestra.run.smithi039.stdout:done 2024-09-14T06:02:46.324 INFO:teuthology.orchestra.run.smithi087.stdout:Building dependency tree... 2024-09-14T06:02:46.325 INFO:teuthology.orchestra.run.smithi087.stdout:Reading state information... 2024-09-14T06:02:46.509 INFO:teuthology.orchestra.run.smithi087.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-14T06:02:46.509 INFO:teuthology.orchestra.run.smithi087.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T06:02:46.509 INFO:teuthology.orchestra.run.smithi087.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T06:02:46.509 INFO:teuthology.orchestra.run.smithi087.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T06:02:46.509 INFO:teuthology.orchestra.run.smithi087.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T06:02:46.510 INFO:teuthology.orchestra.run.smithi087.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T06:02:46.510 INFO:teuthology.orchestra.run.smithi087.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T06:02:46.510 INFO:teuthology.orchestra.run.smithi087.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T06:02:46.550 INFO:teuthology.orchestra.run.smithi087.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T06:02:46.551 DEBUG:teuthology.orchestra.run.smithi087:> dpkg -s linux-image-generic 2024-09-14T06:02:46.566 INFO:teuthology.orchestra.run.smithi087.stdout:Package: linux-image-generic 2024-09-14T06:02:46.567 INFO:teuthology.orchestra.run.smithi087.stdout:Status: install ok installed 2024-09-14T06:02:46.567 INFO:teuthology.orchestra.run.smithi087.stdout:Priority: optional 2024-09-14T06:02:46.567 INFO:teuthology.orchestra.run.smithi087.stdout:Section: kernel 2024-09-14T06:02:46.567 INFO:teuthology.orchestra.run.smithi087.stdout:Installed-Size: 21 2024-09-14T06:02:46.567 INFO:teuthology.orchestra.run.smithi087.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T06:02:46.567 INFO:teuthology.orchestra.run.smithi087.stdout:Architecture: amd64 2024-09-14T06:02:46.567 INFO:teuthology.orchestra.run.smithi087.stdout:Source: linux-meta 2024-09-14T06:02:46.567 INFO:teuthology.orchestra.run.smithi087.stdout:Version: 5.15.0.121.121 2024-09-14T06:02:46.567 INFO:teuthology.orchestra.run.smithi087.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T06:02:46.567 INFO:teuthology.orchestra.run.smithi087.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T06:02:46.567 INFO:teuthology.orchestra.run.smithi087.stdout:Recommends: thermald 2024-09-14T06:02:46.567 INFO:teuthology.orchestra.run.smithi087.stdout:Description: Generic Linux kernel image 2024-09-14T06:02:46.568 INFO:teuthology.orchestra.run.smithi087.stdout: This package will always depend on the latest generic kernel image 2024-09-14T06:02:46.568 INFO:teuthology.orchestra.run.smithi087.stdout: available. 2024-09-14T06:02:46.568 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2024-09-14T06:02:46.611 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.PdvhPrA7Gj 2024-09-14T06:02:46.612 DEBUG:teuthology.orchestra.run.smithi087:> sudo cp /boot/grub/grub.cfg /tmp/tmp.PdvhPrA7Gj 2024-09-14T06:02:46.676 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.PdvhPrA7Gj 2024-09-14T06:02:46.720 DEBUG:teuthology.orchestra.run.smithi039:> dpkg -s linux-image-generic 2024-09-14T06:02:46.858 INFO:teuthology.orchestra.run.smithi039.stdout:Package: linux-image-generic 2024-09-14T06:02:46.858 INFO:teuthology.orchestra.run.smithi039.stdout:Status: install ok installed 2024-09-14T06:02:46.858 INFO:teuthology.orchestra.run.smithi039.stdout:Priority: optional 2024-09-14T06:02:46.858 INFO:teuthology.orchestra.run.smithi039.stdout:Section: kernel 2024-09-14T06:02:46.858 INFO:teuthology.orchestra.run.smithi039.stdout:Installed-Size: 21 2024-09-14T06:02:46.858 INFO:teuthology.orchestra.run.smithi039.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T06:02:46.858 INFO:teuthology.orchestra.run.smithi039.stdout:Architecture: amd64 2024-09-14T06:02:46.858 INFO:teuthology.orchestra.run.smithi039.stdout:Source: linux-meta 2024-09-14T06:02:46.859 INFO:teuthology.orchestra.run.smithi039.stdout:Version: 5.15.0.121.121 2024-09-14T06:02:46.859 INFO:teuthology.orchestra.run.smithi039.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T06:02:46.859 INFO:teuthology.orchestra.run.smithi039.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T06:02:46.859 INFO:teuthology.orchestra.run.smithi039.stdout:Recommends: thermald 2024-09-14T06:02:46.859 INFO:teuthology.orchestra.run.smithi039.stdout:Description: Generic Linux kernel image 2024-09-14T06:02:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: This package will always depend on the latest generic kernel image 2024-09-14T06:02:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: available. 2024-09-14T06:02:46.860 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-14T06:02:46.860 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T06:02:46.860 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T06:02:46.860 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-14T06:02:46.860 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi039.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T06:02:46.860 DEBUG:teuthology.orchestra.run.smithi039:> sudo apt-get clean 2024-09-14T06:02:46.893 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.PdvhPrA7Gj is 10KB 2024-09-14T06:02:46.907 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.PdvhPrA7Gj 2024-09-14T06:02:46.912 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-14T06:02:46.967 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:02:46.967 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-14T06:02:46.967 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-14T06:02:47.007 DEBUG:teuthology.orchestra.run.smithi039:> sudo apt-get update 2024-09-14T06:02:47.041 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-14T06:02:47.041 DEBUG:teuthology.orchestra.run.smithi087:> sudo update-grub 2024-09-14T06:02:47.258 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T06:02:47.267 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T06:02:47.349 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T06:02:47.431 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T06:02:48.195 INFO:teuthology.orchestra.run.smithi087.stderr:Sourcing file `/etc/default/grub' 2024-09-14T06:02:48.196 INFO:teuthology.orchestra.run.smithi087.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T06:02:48.198 INFO:teuthology.orchestra.run.smithi087.stderr:Generating grub configuration file ... 2024-09-14T06:02:48.386 INFO:teuthology.orchestra.run.smithi087.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T06:02:48.392 INFO:teuthology.orchestra.run.smithi087.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T06:02:48.658 INFO:teuthology.orchestra.run.smithi087.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T06:02:48.663 INFO:teuthology.orchestra.run.smithi087.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T06:02:48.703 INFO:teuthology.orchestra.run.smithi087.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T06:02:48.708 INFO:teuthology.orchestra.run.smithi087.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T06:02:48.712 INFO:teuthology.orchestra.run.smithi039.stdout:Reading package lists... 2024-09-14T06:02:48.731 DEBUG:teuthology.orchestra.run.smithi039:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T06:02:48.792 INFO:teuthology.orchestra.run.smithi039.stdout:Reading package lists... 2024-09-14T06:02:48.824 INFO:teuthology.orchestra.run.smithi087.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T06:02:48.825 INFO:teuthology.orchestra.run.smithi087.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T06:02:48.825 INFO:teuthology.orchestra.run.smithi087.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T06:02:48.834 INFO:teuthology.orchestra.run.smithi087.stderr:done 2024-09-14T06:02:48.837 DEBUG:teuthology.orchestra.run.smithi087:> sudo shutdown -r now 2024-09-14T06:02:49.005 INFO:teuthology.orchestra.run.smithi039.stdout:Building dependency tree... 2024-09-14T06:02:49.005 INFO:teuthology.orchestra.run.smithi039.stdout:Reading state information... 2024-09-14T06:02:49.207 INFO:teuthology.orchestra.run.smithi039.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-14T06:02:49.207 INFO:teuthology.orchestra.run.smithi039.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T06:02:49.207 INFO:teuthology.orchestra.run.smithi039.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T06:02:49.207 INFO:teuthology.orchestra.run.smithi039.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T06:02:49.207 INFO:teuthology.orchestra.run.smithi039.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T06:02:49.207 INFO:teuthology.orchestra.run.smithi039.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T06:02:49.208 INFO:teuthology.orchestra.run.smithi039.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T06:02:49.208 INFO:teuthology.orchestra.run.smithi039.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T06:02:49.253 INFO:teuthology.orchestra.run.smithi039.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T06:02:49.255 DEBUG:teuthology.orchestra.run.smithi039:> dpkg -s linux-image-generic 2024-09-14T06:02:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:Package: linux-image-generic 2024-09-14T06:02:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:Status: install ok installed 2024-09-14T06:02:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:Priority: optional 2024-09-14T06:02:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:Section: kernel 2024-09-14T06:02:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:Installed-Size: 21 2024-09-14T06:02:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T06:02:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:Architecture: amd64 2024-09-14T06:02:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:Source: linux-meta 2024-09-14T06:02:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:Version: 5.15.0.121.121 2024-09-14T06:02:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T06:02:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T06:02:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:Recommends: thermald 2024-09-14T06:02:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:Description: Generic Linux kernel image 2024-09-14T06:02:49.273 INFO:teuthology.orchestra.run.smithi039.stdout: This package will always depend on the latest generic kernel image 2024-09-14T06:02:49.273 INFO:teuthology.orchestra.run.smithi039.stdout: available. 2024-09-14T06:02:49.273 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-09-14T06:02:49.319 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.zh6agtIxxJ 2024-09-14T06:02:49.320 DEBUG:teuthology.orchestra.run.smithi039:> sudo cp /boot/grub/grub.cfg /tmp/tmp.zh6agtIxxJ 2024-09-14T06:02:49.375 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.zh6agtIxxJ 2024-09-14T06:02:49.496 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.zh6agtIxxJ is 10KB 2024-09-14T06:02:49.510 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.zh6agtIxxJ 2024-09-14T06:02:49.515 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-14T06:02:49.571 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:02:49.571 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-14T06:02:49.571 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-14T06:02:49.639 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-14T06:02:49.639 DEBUG:teuthology.orchestra.run.smithi039:> sudo update-grub 2024-09-14T06:02:50.758 INFO:teuthology.orchestra.run.smithi039.stderr:Sourcing file `/etc/default/grub' 2024-09-14T06:02:50.759 INFO:teuthology.orchestra.run.smithi039.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T06:02:50.762 INFO:teuthology.orchestra.run.smithi039.stderr:Generating grub configuration file ... 2024-09-14T06:02:50.946 INFO:teuthology.orchestra.run.smithi039.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T06:02:50.953 INFO:teuthology.orchestra.run.smithi039.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T06:02:51.295 INFO:teuthology.orchestra.run.smithi039.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T06:02:51.302 INFO:teuthology.orchestra.run.smithi039.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T06:02:51.347 INFO:teuthology.orchestra.run.smithi039.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T06:02:51.353 INFO:teuthology.orchestra.run.smithi039.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T06:02:51.486 INFO:teuthology.orchestra.run.smithi039.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T06:02:51.487 INFO:teuthology.orchestra.run.smithi039.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T06:02:51.487 INFO:teuthology.orchestra.run.smithi039.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T06:02:51.498 INFO:teuthology.orchestra.run.smithi039.stderr:done 2024-09-14T06:02:51.511 DEBUG:teuthology.orchestra.run.smithi039:> sudo shutdown -r now 2024-09-14T06:03:18.862 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-14T06:03:18.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-09-14T06:03:18.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:03:21.516 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-14T06:03:21.517 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-14T06:03:21.517 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:03:56.093 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-09-14T06:04:05.101 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-14T06:04:05.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:04:08.157 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-09-14T06:04:18.877 DEBUG:teuthology.orchestra.remote:timed out 2024-09-14T06:04:20.159 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-14T06:04:20.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:04:27.885 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-09-14T06:04:27.885 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:04:27.887 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-09-14T06:04:39.893 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-09-14T06:04:39.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:04:40.179 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-09-14T06:04:40.705 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-09-14T06:04:40.705 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-121-generic"... 2024-09-14T06:04:40.705 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-09-14T06:04:40.751 INFO:teuthology.orchestra.run.smithi087.stdout:5.15.0-121-generic 2024-09-14T06:04:40.751 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-14T06:04:40.751 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T06:04:40.751 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-14T06:04:41.752 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-14T06:04:41.752 DEBUG:teuthology.orchestra.run.smithi087:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T06:04:42.103 INFO:teuthology.orchestra.run.smithi087.stdout:ttyS1 2024-09-14T06:04:42.125 DEBUG:teuthology.parallel:result is None 2024-09-14T06:05:20.162 DEBUG:teuthology.orchestra.remote:timed out 2024-09-14T06:05:35.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-14T06:05:35.168 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:05:35.361 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-09-14T06:05:35.928 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-14T06:05:35.928 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-121-generic"... 2024-09-14T06:05:35.929 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-09-14T06:05:35.973 INFO:teuthology.orchestra.run.smithi039.stdout:5.15.0-121-generic 2024-09-14T06:05:35.974 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-14T06:05:35.974 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T06:05:35.974 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-14T06:05:36.974 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-14T06:05:36.975 DEBUG:teuthology.orchestra.run.smithi039:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T06:05:37.072 INFO:teuthology.orchestra.run.smithi039.stdout:ttyS1 2024-09-14T06:05:37.099 DEBUG:teuthology.parallel:result is None 2024-09-14T06:05:37.099 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-14T06:05:37.110 INFO:teuthology.task.internal:Creating test directory... 2024-09-14T06:05:37.110 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T06:05:37.112 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T06:05:37.119 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-14T06:05:37.125 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-14T06:05:37.131 INFO:teuthology.task.internal:Creating archive directory... 2024-09-14T06:05:37.131 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T06:05:37.163 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T06:05:37.185 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-14T06:05:37.192 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-14T06:05:37.192 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T06:05:37.215 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T06:05:37.237 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:05:37.244 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:05:37.245 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:05:37.252 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:05:37.254 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-14T06:05:37.261 INFO:teuthology.task.internal:Configuring sudo... 2024-09-14T06:05:37.261 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T06:05:37.291 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T06:05:37.309 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-14T06:05:37.318 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-14T06:05:37.318 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T06:05:37.347 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T06:05:37.358 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T06:05:37.398 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T06:05:37.446 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:05:37.446 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T06:05:37.502 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T06:05:37.508 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T06:05:37.554 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:05:37.555 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T06:05:37.618 DEBUG:teuthology.orchestra.run.smithi039:> sudo service rsyslog restart 2024-09-14T06:05:37.620 DEBUG:teuthology.orchestra.run.smithi087:> sudo service rsyslog restart 2024-09-14T06:05:37.693 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-14T06:05:37.700 INFO:teuthology.task.internal:Starting timer... 2024-09-14T06:05:37.700 INFO:teuthology.run_tasks:Running task pcp... 2024-09-14T06:05:37.711 INFO:teuthology.run_tasks:Running task selinux... 2024-09-14T06:05:37.719 DEBUG:teuthology.task.selinux:Excluding smithi039: OS 'ubuntu' does not support SELinux 2024-09-14T06:05:37.719 DEBUG:teuthology.task.selinux:Excluding smithi087: OS 'ubuntu' does not support SELinux 2024-09-14T06:05:37.719 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-14T06:05:37.719 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-14T06:05:37.719 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-14T06:05:37.719 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-14T06:05:37.728 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-14T06:05:37.741 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-14T06:05:37.742 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi039.front.sepia.ceph.com,smithi087.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-14T06:13:08.374 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi039.front.sepia.ceph.com'), Remote(name='ubuntu@smithi087.front.sepia.ceph.com')] 2024-09-14T06:13:08.376 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-14T06:13:08.377 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:13:08.460 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-09-14T06:13:08.537 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-14T06:13:08.537 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-09-14T06:13:08.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:13:08.615 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-09-14T06:13:08.689 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-09-14T06:13:08.689 INFO:teuthology.run_tasks:Running task clock... 2024-09-14T06:13:08.700 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-14T06:13:08.701 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T06:13:08.701 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T06:13:08.703 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T06:13:08.703 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T06:13:08.733 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-14T06:13:08.733 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: Command line: ntpd -gq 2024-09-14T06:13:08.733 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: ---------------------------------------------------- 2024-09-14T06:13:08.733 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: ntp-4 is maintained by Network Time Foundation, 2024-09-14T06:13:08.733 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-14T06:13:08.733 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: corporation. Support and training for ntp-4 are 2024-09-14T06:13:08.733 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: available at https://www.nwtime.org/support 2024-09-14T06:13:08.733 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: ---------------------------------------------------- 2024-09-14T06:13:08.734 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: proto: precision = 0.046 usec (-24) 2024-09-14T06:13:08.734 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: basedate set to 2022-02-04 2024-09-14T06:13:08.734 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: gps base set to 2022-02-06 (week 2196) 2024-09-14T06:13:08.734 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T06:13:08.734 INFO:teuthology.orchestra.run.smithi039.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T06:13:08.734 INFO:teuthology.orchestra.run.smithi039.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-14T06:13:08.736 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: restrict ::: KOD does nothing without LIMITED. 2024-09-14T06:13:08.736 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: Listen and drop on 0 v6wildcard [::]:123 2024-09-14T06:13:08.736 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-14T06:13:08.736 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: Listen normally on 2 lo 127.0.0.1:123 2024-09-14T06:13:08.736 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: Listen normally on 3 ens1f0 172.21.15.39:123 2024-09-14T06:13:08.736 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: Listen normally on 4 lo [::1]:123 2024-09-14T06:13:08.736 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1370%4]:123 2024-09-14T06:13:08.736 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:08 ntpd[18350]: Listening on routing socket on fd #22 for interface updates 2024-09-14T06:13:08.759 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-14T06:13:08.759 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: Command line: ntpd -gq 2024-09-14T06:13:08.759 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: ---------------------------------------------------- 2024-09-14T06:13:08.759 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: ntp-4 is maintained by Network Time Foundation, 2024-09-14T06:13:08.759 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-14T06:13:08.759 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: corporation. Support and training for ntp-4 are 2024-09-14T06:13:08.759 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: available at https://www.nwtime.org/support 2024-09-14T06:13:08.760 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: ---------------------------------------------------- 2024-09-14T06:13:08.760 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: proto: precision = 0.045 usec (-24) 2024-09-14T06:13:08.760 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: basedate set to 2022-02-04 2024-09-14T06:13:08.760 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: gps base set to 2022-02-06 (week 2196) 2024-09-14T06:13:08.761 INFO:teuthology.orchestra.run.smithi087.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T06:13:08.761 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T06:13:08.761 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: restrict ::: KOD does nothing without LIMITED. 2024-09-14T06:13:08.761 INFO:teuthology.orchestra.run.smithi087.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-14T06:13:08.762 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: Listen and drop on 0 v6wildcard [::]:123 2024-09-14T06:13:08.762 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-14T06:13:08.762 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: Listen normally on 2 lo 127.0.0.1:123 2024-09-14T06:13:08.763 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: Listen normally on 3 enp3s0f1 172.21.15.87:123 2024-09-14T06:13:08.763 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: Listen normally on 4 lo [::1]:123 2024-09-14T06:13:08.763 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d5f5%5]:123 2024-09-14T06:13:08.763 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:08 ntpd[18490]: Listening on routing socket on fd #22 for interface updates 2024-09-14T06:13:09.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:09 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:09.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:09 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:09.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:09 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:09.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:09 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:09.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:09 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:09.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:09 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:09.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:09 ntpd[18350]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:09.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:09 ntpd[18350]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:09.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:09 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:09.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:09 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:09.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:09 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:09.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:09 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:09.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:09 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:09.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:09 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:09.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:09 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:09.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:09 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:09.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:09 ntpd[18490]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:09.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:09 ntpd[18490]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:09.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:09 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:09.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:09 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:09.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:09 ntpd[18490]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:09.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:09 ntpd[18490]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:10.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:10 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:10.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:10 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:10.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:10 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:10.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:10 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:10.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:10 ntpd[18350]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:10.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:10 ntpd[18350]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:11.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:11 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:11.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:11 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:11.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:11 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:11.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:11 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:11.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:11 ntpd[18350]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:11.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:11 ntpd[18350]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:11.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:11 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:11.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:11 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:11.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:11 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:11.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:11 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:11.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:11 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:11.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:11 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:11.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:11 ntpd[18490]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:11.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:11 ntpd[18490]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:11.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:11 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:11.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:11 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:11.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:11 ntpd[18490]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:11.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:11 ntpd[18490]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:12.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:12 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:12 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:12 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:12 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:12 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:12 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:12 ntpd[18350]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:12 ntpd[18350]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:12 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:12 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:12 ntpd[18350]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:12 ntpd[18350]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:13.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:13 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:13.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:13 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:13.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:13 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:13.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:13 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:13.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:13 ntpd[18350]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:13.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:13 ntpd[18350]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:13.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:13.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:13.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:13.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:13.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:13.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:13.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:13.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:13.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:13.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:13.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:13.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:13.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:13.762 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:13.762 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:13.762 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:13.762 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:13.762 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:13 ntpd[18490]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:14.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:14 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:14.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:14 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:14.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:14 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:14.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:14 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:14.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:14 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:14.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:14 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:14.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:14 ntpd[18350]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:14.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:14 ntpd[18350]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:14.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:14 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:14.735 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:14 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:14.735 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:14 ntpd[18350]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:14.735 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:14 ntpd[18350]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:15.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:15 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:15.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:15 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:15.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:15 ntpd[18350]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:15.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:15 ntpd[18350]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:15.733 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:15 ntpd[18350]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:15.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:15 ntpd[18350]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:15.734 INFO:teuthology.orchestra.run.smithi039.stdout:14 Sep 06:13:15 ntpd[18350]: ntpd: time slew -0.000776 s 2024-09-14T06:13:15.734 INFO:teuthology.orchestra.run.smithi039.stdout:ntpd: time slew -0.000776s 2024-09-14T06:13:15.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:15 ntpd[18350]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-14T06:13:15.734 INFO:teuthology.orchestra.run.smithi039.stderr:14 Sep 06:13:15 ntpd[18350]: can't open /var/log/ntpstats/loopstats.20240914: Permission denied 2024-09-14T06:13:15.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:15 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:15.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:15 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:15.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:15 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:15.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:15 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:15.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:15 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:15.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:15 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:15.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:15 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T06:13:15.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:15 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T06:13:15.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:15 ntpd[18490]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T06:13:15.760 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:15 ntpd[18490]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T06:13:15.761 INFO:teuthology.orchestra.run.smithi087.stdout:14 Sep 06:13:15 ntpd[18490]: ntpd: time slew -0.000479 s 2024-09-14T06:13:15.761 INFO:teuthology.orchestra.run.smithi087.stdout:ntpd: time slew -0.000479s 2024-09-14T06:13:15.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:15 ntpd[18490]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-14T06:13:15.761 INFO:teuthology.orchestra.run.smithi087.stderr:14 Sep 06:13:15 ntpd[18490]: can't open /var/log/ntpstats/loopstats.20240914: Permission denied 2024-09-14T06:13:15.794 INFO:teuthology.orchestra.run.smithi039.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T06:13:15.794 INFO:teuthology.orchestra.run.smithi039.stdout:============================================================================== 2024-09-14T06:13:15.794 INFO:teuthology.orchestra.run.smithi039.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T06:13:15.794 INFO:teuthology.orchestra.run.smithi039.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T06:13:15.794 INFO:teuthology.orchestra.run.smithi039.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T06:13:15.794 INFO:teuthology.orchestra.run.smithi039.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T06:13:15.813 INFO:teuthology.orchestra.run.smithi087.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T06:13:15.813 INFO:teuthology.orchestra.run.smithi087.stdout:============================================================================== 2024-09-14T06:13:15.813 INFO:teuthology.orchestra.run.smithi087.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T06:13:15.813 INFO:teuthology.orchestra.run.smithi087.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T06:13:15.813 INFO:teuthology.orchestra.run.smithi087.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T06:13:15.813 INFO:teuthology.orchestra.run.smithi087.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T06:13:15.814 INFO:teuthology.run_tasks:Running task install... 2024-09-14T06:13:15.823 DEBUG:teuthology.task.install:project ceph 2024-09-14T06:13:15.823 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}} 2024-09-14T06:13:15.823 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'} 2024-09-14T06:13:15.823 INFO:teuthology.task.install:Using flavor: default 2024-09-14T06:13:15.829 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-14T06:13:15.829 INFO:teuthology.task.install:extra packages: [] 2024-09-14T06:13:15.829 DEBUG:teuthology.orchestra.run.smithi039:> sudo apt-key list | grep Ceph 2024-09-14T06:13:15.830 DEBUG:teuthology.orchestra.run.smithi087:> sudo apt-key list | grep Ceph 2024-09-14T06:13:15.901 INFO:teuthology.orchestra.run.smithi039.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-14T06:13:15.911 INFO:teuthology.orchestra.run.smithi087.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-14T06:13:15.928 INFO:teuthology.orchestra.run.smithi039.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-14T06:13:15.928 INFO:teuthology.orchestra.run.smithi039.stdout:uid [ unknown] Ceph.com (release key) 2024-09-14T06:13:15.929 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-14T06:13:15.930 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T06:13:15.937 INFO:teuthology.orchestra.run.smithi087.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-14T06:13:15.937 INFO:teuthology.orchestra.run.smithi087.stdout:uid [ unknown] Ceph.com (release key) 2024-09-14T06:13:15.938 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-14T06:13:15.938 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T06:13:16.073 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default/ 2024-09-14T06:13:16.074 INFO:teuthology.task.install.deb:Package version is 19.3.0-4775-g7f24e589-1jammy 2024-09-14T06:13:16.081 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default/ 2024-09-14T06:13:16.081 INFO:teuthology.task.install.deb:Package version is 19.3.0-4775-g7f24e589-1jammy 2024-09-14T06:13:16.183 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:13:16.183 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-14T06:13:16.189 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:13:16.189 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-14T06:13:16.197 DEBUG:teuthology.orchestra.run.smithi039:> sudo apt-get update 2024-09-14T06:13:16.203 DEBUG:teuthology.orchestra.run.smithi087:> sudo apt-get update 2024-09-14T06:13:16.367 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T06:13:16.368 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T06:13:16.368 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T06:13:16.377 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T06:13:16.382 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T06:13:16.394 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T06:13:16.399 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T06:13:16.421 INFO:teuthology.orchestra.run.smithi087.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy InRelease 2024-09-14T06:13:16.421 INFO:teuthology.orchestra.run.smithi039.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy InRelease 2024-09-14T06:13:16.442 INFO:teuthology.orchestra.run.smithi087.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-14T06:13:16.446 INFO:teuthology.orchestra.run.smithi039.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-14T06:13:16.464 INFO:teuthology.orchestra.run.smithi087.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-14T06:13:16.468 INFO:teuthology.orchestra.run.smithi039.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-14T06:13:16.485 INFO:teuthology.orchestra.run.smithi087.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-09-14T06:13:16.489 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:7 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T06:13:16.489 INFO:teuthology.orchestra.run.smithi039.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-14T06:13:16.514 INFO:teuthology.orchestra.run.smithi087.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-14T06:13:16.517 INFO:teuthology.orchestra.run.smithi039.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-09-14T06:13:16.723 INFO:teuthology.orchestra.run.smithi087.stdout:Fetched 29.3 kB in 0s (86.2 kB/s) 2024-09-14T06:13:16.744 INFO:teuthology.orchestra.run.smithi039.stdout:Fetched 29.3 kB in 0s (81.8 kB/s) 2024-09-14T06:13:17.747 INFO:teuthology.orchestra.run.smithi087.stdout:Reading package lists... 2024-09-14T06:13:17.764 DEBUG:teuthology.orchestra.run.smithi087:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4775-g7f24e589-1jammy cephadm=19.3.0-4775-g7f24e589-1jammy ceph-mds=19.3.0-4775-g7f24e589-1jammy ceph-mgr=19.3.0-4775-g7f24e589-1jammy ceph-common=19.3.0-4775-g7f24e589-1jammy ceph-fuse=19.3.0-4775-g7f24e589-1jammy ceph-test=19.3.0-4775-g7f24e589-1jammy ceph-volume=19.3.0-4775-g7f24e589-1jammy radosgw=19.3.0-4775-g7f24e589-1jammy python3-rados=19.3.0-4775-g7f24e589-1jammy python3-rgw=19.3.0-4775-g7f24e589-1jammy python3-cephfs=19.3.0-4775-g7f24e589-1jammy python3-rbd=19.3.0-4775-g7f24e589-1jammy libcephfs2=19.3.0-4775-g7f24e589-1jammy libcephfs-dev=19.3.0-4775-g7f24e589-1jammy librados2=19.3.0-4775-g7f24e589-1jammy librbd1=19.3.0-4775-g7f24e589-1jammy rbd-fuse=19.3.0-4775-g7f24e589-1jammy 2024-09-14T06:13:17.797 INFO:teuthology.orchestra.run.smithi039.stdout:Reading package lists... 2024-09-14T06:13:17.814 DEBUG:teuthology.orchestra.run.smithi039:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4775-g7f24e589-1jammy cephadm=19.3.0-4775-g7f24e589-1jammy ceph-mds=19.3.0-4775-g7f24e589-1jammy ceph-mgr=19.3.0-4775-g7f24e589-1jammy ceph-common=19.3.0-4775-g7f24e589-1jammy ceph-fuse=19.3.0-4775-g7f24e589-1jammy ceph-test=19.3.0-4775-g7f24e589-1jammy ceph-volume=19.3.0-4775-g7f24e589-1jammy radosgw=19.3.0-4775-g7f24e589-1jammy python3-rados=19.3.0-4775-g7f24e589-1jammy python3-rgw=19.3.0-4775-g7f24e589-1jammy python3-cephfs=19.3.0-4775-g7f24e589-1jammy python3-rbd=19.3.0-4775-g7f24e589-1jammy libcephfs2=19.3.0-4775-g7f24e589-1jammy libcephfs-dev=19.3.0-4775-g7f24e589-1jammy librados2=19.3.0-4775-g7f24e589-1jammy librbd1=19.3.0-4775-g7f24e589-1jammy rbd-fuse=19.3.0-4775-g7f24e589-1jammy 2024-09-14T06:13:17.818 INFO:teuthology.orchestra.run.smithi087.stdout:Reading package lists... 2024-09-14T06:13:17.869 INFO:teuthology.orchestra.run.smithi039.stdout:Reading package lists... 2024-09-14T06:13:18.010 INFO:teuthology.orchestra.run.smithi087.stdout:Building dependency tree... 2024-09-14T06:13:18.010 INFO:teuthology.orchestra.run.smithi087.stdout:Reading state information... 2024-09-14T06:13:18.060 INFO:teuthology.orchestra.run.smithi039.stdout:Building dependency tree... 2024-09-14T06:13:18.061 INFO:teuthology.orchestra.run.smithi039.stdout:Reading state information... 2024-09-14T06:13:18.195 INFO:teuthology.orchestra.run.smithi087.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T06:13:18.195 INFO:teuthology.orchestra.run.smithi087.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T06:13:18.195 INFO:teuthology.orchestra.run.smithi087.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T06:13:18.195 INFO:teuthology.orchestra.run.smithi087.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T06:13:18.196 INFO:teuthology.orchestra.run.smithi087.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T06:13:18.196 INFO:teuthology.orchestra.run.smithi087.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T06:13:18.196 INFO:teuthology.orchestra.run.smithi087.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T06:13:18.197 INFO:teuthology.orchestra.run.smithi087.stdout:The following additional packages will be installed: 2024-09-14T06:13:18.197 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-14T06:13:18.197 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-14T06:13:18.197 INFO:teuthology.orchestra.run.smithi087.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-14T06:13:18.197 INFO:teuthology.orchestra.run.smithi087.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-14T06:13:18.197 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-14T06:13:18.198 INFO:teuthology.orchestra.run.smithi087.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-14T06:13:18.198 INFO:teuthology.orchestra.run.smithi087.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-14T06:13:18.198 INFO:teuthology.orchestra.run.smithi087.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-14T06:13:18.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-14T06:13:18.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-14T06:13:18.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-14T06:13:18.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-14T06:13:18.198 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-14T06:13:18.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-14T06:13:18.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-14T06:13:18.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-14T06:13:18.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-14T06:13:18.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-14T06:13:18.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-14T06:13:18.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-14T06:13:18.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-14T06:13:18.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-14T06:13:18.199 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-14T06:13:18.200 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet zip 2024-09-14T06:13:18.200 INFO:teuthology.orchestra.run.smithi087.stdout:Suggested packages: 2024-09-14T06:13:18.200 INFO:teuthology.orchestra.run.smithi087.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-14T06:13:18.200 INFO:teuthology.orchestra.run.smithi087.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-14T06:13:18.200 INFO:teuthology.orchestra.run.smithi087.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-14T06:13:18.200 INFO:teuthology.orchestra.run.smithi087.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-14T06:13:18.200 INFO:teuthology.orchestra.run.smithi087.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-14T06:13:18.200 INFO:teuthology.orchestra.run.smithi087.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-14T06:13:18.200 INFO:teuthology.orchestra.run.smithi087.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-14T06:13:18.201 INFO:teuthology.orchestra.run.smithi087.stdout:Recommended packages: 2024-09-14T06:13:18.201 INFO:teuthology.orchestra.run.smithi087.stdout: btrfs-tools 2024-09-14T06:13:18.253 INFO:teuthology.orchestra.run.smithi039.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T06:13:18.253 INFO:teuthology.orchestra.run.smithi039.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T06:13:18.253 INFO:teuthology.orchestra.run.smithi039.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T06:13:18.253 INFO:teuthology.orchestra.run.smithi039.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T06:13:18.254 INFO:teuthology.orchestra.run.smithi039.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T06:13:18.254 INFO:teuthology.orchestra.run.smithi039.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T06:13:18.254 INFO:teuthology.orchestra.run.smithi039.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T06:13:18.254 INFO:teuthology.orchestra.run.smithi039.stdout:The following additional packages will be installed: 2024-09-14T06:13:18.255 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-14T06:13:18.255 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-14T06:13:18.255 INFO:teuthology.orchestra.run.smithi039.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-14T06:13:18.255 INFO:teuthology.orchestra.run.smithi039.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-14T06:13:18.255 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-14T06:13:18.256 INFO:teuthology.orchestra.run.smithi039.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-14T06:13:18.256 INFO:teuthology.orchestra.run.smithi039.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-14T06:13:18.256 INFO:teuthology.orchestra.run.smithi039.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-14T06:13:18.256 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-14T06:13:18.256 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-14T06:13:18.256 INFO:teuthology.orchestra.run.smithi039.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-14T06:13:18.256 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-14T06:13:18.256 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-14T06:13:18.257 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-14T06:13:18.257 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-14T06:13:18.257 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-14T06:13:18.257 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-14T06:13:18.257 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-14T06:13:18.257 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-14T06:13:18.257 INFO:teuthology.orchestra.run.smithi039.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-14T06:13:18.257 INFO:teuthology.orchestra.run.smithi039.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-14T06:13:18.257 INFO:teuthology.orchestra.run.smithi039.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-14T06:13:18.257 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-14T06:13:18.257 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet zip 2024-09-14T06:13:18.258 INFO:teuthology.orchestra.run.smithi039.stdout:Suggested packages: 2024-09-14T06:13:18.258 INFO:teuthology.orchestra.run.smithi039.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-14T06:13:18.258 INFO:teuthology.orchestra.run.smithi039.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-14T06:13:18.258 INFO:teuthology.orchestra.run.smithi039.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-14T06:13:18.258 INFO:teuthology.orchestra.run.smithi039.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-14T06:13:18.258 INFO:teuthology.orchestra.run.smithi039.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-14T06:13:18.258 INFO:teuthology.orchestra.run.smithi039.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-14T06:13:18.258 INFO:teuthology.orchestra.run.smithi039.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-14T06:13:18.258 INFO:teuthology.orchestra.run.smithi039.stdout:Recommended packages: 2024-09-14T06:13:18.258 INFO:teuthology.orchestra.run.smithi039.stdout: btrfs-tools 2024-09-14T06:13:18.303 INFO:teuthology.orchestra.run.smithi087.stdout:The following NEW packages will be installed: 2024-09-14T06:13:18.303 INFO:teuthology.orchestra.run.smithi087.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-14T06:13:18.303 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-14T06:13:18.304 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-14T06:13:18.305 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-14T06:13:18.305 INFO:teuthology.orchestra.run.smithi087.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-14T06:13:18.305 INFO:teuthology.orchestra.run.smithi087.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-14T06:13:18.305 INFO:teuthology.orchestra.run.smithi087.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-14T06:13:18.305 INFO:teuthology.orchestra.run.smithi087.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-14T06:13:18.305 INFO:teuthology.orchestra.run.smithi087.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-14T06:13:18.305 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-14T06:13:18.305 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-14T06:13:18.305 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-14T06:13:18.305 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-14T06:13:18.305 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-14T06:13:18.305 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-14T06:13:18.305 INFO:teuthology.orchestra.run.smithi087.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-14T06:13:18.306 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-14T06:13:18.306 INFO:teuthology.orchestra.run.smithi087.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-14T06:13:18.306 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-14T06:13:18.306 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-14T06:13:18.306 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-14T06:13:18.306 INFO:teuthology.orchestra.run.smithi087.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-14T06:13:18.306 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-14T06:13:18.306 INFO:teuthology.orchestra.run.smithi087.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-14T06:13:18.306 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-14T06:13:18.306 INFO:teuthology.orchestra.run.smithi087.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-14T06:13:18.364 INFO:teuthology.orchestra.run.smithi039.stdout:The following NEW packages will be installed: 2024-09-14T06:13:18.364 INFO:teuthology.orchestra.run.smithi039.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-14T06:13:18.364 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-14T06:13:18.364 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-14T06:13:18.364 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-14T06:13:18.364 INFO:teuthology.orchestra.run.smithi039.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-14T06:13:18.365 INFO:teuthology.orchestra.run.smithi039.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-14T06:13:18.366 INFO:teuthology.orchestra.run.smithi039.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-14T06:13:18.366 INFO:teuthology.orchestra.run.smithi039.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-14T06:13:18.366 INFO:teuthology.orchestra.run.smithi039.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-14T06:13:18.366 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-14T06:13:18.366 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-14T06:13:18.366 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-14T06:13:18.366 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-14T06:13:18.366 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-14T06:13:18.366 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-14T06:13:18.366 INFO:teuthology.orchestra.run.smithi039.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-14T06:13:18.366 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-14T06:13:18.366 INFO:teuthology.orchestra.run.smithi039.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-14T06:13:18.366 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-14T06:13:18.367 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-14T06:13:18.367 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-14T06:13:18.367 INFO:teuthology.orchestra.run.smithi039.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-14T06:13:18.367 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-14T06:13:18.367 INFO:teuthology.orchestra.run.smithi039.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-14T06:13:18.367 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-14T06:13:18.367 INFO:teuthology.orchestra.run.smithi039.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-14T06:13:18.448 INFO:teuthology.orchestra.run.smithi087.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-14T06:13:18.448 INFO:teuthology.orchestra.run.smithi087.stdout:Need to get 251 MB of archives. 2024-09-14T06:13:18.448 INFO:teuthology.orchestra.run.smithi087.stdout:After this operation, 934 MB of additional disk space will be used. 2024-09-14T06:13:18.448 INFO:teuthology.orchestra.run.smithi087.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4775-g7f24e589-1jammy [3,631 kB] 2024-09-14T06:13:18.494 INFO:teuthology.orchestra.run.smithi087.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-14T06:13:18.510 INFO:teuthology.orchestra.run.smithi039.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-14T06:13:18.510 INFO:teuthology.orchestra.run.smithi039.stdout:Need to get 251 MB of archives. 2024-09-14T06:13:18.510 INFO:teuthology.orchestra.run.smithi039.stdout:After this operation, 934 MB of additional disk space will be used. 2024-09-14T06:13:18.510 INFO:teuthology.orchestra.run.smithi039.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4775-g7f24e589-1jammy [3,631 kB] 2024-09-14T06:13:18.555 INFO:teuthology.orchestra.run.smithi039.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-14T06:13:18.633 INFO:teuthology.orchestra.run.smithi087.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4775-g7f24e589-1jammy [3,330 kB] 2024-09-14T06:13:18.689 INFO:teuthology.orchestra.run.smithi087.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4775-g7f24e589-1jammy [722 kB] 2024-09-14T06:13:18.703 INFO:teuthology.orchestra.run.smithi087.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4775-g7f24e589-1jammy [356 kB] 2024-09-14T06:13:18.709 INFO:teuthology.orchestra.run.smithi087.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4775-g7f24e589-1jammy [32.8 kB] 2024-09-14T06:13:18.709 INFO:teuthology.orchestra.run.smithi087.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4775-g7f24e589-1jammy [180 kB] 2024-09-14T06:13:18.712 INFO:teuthology.orchestra.run.smithi087.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4775-g7f24e589-1jammy [71.2 kB] 2024-09-14T06:13:18.714 INFO:teuthology.orchestra.run.smithi087.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4775-g7f24e589-1jammy [338 kB] 2024-09-14T06:13:18.719 INFO:teuthology.orchestra.run.smithi087.stdout:Get:10 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4775-g7f24e589-1jammy [7,984 kB] 2024-09-14T06:13:18.884 INFO:teuthology.orchestra.run.smithi087.stdout:Get:11 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4775-g7f24e589-1jammy [112 kB] 2024-09-14T06:13:18.886 INFO:teuthology.orchestra.run.smithi087.stdout:Get:12 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4775-g7f24e589-1jammy [473 kB] 2024-09-14T06:13:18.893 INFO:teuthology.orchestra.run.smithi087.stdout:Get:13 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4775-g7f24e589-1jammy [28.3 MB] 2024-09-14T06:13:18.919 INFO:teuthology.orchestra.run.smithi087.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-14T06:13:18.931 INFO:teuthology.orchestra.run.smithi087.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-14T06:13:18.946 INFO:teuthology.orchestra.run.smithi039.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4775-g7f24e589-1jammy [3,330 kB] 2024-09-14T06:13:18.978 INFO:teuthology.orchestra.run.smithi039.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-14T06:13:18.990 INFO:teuthology.orchestra.run.smithi039.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-14T06:13:19.013 INFO:teuthology.orchestra.run.smithi087.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-14T06:13:19.071 INFO:teuthology.orchestra.run.smithi039.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-14T06:13:19.243 INFO:teuthology.orchestra.run.smithi087.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-14T06:13:19.254 INFO:teuthology.orchestra.run.smithi087.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-14T06:13:19.273 INFO:teuthology.orchestra.run.smithi039.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4775-g7f24e589-1jammy [722 kB] 2024-09-14T06:13:19.286 INFO:teuthology.orchestra.run.smithi087.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-14T06:13:19.294 INFO:teuthology.orchestra.run.smithi087.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-14T06:13:19.297 INFO:teuthology.orchestra.run.smithi087.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-14T06:13:19.298 INFO:teuthology.orchestra.run.smithi087.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-14T06:13:19.299 INFO:teuthology.orchestra.run.smithi087.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-14T06:13:19.301 INFO:teuthology.orchestra.run.smithi039.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-14T06:13:19.312 INFO:teuthology.orchestra.run.smithi039.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-14T06:13:19.316 INFO:teuthology.orchestra.run.smithi087.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-14T06:13:19.321 INFO:teuthology.orchestra.run.smithi087.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-14T06:13:19.325 INFO:teuthology.orchestra.run.smithi087.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-14T06:13:19.330 INFO:teuthology.orchestra.run.smithi039.stdout:Get:10 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4775-g7f24e589-1jammy [356 kB] 2024-09-14T06:13:19.341 INFO:teuthology.orchestra.run.smithi039.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-14T06:13:19.350 INFO:teuthology.orchestra.run.smithi039.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-14T06:13:19.352 INFO:teuthology.orchestra.run.smithi039.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-14T06:13:19.353 INFO:teuthology.orchestra.run.smithi039.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-14T06:13:19.355 INFO:teuthology.orchestra.run.smithi039.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-14T06:13:19.372 INFO:teuthology.orchestra.run.smithi039.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-14T06:13:19.376 INFO:teuthology.orchestra.run.smithi039.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-14T06:13:19.383 INFO:teuthology.orchestra.run.smithi039.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-14T06:13:19.383 INFO:teuthology.orchestra.run.smithi039.stdout:Get:19 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4775-g7f24e589-1jammy [32.8 kB] 2024-09-14T06:13:19.385 INFO:teuthology.orchestra.run.smithi039.stdout:Get:20 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4775-g7f24e589-1jammy [180 kB] 2024-09-14T06:13:19.389 INFO:teuthology.orchestra.run.smithi039.stdout:Get:21 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4775-g7f24e589-1jammy [71.2 kB] 2024-09-14T06:13:19.391 INFO:teuthology.orchestra.run.smithi039.stdout:Get:22 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4775-g7f24e589-1jammy [338 kB] 2024-09-14T06:13:19.407 INFO:teuthology.orchestra.run.smithi087.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-14T06:13:19.407 INFO:teuthology.orchestra.run.smithi087.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-14T06:13:19.411 INFO:teuthology.orchestra.run.smithi087.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-14T06:13:19.415 INFO:teuthology.orchestra.run.smithi087.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-14T06:13:19.419 INFO:teuthology.orchestra.run.smithi087.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-14T06:13:19.420 INFO:teuthology.orchestra.run.smithi087.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-14T06:13:19.421 INFO:teuthology.orchestra.run.smithi087.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-14T06:13:19.424 INFO:teuthology.orchestra.run.smithi087.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-14T06:13:19.424 INFO:teuthology.orchestra.run.smithi087.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-14T06:13:19.443 INFO:teuthology.orchestra.run.smithi039.stdout:Get:23 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4775-g7f24e589-1jammy [7,984 kB] 2024-09-14T06:13:19.463 INFO:teuthology.orchestra.run.smithi039.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-14T06:13:19.463 INFO:teuthology.orchestra.run.smithi039.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-14T06:13:19.468 INFO:teuthology.orchestra.run.smithi039.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-14T06:13:19.473 INFO:teuthology.orchestra.run.smithi039.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-14T06:13:19.477 INFO:teuthology.orchestra.run.smithi039.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-14T06:13:19.478 INFO:teuthology.orchestra.run.smithi039.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-14T06:13:19.479 INFO:teuthology.orchestra.run.smithi039.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-14T06:13:19.481 INFO:teuthology.orchestra.run.smithi039.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-14T06:13:19.482 INFO:teuthology.orchestra.run.smithi039.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-14T06:13:19.488 INFO:teuthology.orchestra.run.smithi087.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-14T06:13:19.509 INFO:teuthology.orchestra.run.smithi087.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-14T06:13:19.510 INFO:teuthology.orchestra.run.smithi087.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-14T06:13:19.510 INFO:teuthology.orchestra.run.smithi087.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-14T06:13:19.546 INFO:teuthology.orchestra.run.smithi039.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-14T06:13:19.563 INFO:teuthology.orchestra.run.smithi087.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-14T06:13:19.564 INFO:teuthology.orchestra.run.smithi087.stdout:Get:41 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4775-g7f24e589-1jammy [5,206 kB] 2024-09-14T06:13:19.565 INFO:teuthology.orchestra.run.smithi039.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-14T06:13:19.566 INFO:teuthology.orchestra.run.smithi039.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-14T06:13:19.567 INFO:teuthology.orchestra.run.smithi039.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-14T06:13:19.568 INFO:teuthology.orchestra.run.smithi039.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-14T06:13:19.569 INFO:teuthology.orchestra.run.smithi087.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-14T06:13:19.570 INFO:teuthology.orchestra.run.smithi087.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-14T06:13:19.571 INFO:teuthology.orchestra.run.smithi087.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-14T06:13:19.572 INFO:teuthology.orchestra.run.smithi087.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-14T06:13:19.573 INFO:teuthology.orchestra.run.smithi039.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-14T06:13:19.573 INFO:teuthology.orchestra.run.smithi087.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-14T06:13:19.575 INFO:teuthology.orchestra.run.smithi039.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-14T06:13:19.628 INFO:teuthology.orchestra.run.smithi039.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-14T06:13:19.629 INFO:teuthology.orchestra.run.smithi039.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-14T06:13:19.631 INFO:teuthology.orchestra.run.smithi039.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-14T06:13:19.635 INFO:teuthology.orchestra.run.smithi087.stdout:Get:47 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4775-g7f24e589-1jammy [251 kB] 2024-09-14T06:13:19.638 INFO:teuthology.orchestra.run.smithi087.stdout:Get:48 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4775-g7f24e589-1jammy [124 kB] 2024-09-14T06:13:19.639 INFO:teuthology.orchestra.run.smithi087.stdout:Get:49 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4775-g7f24e589-1jammy [1,561 kB] 2024-09-14T06:13:19.651 INFO:teuthology.orchestra.run.smithi087.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-14T06:13:19.652 INFO:teuthology.orchestra.run.smithi087.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-14T06:13:19.654 INFO:teuthology.orchestra.run.smithi087.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-14T06:13:19.655 INFO:teuthology.orchestra.run.smithi087.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-14T06:13:19.661 INFO:teuthology.orchestra.run.smithi087.stdout:Get:54 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4775-g7f24e589-1jammy [6,311 kB] 2024-09-14T06:13:19.709 INFO:teuthology.orchestra.run.smithi039.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-14T06:13:19.709 INFO:teuthology.orchestra.run.smithi039.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-14T06:13:19.712 INFO:teuthology.orchestra.run.smithi039.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-14T06:13:19.714 INFO:teuthology.orchestra.run.smithi039.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-14T06:13:19.715 INFO:teuthology.orchestra.run.smithi039.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-14T06:13:19.726 INFO:teuthology.orchestra.run.smithi039.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-14T06:13:19.733 INFO:teuthology.orchestra.run.smithi087.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-14T06:13:19.744 INFO:teuthology.orchestra.run.smithi087.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-14T06:13:19.744 INFO:teuthology.orchestra.run.smithi087.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-14T06:13:19.745 INFO:teuthology.orchestra.run.smithi087.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-14T06:13:19.746 INFO:teuthology.orchestra.run.smithi087.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-14T06:13:19.749 INFO:teuthology.orchestra.run.smithi087.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-14T06:13:19.790 INFO:teuthology.orchestra.run.smithi039.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-14T06:13:19.791 INFO:teuthology.orchestra.run.smithi039.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-14T06:13:19.792 INFO:teuthology.orchestra.run.smithi039.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-14T06:13:19.795 INFO:teuthology.orchestra.run.smithi039.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-14T06:13:19.799 INFO:teuthology.orchestra.run.smithi087.stdout:Get:61 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4775-g7f24e589-1jammy [23.1 MB] 2024-09-14T06:13:19.814 INFO:teuthology.orchestra.run.smithi087.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-14T06:13:19.816 INFO:teuthology.orchestra.run.smithi087.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-14T06:13:19.819 INFO:teuthology.orchestra.run.smithi087.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-14T06:13:19.820 INFO:teuthology.orchestra.run.smithi087.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-14T06:13:19.825 INFO:teuthology.orchestra.run.smithi039.stdout:Get:53 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4775-g7f24e589-1jammy [112 kB] 2024-09-14T06:13:19.828 INFO:teuthology.orchestra.run.smithi039.stdout:Get:54 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4775-g7f24e589-1jammy [473 kB] 2024-09-14T06:13:19.856 INFO:teuthology.orchestra.run.smithi039.stdout:Get:55 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4775-g7f24e589-1jammy [28.3 MB] 2024-09-14T06:13:19.871 INFO:teuthology.orchestra.run.smithi039.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-14T06:13:19.873 INFO:teuthology.orchestra.run.smithi039.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-14T06:13:19.875 INFO:teuthology.orchestra.run.smithi039.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-14T06:13:19.877 INFO:teuthology.orchestra.run.smithi039.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-14T06:13:19.882 INFO:teuthology.orchestra.run.smithi039.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-14T06:13:19.883 INFO:teuthology.orchestra.run.smithi039.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-14T06:13:19.896 INFO:teuthology.orchestra.run.smithi087.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-14T06:13:19.897 INFO:teuthology.orchestra.run.smithi087.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-14T06:13:19.898 INFO:teuthology.orchestra.run.smithi087.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-14T06:13:19.903 INFO:teuthology.orchestra.run.smithi087.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-14T06:13:19.953 INFO:teuthology.orchestra.run.smithi039.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-14T06:13:19.958 INFO:teuthology.orchestra.run.smithi039.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-14T06:13:20.067 INFO:teuthology.orchestra.run.smithi087.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-14T06:13:20.068 INFO:teuthology.orchestra.run.smithi087.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-14T06:13:20.071 INFO:teuthology.orchestra.run.smithi087.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-14T06:13:20.072 INFO:teuthology.orchestra.run.smithi087.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-14T06:13:20.074 INFO:teuthology.orchestra.run.smithi087.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-14T06:13:20.108 INFO:teuthology.orchestra.run.smithi087.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-14T06:13:20.121 INFO:teuthology.orchestra.run.smithi039.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-14T06:13:20.122 INFO:teuthology.orchestra.run.smithi039.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-14T06:13:20.125 INFO:teuthology.orchestra.run.smithi039.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-14T06:13:20.126 INFO:teuthology.orchestra.run.smithi039.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-14T06:13:20.128 INFO:teuthology.orchestra.run.smithi039.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-14T06:13:20.142 INFO:teuthology.orchestra.run.smithi087.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-14T06:13:20.143 INFO:teuthology.orchestra.run.smithi087.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-14T06:13:20.166 INFO:teuthology.orchestra.run.smithi039.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-14T06:13:20.168 INFO:teuthology.orchestra.run.smithi087.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-14T06:13:20.168 INFO:teuthology.orchestra.run.smithi087.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-14T06:13:20.197 INFO:teuthology.orchestra.run.smithi039.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-14T06:13:20.198 INFO:teuthology.orchestra.run.smithi039.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-14T06:13:20.228 INFO:teuthology.orchestra.run.smithi039.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-14T06:13:20.228 INFO:teuthology.orchestra.run.smithi039.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-14T06:13:20.234 INFO:teuthology.orchestra.run.smithi087.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-14T06:13:20.235 INFO:teuthology.orchestra.run.smithi087.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-14T06:13:20.235 INFO:teuthology.orchestra.run.smithi087.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-14T06:13:20.237 INFO:teuthology.orchestra.run.smithi087.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-14T06:13:20.237 INFO:teuthology.orchestra.run.smithi087.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-14T06:13:20.238 INFO:teuthology.orchestra.run.smithi087.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-14T06:13:20.246 INFO:teuthology.orchestra.run.smithi087.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-14T06:13:20.287 INFO:teuthology.orchestra.run.smithi039.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-14T06:13:20.288 INFO:teuthology.orchestra.run.smithi039.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-14T06:13:20.289 INFO:teuthology.orchestra.run.smithi039.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-14T06:13:20.290 INFO:teuthology.orchestra.run.smithi039.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-14T06:13:20.291 INFO:teuthology.orchestra.run.smithi039.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-14T06:13:20.292 INFO:teuthology.orchestra.run.smithi039.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-14T06:13:20.299 INFO:teuthology.orchestra.run.smithi039.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-14T06:13:20.315 INFO:teuthology.orchestra.run.smithi087.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-14T06:13:20.319 INFO:teuthology.orchestra.run.smithi087.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-14T06:13:20.321 INFO:teuthology.orchestra.run.smithi087.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-14T06:13:20.369 INFO:teuthology.orchestra.run.smithi039.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-14T06:13:20.372 INFO:teuthology.orchestra.run.smithi039.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-14T06:13:20.373 INFO:teuthology.orchestra.run.smithi039.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-14T06:13:20.406 INFO:teuthology.orchestra.run.smithi087.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-14T06:13:20.407 INFO:teuthology.orchestra.run.smithi087.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4775-g7f24e589-1jammy [14.1 kB] 2024-09-14T06:13:20.408 INFO:teuthology.orchestra.run.smithi087.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4775-g7f24e589-1jammy [880 kB] 2024-09-14T06:13:20.412 INFO:teuthology.orchestra.run.smithi087.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-14T06:13:20.414 INFO:teuthology.orchestra.run.smithi087.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-14T06:13:20.415 INFO:teuthology.orchestra.run.smithi087.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-14T06:13:20.421 INFO:teuthology.orchestra.run.smithi087.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4775-g7f24e589-1jammy [2,452 kB] 2024-09-14T06:13:20.427 INFO:teuthology.orchestra.run.smithi087.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-14T06:13:20.428 INFO:teuthology.orchestra.run.smithi087.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-14T06:13:20.436 INFO:teuthology.orchestra.run.smithi087.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-14T06:13:20.452 INFO:teuthology.orchestra.run.smithi039.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-14T06:13:20.456 INFO:teuthology.orchestra.run.smithi039.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-14T06:13:20.457 INFO:teuthology.orchestra.run.smithi087.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4775-g7f24e589-1jammy [812 kB] 2024-09-14T06:13:20.459 INFO:teuthology.orchestra.run.smithi039.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-14T06:13:20.460 INFO:teuthology.orchestra.run.smithi039.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-14T06:13:20.466 INFO:teuthology.orchestra.run.smithi087.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4775-g7f24e589-1jammy [160 kB] 2024-09-14T06:13:20.468 INFO:teuthology.orchestra.run.smithi087.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4775-g7f24e589-1jammy [61.4 MB] 2024-09-14T06:13:20.471 INFO:teuthology.orchestra.run.smithi039.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-14T06:13:20.472 INFO:teuthology.orchestra.run.smithi039.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-14T06:13:20.478 INFO:teuthology.orchestra.run.smithi087.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-14T06:13:20.480 INFO:teuthology.orchestra.run.smithi087.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-14T06:13:20.480 INFO:teuthology.orchestra.run.smithi039.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-14T06:13:20.532 INFO:teuthology.orchestra.run.smithi039.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-14T06:13:20.533 INFO:teuthology.orchestra.run.smithi039.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-14T06:13:20.534 INFO:teuthology.orchestra.run.smithi039.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-14T06:13:20.560 INFO:teuthology.orchestra.run.smithi087.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-14T06:13:20.564 INFO:teuthology.orchestra.run.smithi087.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-14T06:13:20.566 INFO:teuthology.orchestra.run.smithi087.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-14T06:13:20.584 INFO:teuthology.orchestra.run.smithi087.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-14T06:13:20.584 INFO:teuthology.orchestra.run.smithi087.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-14T06:13:20.585 INFO:teuthology.orchestra.run.smithi087.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-14T06:13:20.590 INFO:teuthology.orchestra.run.smithi087.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-14T06:13:20.592 INFO:teuthology.orchestra.run.smithi087.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-14T06:13:20.613 INFO:teuthology.orchestra.run.smithi039.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-14T06:13:20.615 INFO:teuthology.orchestra.run.smithi039.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-14T06:13:20.632 INFO:teuthology.orchestra.run.smithi039.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-14T06:13:20.633 INFO:teuthology.orchestra.run.smithi039.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-14T06:13:20.633 INFO:teuthology.orchestra.run.smithi039.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-14T06:13:20.639 INFO:teuthology.orchestra.run.smithi039.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-14T06:13:20.640 INFO:teuthology.orchestra.run.smithi039.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-14T06:13:21.197 INFO:teuthology.orchestra.run.smithi039.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4775-g7f24e589-1jammy [5,206 kB] 2024-09-14T06:13:21.307 INFO:teuthology.orchestra.run.smithi039.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4775-g7f24e589-1jammy [251 kB] 2024-09-14T06:13:21.313 INFO:teuthology.orchestra.run.smithi039.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4775-g7f24e589-1jammy [124 kB] 2024-09-14T06:13:21.315 INFO:teuthology.orchestra.run.smithi039.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4775-g7f24e589-1jammy [1,561 kB] 2024-09-14T06:13:21.398 INFO:teuthology.orchestra.run.smithi039.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4775-g7f24e589-1jammy [6,311 kB] 2024-09-14T06:13:21.631 INFO:teuthology.orchestra.run.smithi039.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4775-g7f24e589-1jammy [23.1 MB] 2024-09-14T06:13:22.476 INFO:teuthology.orchestra.run.smithi087.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4775-g7f24e589-1jammy [8,625 kB] 2024-09-14T06:13:22.497 INFO:teuthology.orchestra.run.smithi039.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4775-g7f24e589-1jammy [14.1 kB] 2024-09-14T06:13:22.498 INFO:teuthology.orchestra.run.smithi039.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4775-g7f24e589-1jammy [880 kB] 2024-09-14T06:13:22.514 INFO:teuthology.orchestra.run.smithi039.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4775-g7f24e589-1jammy [2,452 kB] 2024-09-14T06:13:22.554 INFO:teuthology.orchestra.run.smithi039.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4775-g7f24e589-1jammy [812 kB] 2024-09-14T06:13:22.564 INFO:teuthology.orchestra.run.smithi039.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4775-g7f24e589-1jammy [160 kB] 2024-09-14T06:13:22.566 INFO:teuthology.orchestra.run.smithi039.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4775-g7f24e589-1jammy [61.4 MB] 2024-09-14T06:13:22.634 INFO:teuthology.orchestra.run.smithi087.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4775-g7f24e589-1jammy [14.2 kB] 2024-09-14T06:13:22.635 INFO:teuthology.orchestra.run.smithi087.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4775-g7f24e589-1jammy [55.8 MB] 2024-09-14T06:13:24.391 INFO:teuthology.orchestra.run.smithi087.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4775-g7f24e589-1jammy [130 kB] 2024-09-14T06:13:24.465 INFO:teuthology.orchestra.run.smithi087.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4775-g7f24e589-1jammy [39.0 kB] 2024-09-14T06:13:24.466 INFO:teuthology.orchestra.run.smithi087.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4775-g7f24e589-1jammy [15.9 MB] 2024-09-14T06:13:24.900 INFO:teuthology.orchestra.run.smithi087.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4775-g7f24e589-1jammy [91.3 kB] 2024-09-14T06:13:24.943 INFO:teuthology.orchestra.run.smithi039.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4775-g7f24e589-1jammy [8,625 kB] 2024-09-14T06:13:25.054 INFO:teuthology.orchestra.run.smithi039.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4775-g7f24e589-1jammy [14.2 kB] 2024-09-14T06:13:25.055 INFO:teuthology.orchestra.run.smithi039.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4775-g7f24e589-1jammy [55.8 MB] 2024-09-14T06:13:25.485 INFO:teuthology.orchestra.run.smithi087.stdout:Fetched 251 MB in 7s (38.1 MB/s) 2024-09-14T06:13:25.655 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-14T06:13:25.754 INFO:teuthology.orchestra.run.smithi087.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-14T06:13:25.759 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-14T06:13:25.845 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T06:13:26.165 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-14T06:13:26.192 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-14T06:13:26.230 INFO:teuthology.orchestra.run.smithi039.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4775-g7f24e589-1jammy [130 kB] 2024-09-14T06:13:26.231 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T06:13:26.233 INFO:teuthology.orchestra.run.smithi039.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4775-g7f24e589-1jammy [39.0 kB] 2024-09-14T06:13:26.234 INFO:teuthology.orchestra.run.smithi039.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4775-g7f24e589-1jammy [15.9 MB] 2024-09-14T06:13:26.534 INFO:teuthology.orchestra.run.smithi039.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4775-g7f24e589-1jammy [91.3 kB] 2024-09-14T06:13:26.543 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-14T06:13:26.569 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-14T06:13:26.609 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T06:13:27.046 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-14T06:13:27.073 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T06:13:27.120 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T06:13:27.129 INFO:teuthology.orchestra.run.smithi039.stdout:Fetched 251 MB in 8s (30.8 MB/s) 2024-09-14T06:13:27.297 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-14T06:13:27.401 INFO:teuthology.orchestra.run.smithi039.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-14T06:13:27.407 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-14T06:13:27.499 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-14T06:13:27.526 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T06:13:27.557 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T06:13:27.604 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T06:13:27.886 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-14T06:13:27.912 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T06:13:27.952 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T06:13:27.957 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-14T06:13:27.984 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-14T06:13:28.024 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T06:13:28.294 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-14T06:13:28.297 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-14T06:13:28.321 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-14T06:13:28.323 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-14T06:13:28.352 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T06:13:28.364 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T06:13:28.666 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package librados2. 2024-09-14T06:13:28.693 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../007-librados2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:28.733 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking librados2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:28.764 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-14T06:13:28.791 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T06:13:28.839 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T06:13:29.177 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libnbd0. 2024-09-14T06:13:29.204 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-14T06:13:29.244 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-14T06:13:29.250 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-14T06:13:29.277 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T06:13:29.317 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T06:13:29.572 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package librbd1. 2024-09-14T06:13:29.598 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../009-librbd1_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:29.603 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-14T06:13:29.630 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T06:13:29.638 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking librbd1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:29.661 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T06:13:30.015 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-14T06:13:30.042 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-14T06:13:30.081 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T06:13:30.116 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libcephfs2. 2024-09-14T06:13:30.143 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:30.183 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libcephfs2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:30.401 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package librados2. 2024-09-14T06:13:30.428 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../007-librados2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:30.443 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-rados. 2024-09-14T06:13:30.467 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking librados2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:30.469 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:30.501 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-rados (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:30.813 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-14T06:13:30.839 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T06:13:30.879 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-ceph-argparse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:30.912 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libnbd0. 2024-09-14T06:13:30.939 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-14T06:13:30.978 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-14T06:13:31.132 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-cephfs. 2024-09-14T06:13:31.147 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:31.190 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-cephfs (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:31.256 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package librbd1. 2024-09-14T06:13:31.283 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../009-librbd1_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:31.323 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking librbd1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:31.792 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libcephfs2. 2024-09-14T06:13:31.818 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-14T06:13:31.819 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:31.844 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T06:13:31.859 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libcephfs2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:31.884 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:32.136 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-rados. 2024-09-14T06:13:32.163 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:32.203 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-rados (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:32.212 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-14T06:13:32.239 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-14T06:13:32.279 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T06:13:32.481 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-14T06:13:32.497 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T06:13:32.532 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-ceph-argparse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:32.582 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-prettytable. 2024-09-14T06:13:32.608 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-14T06:13:32.648 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-14T06:13:32.749 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-cephfs. 2024-09-14T06:13:32.765 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:32.808 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-cephfs (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:32.934 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-rbd. 2024-09-14T06:13:32.961 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:33.001 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-rbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:33.152 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-14T06:13:33.179 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T06:13:33.219 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:33.345 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-14T06:13:33.372 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-14T06:13:33.412 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T06:13:33.530 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-14T06:13:33.557 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-14T06:13:33.597 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T06:13:33.689 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-14T06:13:33.704 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-14T06:13:33.739 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T06:13:33.891 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-prettytable. 2024-09-14T06:13:33.918 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-14T06:13:33.958 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-14T06:13:34.017 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-14T06:13:34.032 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-14T06:13:34.067 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T06:13:34.168 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-rbd. 2024-09-14T06:13:34.185 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:34.218 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-rbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:34.395 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package lua5.1. 2024-09-14T06:13:34.422 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-14T06:13:34.462 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-14T06:13:34.563 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-14T06:13:34.590 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-14T06:13:34.630 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T06:13:34.773 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package lua-any. 2024-09-14T06:13:34.788 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-14T06:13:34.840 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-14T06:13:34.915 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-14T06:13:34.932 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-14T06:13:34.976 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T06:13:35.059 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package zip. 2024-09-14T06:13:35.074 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-14T06:13:35.109 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking zip (3.0-12build2) ... 2024-09-14T06:13:35.276 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-14T06:13:35.293 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-14T06:13:35.326 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T06:13:35.404 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package unzip. 2024-09-14T06:13:35.430 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-14T06:13:35.470 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-14T06:13:35.621 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package lua5.1. 2024-09-14T06:13:35.648 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-14T06:13:35.688 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-14T06:13:35.861 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package luarocks. 2024-09-14T06:13:35.886 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-14T06:13:35.926 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-14T06:13:36.008 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package lua-any. 2024-09-14T06:13:36.024 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-14T06:13:36.066 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-14T06:13:36.252 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package zip. 2024-09-14T06:13:36.267 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package librgw2. 2024-09-14T06:13:36.267 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-14T06:13:36.282 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../026-librgw2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:36.302 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking zip (3.0-12build2) ... 2024-09-14T06:13:36.334 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking librgw2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:36.605 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package unzip. 2024-09-14T06:13:36.632 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-14T06:13:36.671 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-14T06:13:36.970 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-rgw. 2024-09-14T06:13:36.985 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:37.028 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-rgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:37.364 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-14T06:13:37.391 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-14T06:13:37.431 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T06:13:37.568 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package luarocks. 2024-09-14T06:13:37.595 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-14T06:13:37.634 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-14T06:13:37.717 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libradosstriper1. 2024-09-14T06:13:37.744 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:37.792 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libradosstriper1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:37.992 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package librgw2. 2024-09-14T06:13:38.019 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../026-librgw2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:38.050 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking librgw2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:38.237 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-common. 2024-09-14T06:13:38.264 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:38.303 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:38.635 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-rgw. 2024-09-14T06:13:38.652 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:38.693 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-rgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:39.021 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-14T06:13:39.049 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-14T06:13:39.088 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T06:13:39.391 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libradosstriper1. 2024-09-14T06:13:39.418 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:39.466 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libradosstriper1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:39.767 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-base. 2024-09-14T06:13:39.786 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-common. 2024-09-14T06:13:39.793 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:39.813 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:39.842 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-base (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:39.852 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:40.438 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-14T06:13:40.464 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-14T06:13:40.504 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-14T06:13:40.807 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-cheroot. 2024-09-14T06:13:40.833 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-14T06:13:40.874 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T06:13:41.193 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-14T06:13:41.209 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-14T06:13:41.243 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-14T06:13:41.441 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-base. 2024-09-14T06:13:41.468 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:41.517 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-base (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:41.530 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-14T06:13:41.556 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-14T06:13:41.596 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-14T06:13:42.099 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-14T06:13:42.103 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-14T06:13:42.125 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-14T06:13:42.130 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-14T06:13:42.165 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-14T06:13:42.169 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-14T06:13:42.431 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-cheroot. 2024-09-14T06:13:42.447 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-14T06:13:42.469 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-tempora. 2024-09-14T06:13:42.481 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T06:13:42.495 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-14T06:13:42.535 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-14T06:13:42.769 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-14T06:13:42.781 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-14T06:13:42.817 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-14T06:13:42.838 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-portend. 2024-09-14T06:13:42.864 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-14T06:13:42.904 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-14T06:13:43.120 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-14T06:13:43.147 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-14T06:13:43.187 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-14T06:13:43.207 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-14T06:13:43.234 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-14T06:13:43.274 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-14T06:13:43.490 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-14T06:13:43.517 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-14T06:13:43.553 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-14T06:13:43.556 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-14T06:13:43.578 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-14T06:13:43.610 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-14T06:13:43.834 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-tempora. 2024-09-14T06:13:43.861 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-14T06:13:43.892 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-14T06:13:43.972 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-natsort. 2024-09-14T06:13:43.999 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-14T06:13:44.039 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-14T06:13:44.170 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-portend. 2024-09-14T06:13:44.197 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-14T06:13:44.236 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-14T06:13:44.317 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-logutils. 2024-09-14T06:13:44.343 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-14T06:13:44.383 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-14T06:13:44.541 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-14T06:13:44.569 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-14T06:13:44.616 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-14T06:13:44.686 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-14T06:13:44.713 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-14T06:13:44.753 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-14T06:13:44.861 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-14T06:13:44.877 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-14T06:13:44.919 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-14T06:13:45.055 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-mako. 2024-09-14T06:13:45.082 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-14T06:13:45.122 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T06:13:45.323 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-natsort. 2024-09-14T06:13:45.350 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-14T06:13:45.389 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-14T06:13:45.450 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-14T06:13:45.477 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-14T06:13:45.517 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-14T06:13:45.660 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-logutils. 2024-09-14T06:13:45.676 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-14T06:13:45.710 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-14T06:13:45.778 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-14T06:13:45.804 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-14T06:13:45.845 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-14T06:13:45.996 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-14T06:13:46.023 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-14T06:13:46.062 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-14T06:13:46.147 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-webob. 2024-09-14T06:13:46.174 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-14T06:13:46.214 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T06:13:46.365 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-mako. 2024-09-14T06:13:46.392 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-14T06:13:46.432 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T06:13:46.575 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-waitress. 2024-09-14T06:13:46.602 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-14T06:13:46.645 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T06:13:46.777 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-14T06:13:46.805 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-14T06:13:46.844 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-14T06:13:46.962 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-tempita. 2024-09-14T06:13:46.988 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-14T06:13:47.028 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T06:13:47.088 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-14T06:13:47.105 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-14T06:13:47.147 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-14T06:13:47.289 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-paste. 2024-09-14T06:13:47.316 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-14T06:13:47.347 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T06:13:47.441 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-webob. 2024-09-14T06:13:47.460 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-14T06:13:47.508 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T06:13:47.701 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-14T06:13:47.716 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-14T06:13:47.759 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T06:13:48.028 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-14T06:13:48.044 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-14T06:13:48.044 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-waitress. 2024-09-14T06:13:48.072 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-14T06:13:48.078 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-14T06:13:48.115 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T06:13:48.381 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-webtest. 2024-09-14T06:13:48.408 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-14T06:13:48.431 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-tempita. 2024-09-14T06:13:48.448 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-14T06:13:48.458 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-14T06:13:48.497 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T06:13:48.734 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pecan. 2024-09-14T06:13:48.759 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-paste. 2024-09-14T06:13:48.760 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-14T06:13:48.775 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-14T06:13:48.800 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T06:13:48.809 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T06:13:49.129 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-14T06:13:49.156 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-14T06:13:49.164 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-certifi. 2024-09-14T06:13:49.190 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-14T06:13:49.195 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T06:13:49.230 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-14T06:13:49.448 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-14T06:13:49.464 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-14T06:13:49.502 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-14T06:13:49.558 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-idna. 2024-09-14T06:13:49.584 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-14T06:13:49.624 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T06:13:49.810 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-webtest. 2024-09-14T06:13:49.837 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-14T06:13:49.876 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-14T06:13:49.936 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-urllib3. 2024-09-14T06:13:49.962 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-14T06:13:50.002 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T06:13:50.204 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pecan. 2024-09-14T06:13:50.232 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-14T06:13:50.271 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T06:13:50.306 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-requests. 2024-09-14T06:13:50.332 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-14T06:13:50.372 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T06:13:50.660 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-certifi. 2024-09-14T06:13:50.674 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-14T06:13:50.683 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-14T06:13:50.710 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-14T06:13:50.718 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-14T06:13:50.750 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T06:13:51.004 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-idna. 2024-09-14T06:13:51.021 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-14T06:13:51.036 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-14T06:13:51.052 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T06:13:51.054 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T06:13:51.111 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:51.366 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-urllib3. 2024-09-14T06:13:51.393 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-14T06:13:51.432 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T06:13:51.508 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-14T06:13:51.535 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:51.660 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:52.186 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-requests. 2024-09-14T06:13:52.214 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-14T06:13:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T06:13:52.903 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-14T06:13:52.914 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-14T06:13:52.930 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-14T06:13:52.941 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-14T06:13:52.969 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-14T06:13:52.981 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T06:13:53.272 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-openssl. 2024-09-14T06:13:53.276 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-14T06:13:53.299 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-14T06:13:53.303 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T06:13:53.339 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-14T06:13:53.342 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:53.675 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr. 2024-09-14T06:13:53.702 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:53.706 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-14T06:13:53.733 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:53.741 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:53.772 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:54.083 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-14T06:13:54.095 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mon. 2024-09-14T06:13:54.111 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-14T06:13:54.121 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:54.153 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mon (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:54.158 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-14T06:13:54.462 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-openssl. 2024-09-14T06:13:54.489 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-14T06:13:54.528 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-14T06:13:54.790 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr. 2024-09-14T06:13:54.798 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-osd. 2024-09-14T06:13:54.817 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:54.825 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:54.856 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:54.881 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-osd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:55.251 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mon. 2024-09-14T06:13:55.278 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:55.318 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mon (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:55.926 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph. 2024-09-14T06:13:55.930 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-osd. 2024-09-14T06:13:55.941 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../067-ceph_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:55.957 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:55.984 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:55.996 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-osd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:56.304 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-fuse. 2024-09-14T06:13:56.331 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:56.370 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:56.740 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mds. 2024-09-14T06:13:56.767 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:56.806 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mds (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:57.124 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph. 2024-09-14T06:13:57.141 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../067-ceph_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:57.174 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:57.209 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package cephadm. 2024-09-14T06:13:57.236 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../070-cephadm_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:57.276 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:57.494 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-fuse. 2024-09-14T06:13:57.521 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:57.560 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:57.570 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-14T06:13:57.586 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-14T06:13:57.620 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-14T06:13:57.923 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-14T06:13:57.939 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mds. 2024-09-14T06:13:57.950 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T06:13:57.966 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:57.989 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T06:13:58.005 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mds (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:58.358 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package cephadm. 2024-09-14T06:13:58.385 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../070-cephadm_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:13:58.416 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:13:58.728 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-14T06:13:58.755 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-14T06:13:58.794 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-14T06:13:59.003 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-babel. 2024-09-14T06:13:59.032 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T06:13:59.070 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T06:13:59.072 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-14T06:13:59.088 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T06:13:59.131 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T06:13:59.389 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-jinja2. 2024-09-14T06:13:59.417 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-14T06:13:59.456 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T06:13:59.758 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-14T06:13:59.774 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T06:13:59.808 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:00.127 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-babel. 2024-09-14T06:14:00.128 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-14T06:14:00.152 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-14T06:14:00.155 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T06:14:00.186 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-14T06:14:00.193 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T06:14:00.489 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-routes. 2024-09-14T06:14:00.513 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-jinja2. 2024-09-14T06:14:00.516 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-14T06:14:00.541 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-14T06:14:00.547 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T06:14:00.580 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T06:14:00.859 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-14T06:14:00.886 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T06:14:00.925 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:00.987 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-14T06:14:01.014 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T06:14:01.058 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:01.404 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-14T06:14:01.431 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-14T06:14:01.470 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-14T06:14:01.773 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-routes. 2024-09-14T06:14:01.800 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-14T06:14:01.839 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T06:14:02.176 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-14T06:14:02.204 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T06:14:02.242 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:03.772 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-14T06:14:03.789 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-14T06:14:03.830 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T06:14:04.325 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-joblib. 2024-09-14T06:14:04.353 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-14T06:14:04.392 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T06:14:04.754 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-14T06:14:04.781 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-14T06:14:04.820 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-14T06:14:05.123 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-sklearn. 2024-09-14T06:14:05.150 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-14T06:14:05.190 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-14T06:14:05.208 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-14T06:14:05.240 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T06:14:05.275 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T06:14:05.702 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-joblib. 2024-09-14T06:14:05.730 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-14T06:14:05.768 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T06:14:06.059 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-14T06:14:06.087 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T06:14:06.126 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:06.130 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-14T06:14:06.158 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-14T06:14:06.197 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-14T06:14:06.500 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-sklearn. 2024-09-14T06:14:06.528 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-14T06:14:06.652 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T06:14:07.054 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-cachetools. 2024-09-14T06:14:07.082 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-14T06:14:07.112 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-14T06:14:07.353 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-14T06:14:07.382 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T06:14:07.398 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-14T06:14:07.419 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:07.426 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-14T06:14:07.465 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-14T06:14:07.793 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-14T06:14:07.821 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-14T06:14:07.860 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T06:14:08.172 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-rsa. 2024-09-14T06:14:08.199 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-14T06:14:08.263 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-14T06:14:08.407 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-cachetools. 2024-09-14T06:14:08.435 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-14T06:14:08.465 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-14T06:14:08.575 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-google-auth. 2024-09-14T06:14:08.603 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-14T06:14:08.641 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-14T06:14:08.776 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-14T06:14:08.804 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-14T06:14:08.843 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-14T06:14:08.953 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-14T06:14:08.981 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-14T06:14:09.037 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T06:14:09.163 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-14T06:14:09.191 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-14T06:14:09.229 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T06:14:09.340 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-websocket. 2024-09-14T06:14:09.367 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-14T06:14:09.406 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-14T06:14:09.541 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-rsa. 2024-09-14T06:14:09.569 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-14T06:14:09.607 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-14T06:14:09.726 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-14T06:14:09.753 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-14T06:14:09.792 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T06:14:09.895 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-google-auth. 2024-09-14T06:14:09.923 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-14T06:14:09.953 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-14T06:14:10.248 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-14T06:14:10.277 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-14T06:14:10.314 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T06:14:10.473 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-14T06:14:10.489 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T06:14:10.531 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:10.593 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-websocket. 2024-09-14T06:14:10.621 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-14T06:14:10.668 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-14T06:14:10.859 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-14T06:14:10.886 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-14T06:14:10.925 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T06:14:10.946 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-14T06:14:10.975 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-14T06:14:11.012 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T06:14:11.270 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-14T06:14:11.298 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T06:14:11.337 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T06:14:11.573 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package jq. 2024-09-14T06:14:11.600 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T06:14:11.639 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-14T06:14:11.727 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-14T06:14:11.756 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T06:14:11.793 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:11.875 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package socat. 2024-09-14T06:14:11.891 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-14T06:14:11.933 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-14T06:14:12.105 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-14T06:14:12.133 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-14T06:14:12.171 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T06:14:12.220 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package xmlstarlet. 2024-09-14T06:14:12.244 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-14T06:14:12.278 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-14T06:14:12.483 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-14T06:14:12.511 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T06:14:12.549 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T06:14:12.649 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-test. 2024-09-14T06:14:12.677 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:14:12.715 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-test (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:12.812 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package jq. 2024-09-14T06:14:12.839 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T06:14:12.877 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-14T06:14:13.130 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package socat. 2024-09-14T06:14:13.158 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-14T06:14:13.196 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-14T06:14:13.558 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package xmlstarlet. 2024-09-14T06:14:13.587 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-14T06:14:13.616 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-14T06:14:13.962 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-test. 2024-09-14T06:14:13.979 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:14:14.021 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-test (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:15.228 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-volume. 2024-09-14T06:14:15.256 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T06:14:15.295 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-volume (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:15.574 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-14T06:14:15.590 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:14:15.623 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libcephfs-dev (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:15.935 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-14T06:14:15.963 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-14T06:14:16.001 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T06:14:16.364 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-14T06:14:16.392 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-14T06:14:16.430 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-14T06:14:16.584 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-volume. 2024-09-14T06:14:16.612 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T06:14:16.650 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-volume (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:16.801 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package nvme-cli. 2024-09-14T06:14:16.829 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-14T06:14:16.868 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T06:14:16.963 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-14T06:14:16.980 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:14:17.021 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libcephfs-dev (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:17.266 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package pkg-config. 2024-09-14T06:14:17.294 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-14T06:14:17.315 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-14T06:14:17.332 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-14T06:14:17.333 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T06:14:17.365 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T06:14:17.594 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-14T06:14:17.622 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-14T06:14:17.660 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-14T06:14:17.712 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-14T06:14:17.741 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-14T06:14:17.778 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-14T06:14:18.080 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-14T06:14:18.108 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-14T06:14:18.146 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-14T06:14:18.149 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package nvme-cli. 2024-09-14T06:14:18.178 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-14T06:14:18.216 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T06:14:18.449 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pastescript. 2024-09-14T06:14:18.477 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-14T06:14:18.515 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-14T06:14:18.621 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package pkg-config. 2024-09-14T06:14:18.650 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-14T06:14:18.688 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T06:14:18.835 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pluggy. 2024-09-14T06:14:18.863 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-14T06:14:18.902 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-14T06:14:18.933 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-14T06:14:18.950 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-14T06:14:18.991 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-14T06:14:19.213 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-psutil. 2024-09-14T06:14:19.241 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-14T06:14:19.279 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-14T06:14:19.378 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-14T06:14:19.407 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-14T06:14:19.444 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-14T06:14:19.624 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-py. 2024-09-14T06:14:19.652 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-14T06:14:19.699 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-14T06:14:19.747 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pastescript. 2024-09-14T06:14:19.776 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-14T06:14:19.813 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-14T06:14:20.036 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pygments. 2024-09-14T06:14:20.064 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-14T06:14:20.102 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T06:14:20.125 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pluggy. 2024-09-14T06:14:20.154 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-14T06:14:20.192 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-14T06:14:20.470 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-psutil. 2024-09-14T06:14:20.487 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-14T06:14:20.522 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-14T06:14:20.528 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-14T06:14:20.551 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-14T06:14:20.580 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-14T06:14:20.856 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-py. 2024-09-14T06:14:20.885 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-14T06:14:20.891 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-toml. 2024-09-14T06:14:20.919 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-14T06:14:20.922 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-14T06:14:20.958 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-14T06:14:21.210 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pygments. 2024-09-14T06:14:21.227 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-14T06:14:21.276 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T06:14:21.294 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pytest. 2024-09-14T06:14:21.322 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-14T06:14:21.360 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T06:14:21.689 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-simplejson. 2024-09-14T06:14:21.717 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-14T06:14:21.739 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-14T06:14:21.755 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-14T06:14:21.768 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-14T06:14:21.805 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-14T06:14:22.024 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-14T06:14:22.041 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-14T06:14:22.083 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-14T06:14:22.108 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-toml. 2024-09-14T06:14:22.137 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-14T06:14:22.174 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-14T06:14:22.480 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pytest. 2024-09-14T06:14:22.508 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-14T06:14:22.546 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T06:14:22.786 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package radosgw. 2024-09-14T06:14:22.815 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../117-radosgw_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:14:22.853 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking radosgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:22.866 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-simplejson. 2024-09-14T06:14:22.883 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-14T06:14:22.924 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-14T06:14:23.227 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-14T06:14:23.245 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-14T06:14:23.276 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-14T06:14:23.739 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package rbd-fuse. 2024-09-14T06:14:23.755 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:14:23.805 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking rbd-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:23.939 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package radosgw. 2024-09-14T06:14:23.956 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../117-radosgw_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:14:23.989 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking radosgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:24.093 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-14T06:14:24.335 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T06:14:24.461 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T06:14:24.729 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-14T06:14:24.900 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package rbd-fuse. 2024-09-14T06:14:24.917 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T06:14:24.958 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking rbd-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:25.059 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-14T06:14:25.255 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-14T06:14:25.471 INFO:teuthology.orchestra.run.smithi087.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-14T06:14:25.554 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:25.605 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T06:14:25.731 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T06:14:25.996 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-14T06:14:26.063 INFO:teuthology.orchestra.run.smithi087.stdout:Adding system user cephadm....done 2024-09-14T06:14:26.314 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-14T06:14:26.318 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T06:14:26.611 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-14T06:14:26.735 INFO:teuthology.orchestra.run.smithi039.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-14T06:14:26.832 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:26.871 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-14T06:14:27.014 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-14T06:14:27.273 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-14T06:14:27.365 INFO:teuthology.orchestra.run.smithi039.stdout:Adding system user cephadm....done 2024-09-14T06:14:27.489 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T06:14:27.524 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T06:14:27.625 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-14T06:14:27.757 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-14T06:14:27.927 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T06:14:28.016 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-14T06:14:28.125 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-14T06:14:28.287 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-14T06:14:28.368 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-14T06:14:28.546 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-14T06:14:28.636 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T06:14:28.762 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-14T06:14:28.831 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-14T06:14:29.072 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T06:14:29.182 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-14T06:14:29.474 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-14T06:14:29.716 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-14T06:14:29.758 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-14T06:14:29.993 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-14T06:14:30.020 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-14T06:14:30.236 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-14T06:14:30.278 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-ceph-argparse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:30.496 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-14T06:14:30.537 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T06:14:30.662 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-14T06:14:30.738 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-14T06:14:30.788 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T06:14:30.923 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T06:14:30.980 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-ceph-argparse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:31.048 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-14T06:14:31.232 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T06:14:31.291 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-14T06:14:31.358 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-14T06:14:31.483 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T06:14:31.609 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T06:14:31.735 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-14T06:14:31.812 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-14T06:14:31.839 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-14T06:14:31.911 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T06:14:31.995 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-14T06:14:32.037 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-14T06:14:32.381 INFO:teuthology.orchestra.run.smithi039.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-14T06:14:32.396 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T06:14:32.417 INFO:teuthology.orchestra.run.smithi039.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-14T06:14:32.498 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T06:14:32.522 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-14T06:14:32.599 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-14T06:14:32.798 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T06:14:32.967 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T06:14:33.066 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-14T06:14:33.093 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-14T06:14:33.350 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up zip (3.0-12build2) ... 2024-09-14T06:14:33.360 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T06:14:33.476 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T06:14:33.628 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-14T06:14:33.954 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up zip (3.0-12build2) ... 2024-09-14T06:14:34.055 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T06:14:34.169 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T06:14:34.420 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T06:14:34.521 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-14T06:14:34.664 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T06:14:34.757 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T06:14:34.974 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-14T06:14:35.033 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T06:14:35.159 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-14T06:14:35.285 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T06:14:35.358 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-14T06:14:35.586 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-14T06:14:35.592 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T06:14:35.977 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T06:14:35.979 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-14T06:14:36.230 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T06:14:36.261 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T06:14:36.605 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-14T06:14:36.606 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T06:14:36.856 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-14T06:14:36.899 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T06:14:36.982 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:37.250 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-14T06:14:37.299 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T06:14:37.510 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-14T06:14:37.601 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T06:14:37.636 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:37.937 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T06:14:38.246 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T06:14:38.773 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T06:14:38.991 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T06:14:39.100 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-14T06:14:39.360 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T06:14:39.486 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-14T06:14:39.544 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T06:14:39.611 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T06:14:39.804 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T06:14:39.905 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-14T06:14:39.938 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-14T06:14:40.172 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-14T06:14:40.206 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T06:14:40.307 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-14T06:14:40.416 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T06:14:40.423 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T06:14:40.541 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-14T06:14:40.717 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-14T06:14:40.834 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-14T06:14:40.985 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-14T06:14:41.085 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-14T06:14:41.244 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T06:14:41.344 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-14T06:14:41.371 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-14T06:14:41.595 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-14T06:14:41.697 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-14T06:14:41.846 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-14T06:14:41.939 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-14T06:14:42.105 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T06:14:42.199 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-14T06:14:42.240 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-14T06:14:42.450 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-14T06:14:42.508 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T06:14:42.633 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T06:14:42.718 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-14T06:14:42.926 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T06:14:42.993 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T06:14:43.119 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-14T06:14:43.227 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-14T06:14:43.404 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T06:14:43.487 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T06:14:43.530 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T06:14:43.612 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-14T06:14:43.738 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T06:14:43.831 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T06:14:43.836 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-14T06:14:44.065 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T06:14:44.124 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-14T06:14:44.191 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T06:14:44.375 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T06:14:44.501 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-14T06:14:44.576 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-14T06:14:44.627 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T06:14:44.716 INFO:teuthology.orchestra.run.smithi039.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-14T06:14:44.852 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-14T06:14:44.953 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T06:14:44.978 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-14T06:14:45.054 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T06:14:45.323 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T06:14:45.430 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T06:14:45.456 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-14T06:14:45.707 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-14T06:14:45.756 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T06:14:45.816 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-14T06:14:46.049 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T06:14:46.076 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T06:14:46.202 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T06:14:46.316 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-14T06:14:46.425 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-14T06:14:46.536 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T06:14:46.701 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-14T06:14:46.829 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T06:14:47.063 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T06:14:47.096 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-14T06:14:47.214 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-14T06:14:47.322 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-14T06:14:47.465 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-14T06:14:47.699 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-14T06:14:47.818 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T06:14:47.984 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T06:14:48.086 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-14T06:14:48.319 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T06:14:48.463 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-14T06:14:48.472 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up librados2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:48.580 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:48.714 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-14T06:14:48.764 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T06:14:49.050 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libcephfs2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:49.091 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T06:14:49.176 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libradosstriper1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:49.218 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up librados2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:49.302 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-14T06:14:49.344 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:49.470 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-14T06:14:49.587 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up librbd1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:49.713 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr-modules-core (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:49.814 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libcephfs2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:49.855 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:49.932 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libradosstriper1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:50.033 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-14T06:14:50.089 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T06:14:50.089 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T06:14:50.318 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up librbd1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:50.444 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr-modules-core (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:50.545 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:50.590 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libcephfs-dev (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:50.691 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-rados (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:50.756 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T06:14:50.756 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T06:14:50.792 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up librgw2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:50.918 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T06:14:51.268 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libcephfs-dev (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:51.439 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-rados (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:51.573 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up librgw2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:51.725 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T06:14:52.163 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-rbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:52.289 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up rbd-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:52.415 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-rgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:52.541 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-cephfs (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:52.667 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:52.946 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-rbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:53.055 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up rbd-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:53.181 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-rgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:53.184 INFO:teuthology.orchestra.run.smithi087.stdout:Adding group ceph....done 2024-09-14T06:14:53.307 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-cephfs (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:53.433 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:53.541 INFO:teuthology.orchestra.run.smithi087.stdout:Adding system user ceph....done 2024-09-14T06:14:53.696 INFO:teuthology.orchestra.run.smithi087.stdout:Setting system user ceph properties....done 2024-09-14T06:14:53.712 INFO:teuthology.orchestra.run.smithi087.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-14T06:14:53.851 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-14T06:14:53.967 INFO:teuthology.orchestra.run.smithi039.stdout:Adding group ceph....done 2024-09-14T06:14:54.145 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-14T06:14:54.269 INFO:teuthology.orchestra.run.smithi039.stdout:Adding system user ceph....done 2024-09-14T06:14:54.387 INFO:teuthology.orchestra.run.smithi039.stdout:Setting system user ceph properties....done 2024-09-14T06:14:54.401 INFO:teuthology.orchestra.run.smithi039.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-14T06:14:54.553 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-14T06:14:54.871 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-14T06:14:54.904 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-test (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:55.036 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up radosgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:55.325 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-test (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:55.451 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up radosgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:55.506 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T06:14:55.506 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T06:14:55.931 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T06:14:55.931 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T06:14:55.992 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-base (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:56.334 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-14T06:14:56.411 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-base (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:56.749 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-14T06:14:56.835 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mds (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:57.049 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T06:14:57.049 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T06:14:57.221 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mds (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:57.462 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T06:14:57.462 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T06:14:57.521 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:57.781 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T06:14:57.781 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T06:14:57.990 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:58.220 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T06:14:58.220 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T06:14:58.245 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-osd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:58.608 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T06:14:58.609 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T06:14:58.707 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-osd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:59.014 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T06:14:59.014 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T06:14:59.090 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:59.207 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:59.342 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mon (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:59.501 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:59.560 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T06:14:59.560 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T06:14:59.627 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:14:59.778 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mon (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:15:00.017 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T06:15:00.018 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T06:15:00.034 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr-cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:15:00.185 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:15:00.311 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr-dashboard (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:15:00.462 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-volume (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:15:00.479 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr-cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:15:00.613 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:15:00.723 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr-dashboard (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:15:00.815 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-14T06:15:00.849 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-volume (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T06:15:00.932 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-14T06:15:01.068 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-14T06:15:01.193 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-14T06:15:01.302 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-14T06:15:01.480 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-14T06:15:01.851 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-14T06:15:02.495 INFO:teuthology.orchestra.run.smithi087.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-14T06:15:02.501 DEBUG:teuthology.parallel:result is None 2024-09-14T06:15:02.622 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-14T06:15:03.263 INFO:teuthology.orchestra.run.smithi039.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-14T06:15:03.268 DEBUG:teuthology.parallel:result is None 2024-09-14T06:15:03.268 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T06:15:03.410 DEBUG:teuthology.orchestra.run.smithi039:> dpkg-query -W -f '${Version}' ceph 2024-09-14T06:15:03.429 INFO:teuthology.orchestra.run.smithi039.stdout:19.3.0-4775-g7f24e589-1jammy 2024-09-14T06:15:03.430 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4775-g7f24e589-1jammy 2024-09-14T06:15:03.430 INFO:teuthology.task.install:The correct ceph version 19.3.0-4775-g7f24e589-1jammy is installed. 2024-09-14T06:15:03.432 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T06:15:03.567 DEBUG:teuthology.orchestra.run.smithi087:> dpkg-query -W -f '${Version}' ceph 2024-09-14T06:15:03.588 INFO:teuthology.orchestra.run.smithi087.stdout:19.3.0-4775-g7f24e589-1jammy 2024-09-14T06:15:03.588 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4775-g7f24e589-1jammy 2024-09-14T06:15:03.588 INFO:teuthology.task.install:The correct ceph version 19.3.0-4775-g7f24e589-1jammy is installed. 2024-09-14T06:15:03.590 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-14T06:15:03.590 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:03.590 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T06:15:03.605 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:03.605 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T06:15:03.648 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-14T06:15:03.648 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:03.648 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T06:15:03.662 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T06:15:03.715 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:03.716 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T06:15:03.728 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T06:15:03.781 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-14T06:15:03.782 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:03.782 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T06:15:03.796 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T06:15:03.851 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:03.851 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T06:15:03.864 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T06:15:03.918 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-14T06:15:03.919 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:03.919 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T06:15:03.932 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T06:15:03.987 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:03.987 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T06:15:04.000 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T06:15:04.054 INFO:teuthology.run_tasks:Running task ceph... 2024-09-14T06:15:04.147 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-14T06:15:04.147 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 777 /var/log/ceph 2024-09-14T06:15:04.150 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 777 /var/log/ceph 2024-09-14T06:15:04.163 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-14T06:15:04.163 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T06:15:04.205 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T06:15:04.218 INFO:tasks.ceph:Creating extra log directories... 2024-09-14T06:15:04.218 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T06:15:04.261 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T06:15:04.283 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-14T06:15:04.284 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd clone copy on read': True}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-14T06:15:04.284 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900364', 'branch': 'main', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_copy_on_read}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7900364', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd clone copy on read': True}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 950, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 5839, 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47', 'sleep_before_teardown': 0, 'subset': '99/128', 'suite': 'rbd', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a', 'targets': {'smithi039.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIAEO7C4Kjm91cr+Ms+0e8C1iVA2b5X93iIILPW4ZZM99aYEvjXv/95k/fhT5bpE9yyK3msPDvwpzxgE76SUTdY=', 'smithi087.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCuSgZ9zpMJxTBT0CKGFS/T1G6vXMeITTyGGw6bcFJyMHd717i2wRejZOl0clEbUZpGCvL6vmlzQSB/b9XhGG+g='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_20:16:17', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-14T06:15:04.284 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T06:15:04.317 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T06:15:04.324 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T06:15:04.365 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T06:15:04.379 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:04.380 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T06:15:04.424 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T06:15:04.424 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_1 2024-09-14T06:15:04.472 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T06:15:04.472 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:15:04.472 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 857 Links: 1 2024-09-14T06:15:04.472 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:15:04.472 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-14 06:09:36.016196518 +0000 2024-09-14T06:15:04.472 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-14 06:09:35.764198770 +0000 2024-09-14T06:15:04.473 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-14 06:09:35.764198770 +0000 2024-09-14T06:15:04.473 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2024-09-14T06:15:04.473 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T06:15:04.526 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-14T06:15:04.527 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-14T06:15:04.527 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000553962 s, 924 kB/s 2024-09-14T06:15:04.528 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T06:15:04.577 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_2 2024-09-14T06:15:04.628 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T06:15:04.628 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:15:04.628 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 875 Links: 1 2024-09-14T06:15:04.628 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:15:04.628 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-14 06:09:36.460192554 +0000 2024-09-14T06:15:04.628 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-14 06:09:36.224194660 +0000 2024-09-14T06:15:04.629 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-14 06:09:36.224194660 +0000 2024-09-14T06:15:04.629 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2024-09-14T06:15:04.629 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T06:15:04.683 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-14T06:15:04.683 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-14T06:15:04.683 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000483853 s, 1.1 MB/s 2024-09-14T06:15:04.685 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T06:15:04.733 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_3 2024-09-14T06:15:04.780 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T06:15:04.780 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:15:04.780 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 891 Links: 1 2024-09-14T06:15:04.780 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:15:04.780 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-14 06:09:36.964188052 +0000 2024-09-14T06:15:04.780 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-14 06:09:36.720190230 +0000 2024-09-14T06:15:04.780 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-14 06:09:36.720190230 +0000 2024-09-14T06:15:04.780 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2024-09-14T06:15:04.780 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T06:15:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-14T06:15:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-14T06:15:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000547837 s, 935 kB/s 2024-09-14T06:15:04.836 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T06:15:04.885 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_4 2024-09-14T06:15:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T06:15:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:15:04.932 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 901 Links: 1 2024-09-14T06:15:04.932 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:15:04.932 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-14 06:09:37.492183335 +0000 2024-09-14T06:15:04.932 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-14 06:09:37.220185765 +0000 2024-09-14T06:15:04.932 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-14 06:09:37.220185765 +0000 2024-09-14T06:15:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2024-09-14T06:15:04.932 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T06:15:04.986 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-14T06:15:04.986 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-14T06:15:04.986 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000433926 s, 1.2 MB/s 2024-09-14T06:15:04.988 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T06:15:05.037 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T06:15:05.037 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:05.037 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T06:15:05.043 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T06:15:05.043 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_1 2024-09-14T06:15:05.091 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T06:15:05.091 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:15:05.091 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 887 Links: 1 2024-09-14T06:15:05.092 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:15:05.092 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-09-14 06:09:32.881016300 +0000 2024-09-14T06:15:05.092 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-09-14 06:09:32.653018337 +0000 2024-09-14T06:15:05.092 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-09-14 06:09:32.653018337 +0000 2024-09-14T06:15:05.092 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2024-09-14T06:15:05.092 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T06:15:05.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-09-14T06:15:05.146 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-09-14T06:15:05.146 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.00046402 s, 1.1 MB/s 2024-09-14T06:15:05.147 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T06:15:05.197 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_2 2024-09-14T06:15:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T06:15:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:15:05.243 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-09-14T06:15:05.243 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:15:05.243 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-09-14 06:09:33.345012156 +0000 2024-09-14T06:15:05.243 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-09-14 06:09:33.105014300 +0000 2024-09-14T06:15:05.243 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-09-14 06:09:33.105014300 +0000 2024-09-14T06:15:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2024-09-14T06:15:05.244 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T06:15:05.297 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-09-14T06:15:05.297 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-09-14T06:15:05.297 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000444749 s, 1.2 MB/s 2024-09-14T06:15:05.298 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T06:15:05.348 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_3 2024-09-14T06:15:05.395 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T06:15:05.395 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:15:05.395 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 916 Links: 1 2024-09-14T06:15:05.395 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:15:05.395 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-09-14 06:09:33.849007655 +0000 2024-09-14T06:15:05.395 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-09-14 06:09:33.565010191 +0000 2024-09-14T06:15:05.395 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-09-14 06:09:33.565010191 +0000 2024-09-14T06:15:05.395 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2024-09-14T06:15:05.396 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T06:15:05.449 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-09-14T06:15:05.449 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-09-14T06:15:05.449 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000452929 s, 1.1 MB/s 2024-09-14T06:15:05.452 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T06:15:05.500 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_4 2024-09-14T06:15:05.547 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T06:15:05.547 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:15:05.547 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 937 Links: 1 2024-09-14T06:15:05.547 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:15:05.547 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-09-14 06:09:34.349003189 +0000 2024-09-14T06:15:05.547 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-09-14 06:09:34.117005261 +0000 2024-09-14T06:15:05.548 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-09-14 06:09:34.117005261 +0000 2024-09-14T06:15:05.548 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2024-09-14T06:15:05.548 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T06:15:05.601 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-09-14T06:15:05.601 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-09-14T06:15:05.601 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000395664 s, 1.3 MB/s 2024-09-14T06:15:05.603 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T06:15:05.652 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T06:15:05.652 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi039.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi087.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-14T06:15:05.652 INFO:tasks.ceph:Generating config... 2024-09-14T06:15:05.654 INFO:tasks.ceph:[client] rbd cache = True 2024-09-14T06:15:05.654 INFO:tasks.ceph:[client] rbd clone copy on read = True 2024-09-14T06:15:05.654 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-14T06:15:05.654 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-14T06:15:05.654 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-14T06:15:05.654 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-14T06:15:05.654 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-14T06:15:05.654 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-14T06:15:05.654 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-14T06:15:05.654 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-14T06:15:05.654 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-14T06:15:05.655 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2024-09-14T06:15:05.655 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-14T06:15:05.655 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-14T06:15:05.655 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-14T06:15:05.655 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-14T06:15:05.655 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-14T06:15:05.655 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-14T06:15:05.655 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-14T06:15:05.655 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-14T06:15:05.656 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-14T06:15:05.656 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-14T06:15:05.656 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-14T06:15:05.656 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-14T06:15:05.656 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-14T06:15:05.656 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-14T06:15:05.656 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-14T06:15:05.656 INFO:tasks.ceph:Setting up mon.a... 2024-09-14T06:15:05.656 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-14T06:15:05.695 INFO:teuthology.orchestra.run.smithi039.stdout:creating /etc/ceph/ceph.keyring 2024-09-14T06:15:05.705 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-14T06:15:05.784 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T06:15:05.839 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.39'), ('mon.c', '[v2:172.21.15.39:3301,v1:172.21.15.39:6790]'), ('mon.b', '172.21.15.87')] 2024-09-14T06:15:05.840 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790],172.21.15.87', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd clone copy on read': True}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-14T06:15:05.840 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:05.840 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T06:15:05.888 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.39 --addv c '[v2:172.21.15.39:3301,v1:172.21.15.39:6790]' --add b 172.21.15.87 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:15:05.975 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:15:05.975 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: generated fsid 038563b1-268f-461f-b6a6-4b4b41ad0236 2024-09-14T06:15:05.975 INFO:teuthology.orchestra.run.smithi039.stdout:setting min_mon_release = quincy 2024-09-14T06:15:05.975 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 0 2024-09-14T06:15:05.975 INFO:teuthology.orchestra.run.smithi039.stdout:fsid 038563b1-268f-461f-b6a6-4b4b41ad0236 2024-09-14T06:15:05.975 INFO:teuthology.orchestra.run.smithi039.stdout:last_changed 2024-09-14T06:15:05.975716+0000 2024-09-14T06:15:05.976 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-09-14T06:15:05.975716+0000 2024-09-14T06:15:05.976 INFO:teuthology.orchestra.run.smithi039.stdout:min_mon_release 17 (quincy) 2024-09-14T06:15:05.976 INFO:teuthology.orchestra.run.smithi039.stdout:election_strategy: 1 2024-09-14T06:15:05.976 INFO:teuthology.orchestra.run.smithi039.stdout:0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-09-14T06:15:05.976 INFO:teuthology.orchestra.run.smithi039.stdout:1: [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon.b 2024-09-14T06:15:05.976 INFO:teuthology.orchestra.run.smithi039.stdout:2: [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] mon.c 2024-09-14T06:15:05.976 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-14T06:15:05.977 DEBUG:teuthology.orchestra.run.smithi039:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T06:15:05.984 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 038563b1-268f-461f-b6a6-4b4b41ad0236... 2024-09-14T06:15:05.985 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T06:15:06.029 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T06:15:06.058 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2024-09-14T06:15:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: chdir = "" 2024-09-14T06:15:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T06:15:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: auth supported = cephx 2024-09-14T06:15:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: filestore xattr use omap = true 2024-09-14T06:15:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: mon clock drift allowed = 1.000 2024-09-14T06:15:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: osd crush chooseleaf type = 0 2024-09-14T06:15:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: auth debug = true 2024-09-14T06:15:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on old message = true 2024-09-14T06:15:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on bug = true 2024-09-14T06:15:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T06:15:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: mon pg warn max object skew = 0 2024-09-14T06:15:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T06:15:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T06:15:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default size = 2 2024-09-14T06:15:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow primary affinity = true 2024-09-14T06:15:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow pg remap = true 2024-09-14T06:15:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on legacy crush tunables = false 2024-09-14T06:15:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on crush straw calc version zero = false 2024-09-14T06:15:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on no sortbitwise = false 2024-09-14T06:15:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on osd down out interval zero = false 2024-09-14T06:15:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on too few osds = false 2024-09-14T06:15:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T06:15:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T06:15:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: mon_allow_pool_size_one = true 2024-09-14T06:15:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T06:15:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: osd default data pool replay window = 5 2024-09-14T06:15:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: mon allow pool delete = true 2024-09-14T06:15:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: mon cluster log file level = debug 2024-09-14T06:15:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: debug asserts on shutdown = true 2024-09-14T06:15:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: mon health detail to clog = false 2024-09-14T06:15:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: mon host = "172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790],172.21.15.87" 2024-09-14T06:15:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: mon client directed command retry = 5 2024-09-14T06:15:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on pool no app = False 2024-09-14T06:15:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: ms inject socket failures = 5000 2024-09-14T06:15:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = 038563b1-268f-461f-b6a6-4b4b41ad0236 2024-09-14T06:15:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.061 INFO:teuthology.orchestra.run.smithi039.stdout:[osd] 2024-09-14T06:15:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: osd journal size = 100 2024-09-14T06:15:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub load threshold = 5.0 2024-09-14T06:15:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub max interval = 600 2024-09-14T06:15:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: osd mclock profile = high_recovery_ops 2024-09-14T06:15:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: osd recover clone overlap = true 2024-09-14T06:15:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: osd recovery max chunk = 1048576 2024-09-14T06:15:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug shutdown = true 2024-09-14T06:15:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug op order = true 2024-09-14T06:15:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug verify stray on activate = true 2024-09-14T06:15:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug trim objects = true 2024-09-14T06:15:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: osd open classes on start = true 2024-09-14T06:15:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug pg log writeout = true 2024-09-14T06:15:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: osd deep scrub update digest min age = 30 2024-09-14T06:15:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: osd map max advance = 10 2024-09-14T06:15:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: journal zero on create = true 2024-09-14T06:15:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: filestore ondisk finisher threads = 3 2024-09-14T06:15:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: filestore apply finisher threads = 3 2024-09-14T06:15:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: bdev debug aio = true 2024-09-14T06:15:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug misdirected ops = true 2024-09-14T06:15:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore block size = 96636764160 2024-09-14T06:15:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore compression algorithm = lz4 2024-09-14T06:15:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore compression mode = aggressive 2024-09-14T06:15:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore fsck on mount = True 2024-09-14T06:15:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluefs = 1/20 2024-09-14T06:15:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluestore = 1/20 2024-09-14T06:15:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-14T06:15:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: debug osd = 20 2024-09-14T06:15:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: debug rocksdb = 4/10 2024-09-14T06:15:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T06:15:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd full ratio = 0.9 2024-09-14T06:15:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd nearfull ratio = 0.8 2024-09-14T06:15:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: osd blocked scrub grace period = 3600 2024-09-14T06:15:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: osd failsafe full ratio = 0.95 2024-09-14T06:15:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: osd max markdown count = 1000 2024-09-14T06:15:06.067 INFO:teuthology.orchestra.run.smithi039.stdout: osd objectstore = bluestore 2024-09-14T06:15:06.067 INFO:teuthology.orchestra.run.smithi039.stdout: osd shutdown pgref assert = True 2024-09-14T06:15:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.068 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr] 2024-09-14T06:15:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-14T06:15:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: debug mgr = 20 2024-09-14T06:15:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-09-14T06:15:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-09-14T06:15:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-09-14T06:15:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-09-14T06:15:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: mgr/telemetry/nag = false 2024-09-14T06:15:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.069 INFO:teuthology.orchestra.run.smithi039.stdout:[mon] 2024-09-14T06:15:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-14T06:15:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-09-14T06:15:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: debug paxos = 20 2024-09-14T06:15:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-09-14T06:15:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: mon data avail warn = 5 2024-09-14T06:15:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: mon mgr mkfs grace = 240 2024-09-14T06:15:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-09-14T06:15:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd reporter subtree level = osd 2024-09-14T06:15:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd prime pg temp = true 2024-09-14T06:15:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-09-14T06:15:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T06:15:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T06:15:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T06:15:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T06:15:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T06:15:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T06:15:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: # 1m isn't quite enough 2024-09-14T06:15:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: mon_down_mkfs_grace = 2m 2024-09-14T06:15:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_filestore_osds = false 2024-09-14T06:15:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:06.071 INFO:teuthology.orchestra.run.smithi039.stdout:[client] 2024-09-14T06:15:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: rgw cache enabled = true 2024-09-14T06:15:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable ops log = true 2024-09-14T06:15:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable usage log = true 2024-09-14T06:15:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T06:15:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T06:15:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: rbd cache = True 2024-09-14T06:15:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: rbd clone copy on read = True 2024-09-14T06:15:06.072 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.a] 2024-09-14T06:15:06.072 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.c] 2024-09-14T06:15:06.072 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.b] 2024-09-14T06:15:06.074 INFO:teuthology.orchestra.run.smithi087.stdout:[global] 2024-09-14T06:15:06.074 INFO:teuthology.orchestra.run.smithi087.stdout: chdir = "" 2024-09-14T06:15:06.074 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T06:15:06.074 INFO:teuthology.orchestra.run.smithi087.stdout: auth supported = cephx 2024-09-14T06:15:06.074 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.074 INFO:teuthology.orchestra.run.smithi087.stdout: filestore xattr use omap = true 2024-09-14T06:15:06.074 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.074 INFO:teuthology.orchestra.run.smithi087.stdout: mon clock drift allowed = 1.000 2024-09-14T06:15:06.074 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.074 INFO:teuthology.orchestra.run.smithi087.stdout: osd crush chooseleaf type = 0 2024-09-14T06:15:06.074 INFO:teuthology.orchestra.run.smithi087.stdout: auth debug = true 2024-09-14T06:15:06.075 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.075 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on old message = true 2024-09-14T06:15:06.075 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on bug = true 2024-09-14T06:15:06.075 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.075 INFO:teuthology.orchestra.run.smithi087.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T06:15:06.075 INFO:teuthology.orchestra.run.smithi087.stdout: mon pg warn max object skew = 0 2024-09-14T06:15:06.075 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.075 INFO:teuthology.orchestra.run.smithi087.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T06:15:06.075 INFO:teuthology.orchestra.run.smithi087.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T06:15:06.075 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.075 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default size = 2 2024-09-14T06:15:06.075 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.076 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow primary affinity = true 2024-09-14T06:15:06.076 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow pg remap = true 2024-09-14T06:15:06.076 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on legacy crush tunables = false 2024-09-14T06:15:06.076 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on crush straw calc version zero = false 2024-09-14T06:15:06.076 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on no sortbitwise = false 2024-09-14T06:15:06.076 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on osd down out interval zero = false 2024-09-14T06:15:06.076 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on too few osds = false 2024-09-14T06:15:06.076 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T06:15:06.076 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T06:15:06.076 INFO:teuthology.orchestra.run.smithi087.stdout: mon_allow_pool_size_one = true 2024-09-14T06:15:06.076 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.076 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T06:15:06.076 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.077 INFO:teuthology.orchestra.run.smithi087.stdout: osd default data pool replay window = 5 2024-09-14T06:15:06.077 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.077 INFO:teuthology.orchestra.run.smithi087.stdout: mon allow pool delete = true 2024-09-14T06:15:06.077 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.077 INFO:teuthology.orchestra.run.smithi087.stdout: mon cluster log file level = debug 2024-09-14T06:15:06.077 INFO:teuthology.orchestra.run.smithi087.stdout: debug asserts on shutdown = true 2024-09-14T06:15:06.077 INFO:teuthology.orchestra.run.smithi087.stdout: mon health detail to clog = false 2024-09-14T06:15:06.077 INFO:teuthology.orchestra.run.smithi087.stdout: mon host = "172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790],172.21.15.87" 2024-09-14T06:15:06.077 INFO:teuthology.orchestra.run.smithi087.stdout: mon client directed command retry = 5 2024-09-14T06:15:06.077 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on pool no app = False 2024-09-14T06:15:06.077 INFO:teuthology.orchestra.run.smithi087.stdout: ms inject socket failures = 5000 2024-09-14T06:15:06.077 INFO:teuthology.orchestra.run.smithi087.stdout: fsid = 038563b1-268f-461f-b6a6-4b4b41ad0236 2024-09-14T06:15:06.077 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.077 INFO:teuthology.orchestra.run.smithi087.stdout:[osd] 2024-09-14T06:15:06.077 INFO:teuthology.orchestra.run.smithi087.stdout: osd journal size = 100 2024-09-14T06:15:06.078 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.078 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub load threshold = 5.0 2024-09-14T06:15:06.078 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub max interval = 600 2024-09-14T06:15:06.078 INFO:teuthology.orchestra.run.smithi087.stdout: osd mclock profile = high_recovery_ops 2024-09-14T06:15:06.078 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.078 INFO:teuthology.orchestra.run.smithi087.stdout: osd recover clone overlap = true 2024-09-14T06:15:06.078 INFO:teuthology.orchestra.run.smithi087.stdout: osd recovery max chunk = 1048576 2024-09-14T06:15:06.078 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.078 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug shutdown = true 2024-09-14T06:15:06.078 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug op order = true 2024-09-14T06:15:06.078 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug verify stray on activate = true 2024-09-14T06:15:06.079 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug trim objects = true 2024-09-14T06:15:06.079 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.079 INFO:teuthology.orchestra.run.smithi087.stdout: osd open classes on start = true 2024-09-14T06:15:06.079 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug pg log writeout = true 2024-09-14T06:15:06.079 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.079 INFO:teuthology.orchestra.run.smithi087.stdout: osd deep scrub update digest min age = 30 2024-09-14T06:15:06.079 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.079 INFO:teuthology.orchestra.run.smithi087.stdout: osd map max advance = 10 2024-09-14T06:15:06.079 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.079 INFO:teuthology.orchestra.run.smithi087.stdout: journal zero on create = true 2024-09-14T06:15:06.079 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.079 INFO:teuthology.orchestra.run.smithi087.stdout: filestore ondisk finisher threads = 3 2024-09-14T06:15:06.079 INFO:teuthology.orchestra.run.smithi087.stdout: filestore apply finisher threads = 3 2024-09-14T06:15:06.080 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.080 INFO:teuthology.orchestra.run.smithi087.stdout: bdev debug aio = true 2024-09-14T06:15:06.080 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug misdirected ops = true 2024-09-14T06:15:06.080 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore block size = 96636764160 2024-09-14T06:15:06.080 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore compression algorithm = lz4 2024-09-14T06:15:06.080 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore compression mode = aggressive 2024-09-14T06:15:06.080 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore fsck on mount = True 2024-09-14T06:15:06.080 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluefs = 1/20 2024-09-14T06:15:06.080 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluestore = 1/20 2024-09-14T06:15:06.080 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-09-14T06:15:06.080 INFO:teuthology.orchestra.run.smithi087.stdout: debug osd = 20 2024-09-14T06:15:06.080 INFO:teuthology.orchestra.run.smithi087.stdout: debug rocksdb = 4/10 2024-09-14T06:15:06.081 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T06:15:06.081 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd full ratio = 0.9 2024-09-14T06:15:06.081 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd nearfull ratio = 0.8 2024-09-14T06:15:06.081 INFO:teuthology.orchestra.run.smithi087.stdout: osd blocked scrub grace period = 3600 2024-09-14T06:15:06.081 INFO:teuthology.orchestra.run.smithi087.stdout: osd failsafe full ratio = 0.95 2024-09-14T06:15:06.081 INFO:teuthology.orchestra.run.smithi087.stdout: osd max markdown count = 1000 2024-09-14T06:15:06.081 INFO:teuthology.orchestra.run.smithi087.stdout: osd objectstore = bluestore 2024-09-14T06:15:06.081 INFO:teuthology.orchestra.run.smithi087.stdout: osd shutdown pgref assert = True 2024-09-14T06:15:06.081 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.081 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr] 2024-09-14T06:15:06.081 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-09-14T06:15:06.081 INFO:teuthology.orchestra.run.smithi087.stdout: debug mgr = 20 2024-09-14T06:15:06.081 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-09-14T06:15:06.081 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-09-14T06:15:06.082 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-09-14T06:15:06.082 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-09-14T06:15:06.082 INFO:teuthology.orchestra.run.smithi087.stdout: mgr/telemetry/nag = false 2024-09-14T06:15:06.082 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.082 INFO:teuthology.orchestra.run.smithi087.stdout:[mon] 2024-09-14T06:15:06.082 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-09-14T06:15:06.082 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-09-14T06:15:06.082 INFO:teuthology.orchestra.run.smithi087.stdout: debug paxos = 20 2024-09-14T06:15:06.082 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-09-14T06:15:06.082 INFO:teuthology.orchestra.run.smithi087.stdout: mon data avail warn = 5 2024-09-14T06:15:06.082 INFO:teuthology.orchestra.run.smithi087.stdout: mon mgr mkfs grace = 240 2024-09-14T06:15:06.082 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-09-14T06:15:06.082 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd reporter subtree level = osd 2024-09-14T06:15:06.082 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd prime pg temp = true 2024-09-14T06:15:06.082 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-09-14T06:15:06.083 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.083 INFO:teuthology.orchestra.run.smithi087.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T06:15:06.083 INFO:teuthology.orchestra.run.smithi087.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T06:15:06.083 INFO:teuthology.orchestra.run.smithi087.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T06:15:06.083 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.083 INFO:teuthology.orchestra.run.smithi087.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T06:15:06.083 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T06:15:06.083 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T06:15:06.083 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.083 INFO:teuthology.orchestra.run.smithi087.stdout: # 1m isn't quite enough 2024-09-14T06:15:06.083 INFO:teuthology.orchestra.run.smithi087.stdout: mon_down_mkfs_grace = 2m 2024-09-14T06:15:06.083 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.083 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_filestore_osds = false 2024-09-14T06:15:06.083 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-14T06:15:06.084 INFO:teuthology.orchestra.run.smithi087.stdout:[client] 2024-09-14T06:15:06.084 INFO:teuthology.orchestra.run.smithi087.stdout: rgw cache enabled = true 2024-09-14T06:15:06.084 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable ops log = true 2024-09-14T06:15:06.084 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable usage log = true 2024-09-14T06:15:06.084 INFO:teuthology.orchestra.run.smithi087.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T06:15:06.084 INFO:teuthology.orchestra.run.smithi087.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T06:15:06.084 INFO:teuthology.orchestra.run.smithi087.stdout: rbd cache = True 2024-09-14T06:15:06.084 INFO:teuthology.orchestra.run.smithi087.stdout: rbd clone copy on read = True 2024-09-14T06:15:06.084 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.a] 2024-09-14T06:15:06.084 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.c] 2024-09-14T06:15:06.084 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.b] 2024-09-14T06:15:06.085 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-14T06:15:06.085 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-14T06:15:06.166 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-14T06:15:06.166 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:06.166 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-14T06:15:06.172 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:06.172 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-14T06:15:06.219 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi039.front.sepia.ceph.com 2024-09-14T06:15:06.219 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:06.219 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T06:15:06.220 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T06:15:06.284 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:06.284 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:15:06.331 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi087.front.sepia.ceph.com 2024-09-14T06:15:06.332 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:06.332 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T06:15:06.332 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T06:15:06.356 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:06.356 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:15:06.403 INFO:tasks.ceph:Setting up mon nodes... 2024-09-14T06:15:06.403 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-14T06:15:06.404 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-14T06:15:06.449 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-14T06:15:06.452 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T06:15:06.495 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T06:15:06.502 INFO:tasks.ceph:Setting up mds nodes... 2024-09-14T06:15:06.502 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-14T06:15:06.502 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-14T06:15:06.537 INFO:teuthology.orchestra.run.smithi039.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-14T06:15:06.549 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-14T06:15:06.583 INFO:teuthology.orchestra.run.smithi087.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-14T06:15:06.594 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-14T06:15:06.594 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi039.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi087.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-14T06:15:06.595 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-14T06:15:06.608 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T06:15:06.608 INFO:tasks.ceph:role: osd.0 2024-09-14T06:15:06.609 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-14T06:15:06.609 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T06:15:06.692 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:15:06.692 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:15:06.692 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:15:06.692 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T06:15:06.693 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:15:06.693 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:15:06.693 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:15:06.693 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T06:15:06.693 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:15:06.693 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:15:07.997 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-09-14T06:15:07.999 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-09-14T06:15:07.999 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-14T06:15:08.063 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-14T06:15:08.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:15:08.117 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T06:15:08.117 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-14T06:15:08.171 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T06:15:08.171 INFO:tasks.ceph:role: osd.1 2024-09-14T06:15:08.171 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-14T06:15:08.172 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T06:15:08.255 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:15:08.255 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:15:08.255 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:15:08.255 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T06:15:08.255 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:15:08.256 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:15:08.256 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:15:08.256 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T06:15:08.256 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:15:08.256 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:15:09.556 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-09-14T06:15:09.558 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-09-14T06:15:09.558 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-14T06:15:09.618 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-14T06:15:09.669 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T06:15:09.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:15:09.670 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-14T06:15:09.728 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T06:15:09.728 INFO:tasks.ceph:role: osd.2 2024-09-14T06:15:09.729 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-14T06:15:09.729 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T06:15:09.813 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:15:09.813 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:15:09.814 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:15:09.814 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T06:15:09.814 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:15:09.814 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:15:09.814 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:15:09.814 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T06:15:09.814 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:15:09.814 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:15:11.134 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-09-14T06:15:11.136 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-09-14T06:15:11.136 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-14T06:15:11.202 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-14T06:15:11.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:15:11.350 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T06:15:11.350 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-14T06:15:11.415 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T06:15:11.415 INFO:tasks.ceph:role: osd.3 2024-09-14T06:15:11.415 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-14T06:15:11.415 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-14T06:15:11.509 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:15:11.510 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:15:11.510 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:15:11.510 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T06:15:11.510 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:15:11.510 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:15:11.510 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:15:11.510 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T06:15:11.511 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:15:11.511 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:15:12.833 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-09-14T06:15:12.835 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-09-14T06:15:12.836 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-14T06:15:12.899 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-14T06:15:12.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:15:12.957 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T06:15:12.957 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:15:13.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-14T06:15:13.093+0000 7ff31f70d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-14T06:15:13.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-14T06:15:13.093+0000 7ff31f70d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-14T06:15:13.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-14T06:15:13.093+0000 7ff31f70d7c0 -1 bdev(0x55e8537bf800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-14T06:15:13.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-14T06:15:13.093+0000 7ff31f70d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-14T06:15:16.245 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-14T06:15:16.260 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:15:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-14T06:15:16.337+0000 7f99bc48c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-14T06:15:16.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-14T06:15:16.341+0000 7f99bc48c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-14T06:15:16.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-14T06:15:16.341+0000 7f99bc48c7c0 -1 bdev(0x56219c8fd800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-14T06:15:16.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-14T06:15:16.341+0000 7f99bc48c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-14T06:15:19.499 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-14T06:15:19.556 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:15:19.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-14T06:15:19.637+0000 7f4dc2e417c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-14T06:15:19.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-14T06:15:19.637+0000 7f4dc2e417c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-14T06:15:19.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-14T06:15:19.637+0000 7f4dc2e417c0 -1 bdev(0x557090e5f800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-14T06:15:19.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-14T06:15:19.637+0000 7f4dc2e417c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-14T06:15:22.790 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-14T06:15:22.848 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:15:22.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-14T06:15:22.933+0000 7ff59170d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-14T06:15:22.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-14T06:15:22.933+0000 7ff59170d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-14T06:15:22.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-14T06:15:22.933+0000 7ff59170d7c0 -1 bdev(0x55e0b0739800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-14T06:15:22.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-14T06:15:22.933+0000 7ff59170d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-14T06:15:26.082 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-14T06:15:26.140 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-14T06:15:26.154 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T06:15:26.154 INFO:tasks.ceph:role: osd.4 2024-09-14T06:15:26.154 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi087.front.sepia.ceph.com 2024-09-14T06:15:26.155 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T06:15:26.219 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:15:26.219 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:15:26.219 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:15:26.219 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T06:15:26.219 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:15:26.219 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:15:26.219 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:15:26.219 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T06:15:26.219 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:15:26.220 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:15:26.688 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-09-14T06:15:26.690 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-09-14T06:15:26.690 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-14T06:15:26.782 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-14T06:15:26.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:15:26.793 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T06:15:26.793 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-14T06:15:26.848 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T06:15:26.848 INFO:tasks.ceph:role: osd.5 2024-09-14T06:15:26.848 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi087.front.sepia.ceph.com 2024-09-14T06:15:26.849 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T06:15:26.910 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:15:26.910 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:15:26.910 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:15:26.910 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T06:15:26.910 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:15:26.910 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:15:26.911 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:15:26.911 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T06:15:26.911 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:15:26.911 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:15:27.387 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-09-14T06:15:27.388 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-09-14T06:15:27.389 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-14T06:15:27.552 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-14T06:15:27.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:15:27.564 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T06:15:27.564 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-14T06:15:27.622 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T06:15:27.622 INFO:tasks.ceph:role: osd.6 2024-09-14T06:15:27.623 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi087.front.sepia.ceph.com 2024-09-14T06:15:27.623 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T06:15:27.686 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:15:27.687 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:15:27.687 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:15:27.687 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T06:15:27.687 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:15:27.687 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:15:27.687 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:15:27.687 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T06:15:27.687 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:15:27.687 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:15:28.167 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-09-14T06:15:28.169 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-09-14T06:15:28.170 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-14T06:15:28.297 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-14T06:15:28.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:15:28.308 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T06:15:28.308 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-14T06:15:28.361 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T06:15:28.361 INFO:tasks.ceph:role: osd.7 2024-09-14T06:15:28.361 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi087.front.sepia.ceph.com 2024-09-14T06:15:28.361 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-14T06:15:28.423 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:15:28.423 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:15:28.423 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:15:28.423 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T06:15:28.423 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:15:28.423 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:15:28.423 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:15:28.423 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T06:15:28.424 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:15:28.424 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:15:28.850 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-09-14T06:15:28.853 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-09-14T06:15:28.853 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-14T06:15:28.946 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-14T06:15:28.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:15:28.956 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T06:15:28.957 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:15:29.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:29.088+0000 7fa81feda7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-14T06:15:29.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:29.088+0000 7fa81feda7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-14T06:15:29.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:29.088+0000 7fa81feda7c0 -1 bdev(0x556213253800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-14T06:15:29.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:29.088+0000 7fa81feda7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-14T06:15:32.223 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-14T06:15:32.236 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:15:32.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:32.312+0000 7ff2050017c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-14T06:15:32.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:32.312+0000 7ff2050017c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-14T06:15:32.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:32.316+0000 7ff2050017c0 -1 bdev(0x5579f62e5800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-14T06:15:32.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:32.316+0000 7ff2050017c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-14T06:15:35.457 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-14T06:15:35.511 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:15:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:35.588+0000 7f00897c07c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-14T06:15:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:35.588+0000 7f00897c07c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-14T06:15:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:35.588+0000 7f00897c07c0 -1 bdev(0x56008c189800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-14T06:15:35.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:35.588+0000 7f00897c07c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-14T06:15:38.711 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-14T06:15:38.770 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:15:38.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:38.848+0000 7f6409ea17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-14T06:15:38.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:38.848+0000 7f6409ea17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-14T06:15:38.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:38.848+0000 7f6409ea17c0 -1 bdev(0x560f958a3800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-14T06:15:38.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:38.848+0000 7f6409ea17c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-14T06:15:41.968 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-14T06:15:42.027 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-14T06:15:42.027 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:42.027 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-14T06:15:42.042 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:42.042 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-14T06:15:42.099 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:42.099 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-14T06:15:42.155 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:42.156 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-14T06:15:42.213 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:42.213 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-14T06:15:42.268 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:42.268 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-14T06:15:42.282 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:42.282 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-14T06:15:42.337 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:42.337 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-14T06:15:42.390 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:42.390 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-14T06:15:42.445 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:42.455 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-14T06:15:42.498 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:42.498 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-14T06:15:42.504 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:42.504 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-14T06:15:42.547 INFO:tasks.ceph:Adding keys to all mons... 2024-09-14T06:15:42.547 DEBUG:teuthology.orchestra.run.smithi039:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T06:15:42.550 DEBUG:teuthology.orchestra.run.smithi087:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T06:15:42.593 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr.y] 2024-09-14T06:15:42.593 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQDqKeVm6RjZGhAALDa7LxQPGsdHfQldDg+hqA== 2024-09-14T06:15:42.593 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.0] 2024-09-14T06:15:42.593 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQDxKeVm56bSBRAAHEqsmyJAOeaT0rYqjq5M+g== 2024-09-14T06:15:42.593 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.1] 2024-09-14T06:15:42.593 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQD0KeVmA8V6FBAAdwjz6gNp95Biyd2l7xyctQ== 2024-09-14T06:15:42.593 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.2] 2024-09-14T06:15:42.593 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQD3KeVmV5YtJhAAUSs5N5syknCackAOFjoRPw== 2024-09-14T06:15:42.593 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.3] 2024-09-14T06:15:42.594 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQD6KeVmbWzNNxAAPAgu16N7gmKjlG0tJCGbxQ== 2024-09-14T06:15:42.594 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr.x] 2024-09-14T06:15:42.594 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQDqKeVmXgKZHRAA+GGM6BX5yOuWs44C0eCmtg== 2024-09-14T06:15:42.594 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.4] 2024-09-14T06:15:42.594 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQABKuVmaDeVBRAAW/cZiCRvuQ1NkO3lzI2+ag== 2024-09-14T06:15:42.594 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.5] 2024-09-14T06:15:42.594 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQAEKuVmyUMNExAAvVV9E/IgVODs2U8mTZVG7w== 2024-09-14T06:15:42.594 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.6] 2024-09-14T06:15:42.594 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQAHKuVm9XNzIxAA/SNnrT3fISnBAzGITLUilA== 2024-09-14T06:15:42.594 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.7] 2024-09-14T06:15:42.594 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQAKKuVm7MvNMhAA933JBVEmgE2fYaTqMLpjgA== 2024-09-14T06:15:42.594 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-14T06:15:42.594 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQDqKeVmF30SIBAAYaRXAEt6uC+opqtM8oR9aA== 2024-09-14T06:15:42.594 INFO:teuthology.orchestra.run.smithi039.stdout:[client.1] 2024-09-14T06:15:42.594 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQDqKeVm+d/cIhAAxBLpE6lplPYk4QLhkG651Q== 2024-09-14T06:15:42.601 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr.y] 2024-09-14T06:15:42.601 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDqKeVm6RjZGhAALDa7LxQPGsdHfQldDg+hqA== 2024-09-14T06:15:42.601 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.0] 2024-09-14T06:15:42.601 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDxKeVm56bSBRAAHEqsmyJAOeaT0rYqjq5M+g== 2024-09-14T06:15:42.601 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.1] 2024-09-14T06:15:42.601 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQD0KeVmA8V6FBAAdwjz6gNp95Biyd2l7xyctQ== 2024-09-14T06:15:42.601 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.2] 2024-09-14T06:15:42.601 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQD3KeVmV5YtJhAAUSs5N5syknCackAOFjoRPw== 2024-09-14T06:15:42.601 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.3] 2024-09-14T06:15:42.601 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQD6KeVmbWzNNxAAPAgu16N7gmKjlG0tJCGbxQ== 2024-09-14T06:15:42.602 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr.x] 2024-09-14T06:15:42.602 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDqKeVmXgKZHRAA+GGM6BX5yOuWs44C0eCmtg== 2024-09-14T06:15:42.602 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.4] 2024-09-14T06:15:42.602 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQABKuVmaDeVBRAAW/cZiCRvuQ1NkO3lzI2+ag== 2024-09-14T06:15:42.602 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.5] 2024-09-14T06:15:42.602 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQAEKuVmyUMNExAAvVV9E/IgVODs2U8mTZVG7w== 2024-09-14T06:15:42.602 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.6] 2024-09-14T06:15:42.602 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQAHKuVm9XNzIxAA/SNnrT3fISnBAzGITLUilA== 2024-09-14T06:15:42.602 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.7] 2024-09-14T06:15:42.602 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQAKKuVm7MvNMhAA933JBVEmgE2fYaTqMLpjgA== 2024-09-14T06:15:42.602 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-09-14T06:15:42.602 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDqKeVmF30SIBAAYaRXAEt6uC+opqtM8oR9aA== 2024-09-14T06:15:42.602 INFO:teuthology.orchestra.run.smithi087.stdout:[client.1] 2024-09-14T06:15:42.602 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDqKeVm+d/cIhAAxBLpE6lplPYk4QLhkG651Q== 2024-09-14T06:15:42.603 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T06:15:42.640 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T06:15:42.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:15:42.728 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:15:42.775 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:15:42.777 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:15:42.818 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:15:42.860 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:15:42.899 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:15:42.944 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:15:42.990 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T06:15:43.028 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T06:15:43.075 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:15:43.112 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:15:43.161 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:15:43.196 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:15:43.250 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:15:43.280 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:15:43.339 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:15:43.364 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:15:43.405 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T06:15:43.448 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T06:15:43.498 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T06:15:43.536 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T06:15:43.585 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-14T06:15:43.585 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-14T06:15:43.597 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T06:15:43.699 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-14T06:15:43.711 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-14T06:15:43.764 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T06:15:43.868 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-14T06:15:43.879 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-14T06:15:43.894 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T06:15:43.990 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-14T06:15:44.001 DEBUG:teuthology.orchestra.run.smithi039:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:15:44.003 DEBUG:teuthology.orchestra.run.smithi087:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:15:44.046 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-14T06:15:44.046 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-14T06:15:44.046 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-14T06:15:44.052 INFO:tasks.ceph.mon.a:Started 2024-09-14T06:15:44.052 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-14T06:15:44.052 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-14T06:15:44.055 INFO:tasks.ceph.mon.c:Started 2024-09-14T06:15:44.055 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-14T06:15:44.055 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-14T06:15:44.091 INFO:tasks.ceph.mon.b:Started 2024-09-14T06:15:44.091 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-14T06:15:44.092 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-14T06:15:44.092 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-14T06:15:44.094 INFO:tasks.ceph.mgr.y:Started 2024-09-14T06:15:44.094 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-14T06:15:44.094 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-14T06:15:44.097 INFO:tasks.ceph.mgr.x:Started 2024-09-14T06:15:44.097 DEBUG:tasks.ceph:set 0 configs 2024-09-14T06:15:44.097 INFO:tasks.ceph:Setting crush tunables to default 2024-09-14T06:15:44.097 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd crush tunables default 2024-09-14T06:15:49.618 INFO:teuthology.orchestra.run.smithi039.stderr:adjusted tunables profile to default 2024-09-14T06:15:49.638 INFO:tasks.ceph:check_enable_crimson: False 2024-09-14T06:15:49.638 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-14T06:15:49.638 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:49.638 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-14T06:15:49.649 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:49.649 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-14T06:15:49.701 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:49.701 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-14T06:15:49.753 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-14T06:15:49.753 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-14T06:15:49.809 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:49.810 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-14T06:15:49.824 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:49.824 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-14T06:15:49.878 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:49.878 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-14T06:15:49.933 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-14T06:15:49.934 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-14T06:15:49.989 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new c669ea76-ed19-4c12-9e56-d79d62846f95 0 2024-09-14T06:15:50.262 INFO:teuthology.orchestra.run.smithi087.stdout:0 2024-09-14T06:15:50.286 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b 1 2024-09-14T06:15:50.615 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-14T06:15:50.612+0000 7f9f3659c640 0 --2- 172.21.15.87:0/3819466066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f38076330 0x7f9f380ad5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-14T06:15:50.716 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-09-14T06:15:50.734 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 9bda988f-e615-4545-849b-318c408c4c42 2 2024-09-14T06:15:50.939 INFO:teuthology.orchestra.run.smithi087.stdout:2 2024-09-14T06:15:50.956 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 8ea6cd69-2ef1-42ea-b339-7394bf8f4a87 3 2024-09-14T06:15:51.164 INFO:teuthology.orchestra.run.smithi087.stdout:3 2024-09-14T06:15:51.180 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new fea45e8a-e8ea-4963-b93f-0651b84c9e74 4 2024-09-14T06:15:51.382 INFO:teuthology.orchestra.run.smithi087.stdout:4 2024-09-14T06:15:51.398 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new b601e852-a180-477f-86fc-6a213fa866f0 5 2024-09-14T06:15:51.601 INFO:teuthology.orchestra.run.smithi087.stdout:5 2024-09-14T06:15:51.617 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 1113afe0-f524-41b1-8e51-eb5164602d38 6 2024-09-14T06:15:51.820 INFO:teuthology.orchestra.run.smithi087.stdout:6 2024-09-14T06:15:51.836 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new c9162545-318b-4591-be64-cc8ace9097fe 7 2024-09-14T06:15:52.031 INFO:teuthology.orchestra.run.smithi087.stdout:7 2024-09-14T06:15:52.047 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-14T06:15:52.047 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-14T06:15:52.049 INFO:tasks.ceph.osd.0:Started 2024-09-14T06:15:52.050 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-14T06:15:52.050 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-14T06:15:52.052 INFO:tasks.ceph.osd.1:Started 2024-09-14T06:15:52.052 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-14T06:15:52.052 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-14T06:15:52.054 INFO:tasks.ceph.osd.2:Started 2024-09-14T06:15:52.055 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-14T06:15:52.055 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-14T06:15:52.057 INFO:tasks.ceph.osd.3:Started 2024-09-14T06:15:52.057 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-14T06:15:52.057 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-14T06:15:52.059 INFO:tasks.ceph.osd.4:Started 2024-09-14T06:15:52.059 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-14T06:15:52.059 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-14T06:15:52.064 INFO:tasks.ceph.osd.5:Started 2024-09-14T06:15:52.065 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-14T06:15:52.065 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-14T06:15:52.067 INFO:tasks.ceph.osd.6:Started 2024-09-14T06:15:52.067 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-14T06:15:52.067 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-14T06:15:52.069 INFO:tasks.ceph.osd.7:Started 2024-09-14T06:15:52.069 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T06:15:52.291 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:15:52.291 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":10,"fsid":"038563b1-268f-461f-b6a6-4b4b41ad0236","created":"2024-09-14T06:15:49.493035+0000","modified":"2024-09-14T06:15:52.027930+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T06:15:52.027930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c669ea76-ed19-4c12-9e56-d79d62846f95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9bda988f-e615-4545-849b-318c408c4c42","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8ea6cd69-2ef1-42ea-b339-7394bf8f4a87","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fea45e8a-e8ea-4963-b93f-0651b84c9e74","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b601e852-a180-477f-86fc-6a213fa866f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1113afe0-f524-41b1-8e51-eb5164602d38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c9162545-318b-4591-be64-cc8ace9097fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:15:52.308 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-14T06:15:52.308 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-14T06:15:52.412 INFO:tasks.ceph.mgr.y.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-14T06:15:52.412 INFO:tasks.ceph.mgr.y.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-14T06:15:52.412 INFO:tasks.ceph.mgr.y.smithi039.stderr: from numpy import show_config as show_numpy_config 2024-09-14T06:15:52.978 INFO:tasks.ceph.mgr.x.smithi087.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-14T06:15:52.978 INFO:tasks.ceph.mgr.x.smithi087.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-14T06:15:52.978 INFO:tasks.ceph.mgr.x.smithi087.stderr: from numpy import show_config as show_numpy_config 2024-09-14T06:15:52.987 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:15:52.984+0000 7f318b8a17c0 -1 Falling back to public interface 2024-09-14T06:15:53.003 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:15:53.000+0000 7f8b0dd7d7c0 -1 Falling back to public interface 2024-09-14T06:15:53.114 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T06:15:53.183 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:15:53.180+0000 7f88f577a7c0 -1 Falling back to public interface 2024-09-14T06:15:53.255 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:15:53.252+0000 7fd9787ef7c0 -1 Falling back to public interface 2024-09-14T06:15:53.295 INFO:teuthology.misc.health.smithi039.stdout: 2024-09-14T06:15:53.296 INFO:teuthology.misc.health.smithi039.stdout:{"epoch":10,"fsid":"038563b1-268f-461f-b6a6-4b4b41ad0236","created":"2024-09-14T06:15:49.493035+0000","modified":"2024-09-14T06:15:52.027930+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T06:15:52.027930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c669ea76-ed19-4c12-9e56-d79d62846f95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9bda988f-e615-4545-849b-318c408c4c42","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8ea6cd69-2ef1-42ea-b339-7394bf8f4a87","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fea45e8a-e8ea-4963-b93f-0651b84c9e74","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b601e852-a180-477f-86fc-6a213fa866f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1113afe0-f524-41b1-8e51-eb5164602d38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c9162545-318b-4591-be64-cc8ace9097fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:15:53.310 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-14T06:15:53.447 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:15:53.444+0000 7fc9485837c0 -1 Falling back to public interface 2024-09-14T06:15:53.464 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:15:53.460+0000 7fe979fc07c0 -1 Falling back to public interface 2024-09-14T06:15:53.488 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:15:53.484+0000 7f453c6fd7c0 -1 Falling back to public interface 2024-09-14T06:15:53.772 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:15:53.768+0000 7fdedab357c0 -1 Falling back to public interface 2024-09-14T06:15:55.397 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-14T06:15:55.392+0000 7f4add550640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-14T06:15:55.397 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-14T06:15:55.392+0000 7f4add550640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-14T06:15:57.110 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:15:57.108+0000 7f8b0dd7d7c0 -1 osd.7 0 log_to_monitors true 2024-09-14T06:15:57.202 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:15:57.200+0000 7f318b8a17c0 -1 osd.6 0 log_to_monitors true 2024-09-14T06:15:57.502 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:15:57.500+0000 7fc9485837c0 -1 osd.4 0 log_to_monitors true 2024-09-14T06:15:57.550 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:15:57.548+0000 7f88f577a7c0 -1 osd.5 0 log_to_monitors true 2024-09-14T06:15:57.658 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:15:57.652+0000 7fd9787ef7c0 -1 osd.3 0 log_to_monitors true 2024-09-14T06:15:57.711 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:15:57.708+0000 7fe979fc07c0 -1 osd.0 0 log_to_monitors true 2024-09-14T06:15:57.779 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:15:57.776+0000 7f453c6fd7c0 -1 osd.1 0 log_to_monitors true 2024-09-14T06:15:57.861 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:15:57.852+0000 7fdedab357c0 -1 osd.2 0 log_to_monitors true 2024-09-14T06:15:59.809 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:15:59.804+0000 7f318805c640 -1 osd.6 0 waiting for initial osdmap 2024-09-14T06:15:59.818 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:15:59.816+0000 7f8b0a538640 -1 osd.7 0 waiting for initial osdmap 2024-09-14T06:15:59.830 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:15:59.828+0000 7f317f67c640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:15:59.838 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:15:59.836+0000 7f8b01b58640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:15:59.850 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:15:59.848+0000 7fc944d3e640 -1 osd.4 0 waiting for initial osdmap 2024-09-14T06:15:59.889 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:15:59.884+0000 7f88f1f35640 -1 osd.5 0 waiting for initial osdmap 2024-09-14T06:15:59.894 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:15:59.892+0000 7fc93c35e640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:15:59.910 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:15:59.908+0000 7f88e9555640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:16:00.116 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T06:16:00.648 INFO:teuthology.misc.health.smithi039.stdout: 2024-09-14T06:16:00.649 INFO:teuthology.misc.health.smithi039.stdout:{"epoch":13,"fsid":"038563b1-268f-461f-b6a6-4b4b41ad0236","created":"2024-09-14T06:15:49.493035+0000","modified":"2024-09-14T06:15:59.592170+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T06:15:52.027930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c669ea76-ed19-4c12-9e56-d79d62846f95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9bda988f-e615-4545-849b-318c408c4c42","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8ea6cd69-2ef1-42ea-b339-7394bf8f4a87","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fea45e8a-e8ea-4963-b93f-0651b84c9e74","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b601e852-a180-477f-86fc-6a213fa866f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1113afe0-f524-41b1-8e51-eb5164602d38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c9162545-318b-4591-be64-cc8ace9097fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:16:00.676 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-14T06:16:00.969 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:00.964+0000 7fded6ade640 -1 osd.2 0 waiting for initial osdmap 2024-09-14T06:16:00.991 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:00.988+0000 7fdece910640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:16:00.996 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:00.992+0000 7fd974798640 -1 osd.3 0 waiting for initial osdmap 2024-09-14T06:16:01.012 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:01.008+0000 7fe975f69640 -1 osd.0 0 waiting for initial osdmap 2024-09-14T06:16:01.027 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:01.024+0000 7fd96c5ca640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:16:01.039 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:01.036+0000 7f4538eb8640 -1 osd.1 0 waiting for initial osdmap 2024-09-14T06:16:01.039 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:01.036+0000 7fe96dd9b640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:16:01.083 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:01.080+0000 7f45304d8640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:16:07.483 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T06:16:07.797 INFO:teuthology.misc.health.smithi039.stdout: 2024-09-14T06:16:07.797 INFO:teuthology.misc.health.smithi039.stdout:{"epoch":18,"fsid":"038563b1-268f-461f-b6a6-4b4b41ad0236","created":"2024-09-14T06:15:49.493035+0000","modified":"2024-09-14T06:16:05.062681+0000","last_up_change":"2024-09-14T06:16:02.014989+0000","last_in_change":"2024-09-14T06:15:52.027930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:16:02.429225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c669ea76-ed19-4c12-9e56-d79d62846f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6809","nonce":1157559201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6811","nonce":1157559201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6815","nonce":1157559201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6813","nonce":1157559201}]},"public_addr":"172.21.15.39:6809/1157559201","cluster_addr":"172.21.15.39:6811/1157559201","heartbeat_back_addr":"172.21.15.39:6815/1157559201","heartbeat_front_addr":"172.21.15.39:6813/1157559201","state":["exists","up"]},{"osd":1,"uuid":"7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6817","nonce":4062089212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6819","nonce":4062089212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6823","nonce":4062089212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6821","nonce":4062089212}]},"public_addr":"172.21.15.39:6817/4062089212","cluster_addr":"172.21.15.39:6819/4062089212","heartbeat_back_addr":"172.21.15.39:6823/4062089212","heartbeat_front_addr":"172.21.15.39:6821/4062089212","state":["exists","up"]},{"osd":2,"uuid":"9bda988f-e615-4545-849b-318c408c4c42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6825","nonce":1724773882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6827","nonce":1724773882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6831","nonce":1724773882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6829","nonce":1724773882}]},"public_addr":"172.21.15.39:6825/1724773882","cluster_addr":"172.21.15.39:6827/1724773882","heartbeat_back_addr":"172.21.15.39:6831/1724773882","heartbeat_front_addr":"172.21.15.39:6829/1724773882","state":["exists","up"]},{"osd":3,"uuid":"8ea6cd69-2ef1-42ea-b339-7394bf8f4a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6801","nonce":3775031293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6803","nonce":3775031293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6807","nonce":3775031293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6805","nonce":3775031293}]},"public_addr":"172.21.15.39:6801/3775031293","cluster_addr":"172.21.15.39:6803/3775031293","heartbeat_back_addr":"172.21.15.39:6807/3775031293","heartbeat_front_addr":"172.21.15.39:6805/3775031293","state":["exists","up"]},{"osd":4,"uuid":"fea45e8a-e8ea-4963-b93f-0651b84c9e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6825","nonce":2632317380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6827","nonce":2632317380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6831","nonce":2632317380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6829","nonce":2632317380}]},"public_addr":"172.21.15.87:6825/2632317380","cluster_addr":"172.21.15.87:6827/2632317380","heartbeat_back_addr":"172.21.15.87:6831/2632317380","heartbeat_front_addr":"172.21.15.87:6829/2632317380","state":["exists","up"]},{"osd":5,"uuid":"b601e852-a180-477f-86fc-6a213fa866f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6817","nonce":3501664149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6819","nonce":3501664149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6823","nonce":3501664149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6821","nonce":3501664149}]},"public_addr":"172.21.15.87:6817/3501664149","cluster_addr":"172.21.15.87:6819/3501664149","heartbeat_back_addr":"172.21.15.87:6823/3501664149","heartbeat_front_addr":"172.21.15.87:6821/3501664149","state":["exists","up"]},{"osd":6,"uuid":"1113afe0-f524-41b1-8e51-eb5164602d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6801","nonce":1395936437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6803","nonce":1395936437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6807","nonce":1395936437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6805","nonce":1395936437}]},"public_addr":"172.21.15.87:6801/1395936437","cluster_addr":"172.21.15.87:6803/1395936437","heartbeat_back_addr":"172.21.15.87:6807/1395936437","heartbeat_front_addr":"172.21.15.87:6805/1395936437","state":["exists","up"]},{"osd":7,"uuid":"c9162545-318b-4591-be64-cc8ace9097fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6809","nonce":711112216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6811","nonce":711112216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6815","nonce":711112216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6813","nonce":711112216}]},"public_addr":"172.21.15.87:6809/711112216","cluster_addr":"172.21.15.87:6811/711112216","heartbeat_back_addr":"172.21.15.87:6815/711112216","heartbeat_front_addr":"172.21.15.87:6813/711112216","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.705575+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.806946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.853831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.690315+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.494526+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.546122+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.173691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.148509+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:16:07.815 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-14T06:16:07.815 INFO:tasks.ceph:Creating RBD pool 2024-09-14T06:16:07.815 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-14T06:16:09.121 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'rbd' created 2024-09-14T06:16:09.148 DEBUG:teuthology.orchestra.run.smithi039:> rbd --cluster ceph pool init rbd 2024-09-14T06:16:12.199 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-14T06:16:12.200 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-14T06:16:12.200 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-14T06:16:12.523 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:16:12.543 INFO:teuthology.orchestra.run.smithi039.stdout:[{"version":9,"timestamp":"2024-09-14T06:16:01.040441+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"3552.677485"}]}] 2024-09-14T06:16:12.544 INFO:tasks.ceph_manager:config epoch is 9 2024-09-14T06:16:12.544 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-14T06:16:12.544 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-14T06:16:12.544 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-14T06:16:12.902 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:16:12.919 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":679112618},{"type":"v1","addr":"172.21.15.39:6833","nonce":679112618}]},"active_addr":"172.21.15.39:6833/679112618","active_change":"2024-09-14T06:15:54.362755+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":34174542}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":1567029427}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":3503009889}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":2872939124}]}]} 2024-09-14T06:16:12.923 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-14T06:16:12.923 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T06:16:12.924 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T06:16:13.243 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:16:13.243 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":22,"fsid":"038563b1-268f-461f-b6a6-4b4b41ad0236","created":"2024-09-14T06:15:49.493035+0000","modified":"2024-09-14T06:16:12.153642+0000","last_up_change":"2024-09-14T06:16:02.014989+0000","last_in_change":"2024-09-14T06:15:52.027930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:16:02.429225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T06:16:08.129454+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c669ea76-ed19-4c12-9e56-d79d62846f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6809","nonce":1157559201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6811","nonce":1157559201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6815","nonce":1157559201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6813","nonce":1157559201}]},"public_addr":"172.21.15.39:6809/1157559201","cluster_addr":"172.21.15.39:6811/1157559201","heartbeat_back_addr":"172.21.15.39:6815/1157559201","heartbeat_front_addr":"172.21.15.39:6813/1157559201","state":["exists","up"]},{"osd":1,"uuid":"7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6817","nonce":4062089212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6819","nonce":4062089212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6823","nonce":4062089212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6821","nonce":4062089212}]},"public_addr":"172.21.15.39:6817/4062089212","cluster_addr":"172.21.15.39:6819/4062089212","heartbeat_back_addr":"172.21.15.39:6823/4062089212","heartbeat_front_addr":"172.21.15.39:6821/4062089212","state":["exists","up"]},{"osd":2,"uuid":"9bda988f-e615-4545-849b-318c408c4c42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6825","nonce":1724773882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6827","nonce":1724773882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6831","nonce":1724773882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6829","nonce":1724773882}]},"public_addr":"172.21.15.39:6825/1724773882","cluster_addr":"172.21.15.39:6827/1724773882","heartbeat_back_addr":"172.21.15.39:6831/1724773882","heartbeat_front_addr":"172.21.15.39:6829/1724773882","state":["exists","up"]},{"osd":3,"uuid":"8ea6cd69-2ef1-42ea-b339-7394bf8f4a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6801","nonce":3775031293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6803","nonce":3775031293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6807","nonce":3775031293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6805","nonce":3775031293}]},"public_addr":"172.21.15.39:6801/3775031293","cluster_addr":"172.21.15.39:6803/3775031293","heartbeat_back_addr":"172.21.15.39:6807/3775031293","heartbeat_front_addr":"172.21.15.39:6805/3775031293","state":["exists","up"]},{"osd":4,"uuid":"fea45e8a-e8ea-4963-b93f-0651b84c9e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6825","nonce":2632317380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6827","nonce":2632317380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6831","nonce":2632317380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6829","nonce":2632317380}]},"public_addr":"172.21.15.87:6825/2632317380","cluster_addr":"172.21.15.87:6827/2632317380","heartbeat_back_addr":"172.21.15.87:6831/2632317380","heartbeat_front_addr":"172.21.15.87:6829/2632317380","state":["exists","up"]},{"osd":5,"uuid":"b601e852-a180-477f-86fc-6a213fa866f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6817","nonce":3501664149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6819","nonce":3501664149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6823","nonce":3501664149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6821","nonce":3501664149}]},"public_addr":"172.21.15.87:6817/3501664149","cluster_addr":"172.21.15.87:6819/3501664149","heartbeat_back_addr":"172.21.15.87:6823/3501664149","heartbeat_front_addr":"172.21.15.87:6821/3501664149","state":["exists","up"]},{"osd":6,"uuid":"1113afe0-f524-41b1-8e51-eb5164602d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6801","nonce":1395936437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6803","nonce":1395936437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6807","nonce":1395936437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6805","nonce":1395936437}]},"public_addr":"172.21.15.87:6801/1395936437","cluster_addr":"172.21.15.87:6803/1395936437","heartbeat_back_addr":"172.21.15.87:6807/1395936437","heartbeat_front_addr":"172.21.15.87:6805/1395936437","state":["exists","up"]},{"osd":7,"uuid":"c9162545-318b-4591-be64-cc8ace9097fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6809","nonce":711112216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6811","nonce":711112216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6815","nonce":711112216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6813","nonce":711112216}]},"public_addr":"172.21.15.87:6809/711112216","cluster_addr":"172.21.15.87:6811/711112216","heartbeat_back_addr":"172.21.15.87:6815/711112216","heartbeat_front_addr":"172.21.15.87:6813/711112216","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.705575+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.806946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.853831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.690315+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.494526+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.546122+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.173691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.148509+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:16:13.262 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T06:16:13.262 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T06:16:13.571 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:16:13.571 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":22,"fsid":"038563b1-268f-461f-b6a6-4b4b41ad0236","created":"2024-09-14T06:15:49.493035+0000","modified":"2024-09-14T06:16:12.153642+0000","last_up_change":"2024-09-14T06:16:02.014989+0000","last_in_change":"2024-09-14T06:15:52.027930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:16:02.429225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T06:16:08.129454+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c669ea76-ed19-4c12-9e56-d79d62846f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6809","nonce":1157559201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6811","nonce":1157559201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6815","nonce":1157559201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6813","nonce":1157559201}]},"public_addr":"172.21.15.39:6809/1157559201","cluster_addr":"172.21.15.39:6811/1157559201","heartbeat_back_addr":"172.21.15.39:6815/1157559201","heartbeat_front_addr":"172.21.15.39:6813/1157559201","state":["exists","up"]},{"osd":1,"uuid":"7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6817","nonce":4062089212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6819","nonce":4062089212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6823","nonce":4062089212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6821","nonce":4062089212}]},"public_addr":"172.21.15.39:6817/4062089212","cluster_addr":"172.21.15.39:6819/4062089212","heartbeat_back_addr":"172.21.15.39:6823/4062089212","heartbeat_front_addr":"172.21.15.39:6821/4062089212","state":["exists","up"]},{"osd":2,"uuid":"9bda988f-e615-4545-849b-318c408c4c42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6825","nonce":1724773882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6827","nonce":1724773882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6831","nonce":1724773882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6829","nonce":1724773882}]},"public_addr":"172.21.15.39:6825/1724773882","cluster_addr":"172.21.15.39:6827/1724773882","heartbeat_back_addr":"172.21.15.39:6831/1724773882","heartbeat_front_addr":"172.21.15.39:6829/1724773882","state":["exists","up"]},{"osd":3,"uuid":"8ea6cd69-2ef1-42ea-b339-7394bf8f4a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6801","nonce":3775031293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6803","nonce":3775031293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6807","nonce":3775031293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6805","nonce":3775031293}]},"public_addr":"172.21.15.39:6801/3775031293","cluster_addr":"172.21.15.39:6803/3775031293","heartbeat_back_addr":"172.21.15.39:6807/3775031293","heartbeat_front_addr":"172.21.15.39:6805/3775031293","state":["exists","up"]},{"osd":4,"uuid":"fea45e8a-e8ea-4963-b93f-0651b84c9e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6825","nonce":2632317380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6827","nonce":2632317380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6831","nonce":2632317380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6829","nonce":2632317380}]},"public_addr":"172.21.15.87:6825/2632317380","cluster_addr":"172.21.15.87:6827/2632317380","heartbeat_back_addr":"172.21.15.87:6831/2632317380","heartbeat_front_addr":"172.21.15.87:6829/2632317380","state":["exists","up"]},{"osd":5,"uuid":"b601e852-a180-477f-86fc-6a213fa866f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6817","nonce":3501664149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6819","nonce":3501664149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6823","nonce":3501664149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6821","nonce":3501664149}]},"public_addr":"172.21.15.87:6817/3501664149","cluster_addr":"172.21.15.87:6819/3501664149","heartbeat_back_addr":"172.21.15.87:6823/3501664149","heartbeat_front_addr":"172.21.15.87:6821/3501664149","state":["exists","up"]},{"osd":6,"uuid":"1113afe0-f524-41b1-8e51-eb5164602d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6801","nonce":1395936437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6803","nonce":1395936437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6807","nonce":1395936437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6805","nonce":1395936437}]},"public_addr":"172.21.15.87:6801/1395936437","cluster_addr":"172.21.15.87:6803/1395936437","heartbeat_back_addr":"172.21.15.87:6807/1395936437","heartbeat_front_addr":"172.21.15.87:6805/1395936437","state":["exists","up"]},{"osd":7,"uuid":"c9162545-318b-4591-be64-cc8ace9097fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6809","nonce":711112216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6811","nonce":711112216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6815","nonce":711112216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6813","nonce":711112216}]},"public_addr":"172.21.15.87:6809/711112216","cluster_addr":"172.21.15.87:6811/711112216","heartbeat_back_addr":"172.21.15.87:6815/711112216","heartbeat_front_addr":"172.21.15.87:6813/711112216","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.705575+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.806946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.853831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.690315+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.494526+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.546122+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.173691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.148509+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:16:13.589 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T06:16:13.589 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T06:16:13.590 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T06:16:13.590 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T06:16:13.590 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T06:16:13.591 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T06:16:13.591 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-14T06:16:13.592 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-14T06:16:13.841 INFO:teuthology.orchestra.run.smithi039.stdout:64424509445 2024-09-14T06:16:13.842 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T06:16:13.855 INFO:teuthology.orchestra.run.smithi039.stdout:60129542150 2024-09-14T06:16:13.856 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T06:16:13.857 INFO:teuthology.orchestra.run.smithi039.stdout:60129542149 2024-09-14T06:16:13.857 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T06:16:13.860 INFO:teuthology.orchestra.run.smithi039.stdout:64424509446 2024-09-14T06:16:13.860 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T06:16:13.865 INFO:teuthology.orchestra.run.smithi039.stdout:64424509445 2024-09-14T06:16:13.865 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T06:16:13.876 INFO:teuthology.orchestra.run.smithi039.stdout:60129542149 2024-09-14T06:16:13.876 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T06:16:13.878 INFO:teuthology.orchestra.run.smithi039.stdout:60129542148 2024-09-14T06:16:13.878 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T06:16:13.879 INFO:teuthology.orchestra.run.smithi039.stdout:64424509447 2024-09-14T06:16:13.880 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T06:16:14.357 INFO:teuthology.orchestra.run.smithi039.stdout:60129542149 2024-09-14T06:16:14.379 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.4 2024-09-14T06:16:14.391 INFO:teuthology.orchestra.run.smithi039.stdout:64424509443 2024-09-14T06:16:14.407 INFO:teuthology.orchestra.run.smithi039.stdout:60129542148 2024-09-14T06:16:14.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509443 for osd.2 2024-09-14T06:16:14.424 INFO:teuthology.orchestra.run.smithi039.stdout:64424509443 2024-09-14T06:16:14.424 INFO:teuthology.orchestra.run.smithi039.stdout:64424509444 2024-09-14T06:16:14.429 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.7 2024-09-14T06:16:14.441 INFO:teuthology.orchestra.run.smithi039.stdout:60129542148 2024-09-14T06:16:14.449 INFO:teuthology.orchestra.run.smithi039.stdout:64424509445 2024-09-14T06:16:14.454 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509443 for osd.3 2024-09-14T06:16:14.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.0 2024-09-14T06:16:14.464 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2024-09-14T06:16:14.466 INFO:teuthology.orchestra.run.smithi039.stdout:60129542147 2024-09-14T06:16:14.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.1 2024-09-14T06:16:14.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.5 2024-09-14T06:16:15.379 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T06:16:15.419 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T06:16:15.430 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T06:16:15.454 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T06:16:15.456 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T06:16:15.464 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T06:16:15.470 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T06:16:15.484 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T06:16:15.886 INFO:teuthology.orchestra.run.smithi039.stdout:60129542150 2024-09-14T06:16:15.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2024-09-14T06:16:15.912 DEBUG:teuthology.parallel:result is None 2024-09-14T06:16:15.969 INFO:teuthology.orchestra.run.smithi039.stdout:64424509447 2024-09-14T06:16:15.975 INFO:teuthology.orchestra.run.smithi039.stdout:64424509445 2024-09-14T06:16:15.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2024-09-14T06:16:15.990 DEBUG:teuthology.parallel:result is None 2024-09-14T06:16:16.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.2 2024-09-14T06:16:16.004 DEBUG:teuthology.parallel:result is None 2024-09-14T06:16:16.019 INFO:teuthology.orchestra.run.smithi039.stdout:60129542149 2024-09-14T06:16:16.020 INFO:teuthology.orchestra.run.smithi039.stdout:64424509446 2024-09-14T06:16:16.038 INFO:teuthology.orchestra.run.smithi039.stdout:64424509445 2024-09-14T06:16:16.039 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.7 2024-09-14T06:16:16.039 DEBUG:teuthology.parallel:result is None 2024-09-14T06:16:16.040 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.0 2024-09-14T06:16:16.041 DEBUG:teuthology.parallel:result is None 2024-09-14T06:16:16.050 INFO:teuthology.orchestra.run.smithi039.stdout:60129542148 2024-09-14T06:16:16.061 INFO:teuthology.orchestra.run.smithi039.stdout:60129542149 2024-09-14T06:16:16.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.3 2024-09-14T06:16:16.064 DEBUG:teuthology.parallel:result is None 2024-09-14T06:16:16.069 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.5 2024-09-14T06:16:16.070 DEBUG:teuthology.parallel:result is None 2024-09-14T06:16:16.080 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.6 2024-09-14T06:16:16.080 DEBUG:teuthology.parallel:result is None 2024-09-14T06:16:16.080 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T06:16:16.081 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:16:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:16:16.403 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:16:16.421 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-14T06:16:16.386253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234748,"kb_used_data":1856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739972,"statfs":{"total":773094113280,"available":772853731328,"internally_reserved":0,"allocated":1900544,"data_stored":1783174,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.256033"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.179717+0000","last_change":"2024-09-14T06:16:12.179717+0000","last_active":"2024-09-14T06:16:12.179717+0000","last_peered":"2024-09-14T06:16:12.179717+0000","last_clean":"2024-09-14T06:16:12.179717+0000","last_became_active":"2024-09-14T06:16:10.150426+0000","last_became_peered":"2024-09-14T06:16:10.150426+0000","last_unstale":"2024-09-14T06:16:12.179717+0000","last_undegraded":"2024-09-14T06:16:12.179717+0000","last_fullsized":"2024-09-14T06:16:12.179717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:29:55.403337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010688049999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.382896+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:16:12.382896+0000","last_peered":"2024-09-14T06:16:12.382896+0000","last_clean":"2024-09-14T06:16:12.382896+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:16:12.382896+0000","last_undegraded":"2024-09-14T06:16:12.382896+0000","last_fullsized":"2024-09-14T06:16:12.382896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.180261+0000","last_change":"2024-09-14T06:16:12.180261+0000","last_active":"2024-09-14T06:16:12.180261+0000","last_peered":"2024-09-14T06:16:12.180261+0000","last_clean":"2024-09-14T06:16:12.180261+0000","last_became_active":"2024-09-14T06:16:10.147337+0000","last_became_peered":"2024-09-14T06:16:10.147337+0000","last_unstale":"2024-09-14T06:16:12.180261+0000","last_undegraded":"2024-09-14T06:16:12.180261+0000","last_fullsized":"2024-09-14T06:16:12.180261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:54:30.677581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001722187,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.162328+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:16:12.162328+0000","last_peered":"2024-09-14T06:16:12.162328+0000","last_clean":"2024-09-14T06:16:12.162328+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:16:12.162328+0000","last_undegraded":"2024-09-14T06:16:12.162328+0000","last_fullsized":"2024-09-14T06:16:12.162328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.185168+0000","last_change":"2024-09-14T06:16:12.182029+0000","last_active":"2024-09-14T06:16:12.185168+0000","last_peered":"2024-09-14T06:16:12.185168+0000","last_clean":"2024-09-14T06:16:12.185168+0000","last_became_active":"2024-09-14T06:16:10.196390+0000","last_became_peered":"2024-09-14T06:16:10.196390+0000","last_unstale":"2024-09-14T06:16:12.185168+0000","last_undegraded":"2024-09-14T06:16:12.185168+0000","last_fullsized":"2024-09-14T06:16:12.185168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:21.237087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034648000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.919743+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:16:12.919743+0000","last_peered":"2024-09-14T06:16:12.919743+0000","last_clean":"2024-09-14T06:16:12.919743+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:16:12.919743+0000","last_undegraded":"2024-09-14T06:16:12.919743+0000","last_fullsized":"2024-09-14T06:16:12.919743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.179657+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:16:12.179657+0000","last_peered":"2024-09-14T06:16:12.179657+0000","last_clean":"2024-09-14T06:16:12.179657+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:16:12.179657+0000","last_undegraded":"2024-09-14T06:16:12.179657+0000","last_fullsized":"2024-09-14T06:16:12.179657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.181967+0000","last_change":"2024-09-14T06:16:12.181967+0000","last_active":"2024-09-14T06:16:12.181967+0000","last_peered":"2024-09-14T06:16:12.181967+0000","last_clean":"2024-09-14T06:16:12.181967+0000","last_became_active":"2024-09-14T06:16:10.147507+0000","last_became_peered":"2024-09-14T06:16:10.147507+0000","last_unstale":"2024-09-14T06:16:12.181967+0000","last_undegraded":"2024-09-14T06:16:12.181967+0000","last_fullsized":"2024-09-14T06:16:12.181967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:28:01.545390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.640968+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:16:12.640968+0000","last_peered":"2024-09-14T06:16:12.640968+0000","last_clean":"2024-09-14T06:16:12.640968+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:16:12.640968+0000","last_undegraded":"2024-09-14T06:16:12.640968+0000","last_fullsized":"2024-09-14T06:16:12.640968+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":75319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":282624,"data_stored":665668,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":75300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":665668,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:16:16.422 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:16:16.717 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:16:16.718 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:16:16.734 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-14T06:16:16.386253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234748,"kb_used_data":1856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739972,"statfs":{"total":773094113280,"available":772853731328,"internally_reserved":0,"allocated":1900544,"data_stored":1783174,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.256033"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.179717+0000","last_change":"2024-09-14T06:16:12.179717+0000","last_active":"2024-09-14T06:16:12.179717+0000","last_peered":"2024-09-14T06:16:12.179717+0000","last_clean":"2024-09-14T06:16:12.179717+0000","last_became_active":"2024-09-14T06:16:10.150426+0000","last_became_peered":"2024-09-14T06:16:10.150426+0000","last_unstale":"2024-09-14T06:16:12.179717+0000","last_undegraded":"2024-09-14T06:16:12.179717+0000","last_fullsized":"2024-09-14T06:16:12.179717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:29:55.403337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010688049999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.382896+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:16:12.382896+0000","last_peered":"2024-09-14T06:16:12.382896+0000","last_clean":"2024-09-14T06:16:12.382896+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:16:12.382896+0000","last_undegraded":"2024-09-14T06:16:12.382896+0000","last_fullsized":"2024-09-14T06:16:12.382896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.180261+0000","last_change":"2024-09-14T06:16:12.180261+0000","last_active":"2024-09-14T06:16:12.180261+0000","last_peered":"2024-09-14T06:16:12.180261+0000","last_clean":"2024-09-14T06:16:12.180261+0000","last_became_active":"2024-09-14T06:16:10.147337+0000","last_became_peered":"2024-09-14T06:16:10.147337+0000","last_unstale":"2024-09-14T06:16:12.180261+0000","last_undegraded":"2024-09-14T06:16:12.180261+0000","last_fullsized":"2024-09-14T06:16:12.180261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:54:30.677581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001722187,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.162328+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:16:12.162328+0000","last_peered":"2024-09-14T06:16:12.162328+0000","last_clean":"2024-09-14T06:16:12.162328+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:16:12.162328+0000","last_undegraded":"2024-09-14T06:16:12.162328+0000","last_fullsized":"2024-09-14T06:16:12.162328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.185168+0000","last_change":"2024-09-14T06:16:12.182029+0000","last_active":"2024-09-14T06:16:12.185168+0000","last_peered":"2024-09-14T06:16:12.185168+0000","last_clean":"2024-09-14T06:16:12.185168+0000","last_became_active":"2024-09-14T06:16:10.196390+0000","last_became_peered":"2024-09-14T06:16:10.196390+0000","last_unstale":"2024-09-14T06:16:12.185168+0000","last_undegraded":"2024-09-14T06:16:12.185168+0000","last_fullsized":"2024-09-14T06:16:12.185168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:21.237087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034648000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.919743+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:16:12.919743+0000","last_peered":"2024-09-14T06:16:12.919743+0000","last_clean":"2024-09-14T06:16:12.919743+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:16:12.919743+0000","last_undegraded":"2024-09-14T06:16:12.919743+0000","last_fullsized":"2024-09-14T06:16:12.919743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.179657+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:16:12.179657+0000","last_peered":"2024-09-14T06:16:12.179657+0000","last_clean":"2024-09-14T06:16:12.179657+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:16:12.179657+0000","last_undegraded":"2024-09-14T06:16:12.179657+0000","last_fullsized":"2024-09-14T06:16:12.179657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.181967+0000","last_change":"2024-09-14T06:16:12.181967+0000","last_active":"2024-09-14T06:16:12.181967+0000","last_peered":"2024-09-14T06:16:12.181967+0000","last_clean":"2024-09-14T06:16:12.181967+0000","last_became_active":"2024-09-14T06:16:10.147507+0000","last_became_peered":"2024-09-14T06:16:10.147507+0000","last_unstale":"2024-09-14T06:16:12.181967+0000","last_undegraded":"2024-09-14T06:16:12.181967+0000","last_fullsized":"2024-09-14T06:16:12.181967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:28:01.545390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.640968+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:16:12.640968+0000","last_peered":"2024-09-14T06:16:12.640968+0000","last_clean":"2024-09-14T06:16:12.640968+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:16:12.640968+0000","last_undegraded":"2024-09-14T06:16:12.640968+0000","last_fullsized":"2024-09-14T06:16:12.640968+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":75319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":282624,"data_stored":665668,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":75300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":665668,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:16:16.735 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T06:16:16.735 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-14T06:16:16.735 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-14T06:16:16.735 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-14T06:16:17.058 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:16:17.059 INFO:teuthology.orchestra.run.smithi039.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-14T06:16:17.076 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-14T06:16:17.076 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-14T06:16:17.090 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-14T06:16:17.090 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd clone copy on read': True}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a'}} 2024-09-14T06:16:17.090 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-14T06:16:17.091 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-14T06:16:17.424 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-14T06:16:17.425 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T06:16:17.425 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:16:17.712 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:16:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:16:17.729 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-14T06:16:16.386253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234748,"kb_used_data":1856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739972,"statfs":{"total":773094113280,"available":772853731328,"internally_reserved":0,"allocated":1900544,"data_stored":1783174,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.256033"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.179717+0000","last_change":"2024-09-14T06:16:12.179717+0000","last_active":"2024-09-14T06:16:12.179717+0000","last_peered":"2024-09-14T06:16:12.179717+0000","last_clean":"2024-09-14T06:16:12.179717+0000","last_became_active":"2024-09-14T06:16:10.150426+0000","last_became_peered":"2024-09-14T06:16:10.150426+0000","last_unstale":"2024-09-14T06:16:12.179717+0000","last_undegraded":"2024-09-14T06:16:12.179717+0000","last_fullsized":"2024-09-14T06:16:12.179717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:29:55.403337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010688049999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.382896+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:16:12.382896+0000","last_peered":"2024-09-14T06:16:12.382896+0000","last_clean":"2024-09-14T06:16:12.382896+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:16:12.382896+0000","last_undegraded":"2024-09-14T06:16:12.382896+0000","last_fullsized":"2024-09-14T06:16:12.382896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.180261+0000","last_change":"2024-09-14T06:16:12.180261+0000","last_active":"2024-09-14T06:16:12.180261+0000","last_peered":"2024-09-14T06:16:12.180261+0000","last_clean":"2024-09-14T06:16:12.180261+0000","last_became_active":"2024-09-14T06:16:10.147337+0000","last_became_peered":"2024-09-14T06:16:10.147337+0000","last_unstale":"2024-09-14T06:16:12.180261+0000","last_undegraded":"2024-09-14T06:16:12.180261+0000","last_fullsized":"2024-09-14T06:16:12.180261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:54:30.677581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001722187,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.162328+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:16:12.162328+0000","last_peered":"2024-09-14T06:16:12.162328+0000","last_clean":"2024-09-14T06:16:12.162328+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:16:12.162328+0000","last_undegraded":"2024-09-14T06:16:12.162328+0000","last_fullsized":"2024-09-14T06:16:12.162328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.185168+0000","last_change":"2024-09-14T06:16:12.182029+0000","last_active":"2024-09-14T06:16:12.185168+0000","last_peered":"2024-09-14T06:16:12.185168+0000","last_clean":"2024-09-14T06:16:12.185168+0000","last_became_active":"2024-09-14T06:16:10.196390+0000","last_became_peered":"2024-09-14T06:16:10.196390+0000","last_unstale":"2024-09-14T06:16:12.185168+0000","last_undegraded":"2024-09-14T06:16:12.185168+0000","last_fullsized":"2024-09-14T06:16:12.185168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:21.237087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034648000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.919743+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:16:12.919743+0000","last_peered":"2024-09-14T06:16:12.919743+0000","last_clean":"2024-09-14T06:16:12.919743+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:16:12.919743+0000","last_undegraded":"2024-09-14T06:16:12.919743+0000","last_fullsized":"2024-09-14T06:16:12.919743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.179657+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:16:12.179657+0000","last_peered":"2024-09-14T06:16:12.179657+0000","last_clean":"2024-09-14T06:16:12.179657+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:16:12.179657+0000","last_undegraded":"2024-09-14T06:16:12.179657+0000","last_fullsized":"2024-09-14T06:16:12.179657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.181967+0000","last_change":"2024-09-14T06:16:12.181967+0000","last_active":"2024-09-14T06:16:12.181967+0000","last_peered":"2024-09-14T06:16:12.181967+0000","last_clean":"2024-09-14T06:16:12.181967+0000","last_became_active":"2024-09-14T06:16:10.147507+0000","last_became_peered":"2024-09-14T06:16:10.147507+0000","last_unstale":"2024-09-14T06:16:12.181967+0000","last_undegraded":"2024-09-14T06:16:12.181967+0000","last_fullsized":"2024-09-14T06:16:12.181967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:28:01.545390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.640968+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:16:12.640968+0000","last_peered":"2024-09-14T06:16:12.640968+0000","last_clean":"2024-09-14T06:16:12.640968+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:16:12.640968+0000","last_undegraded":"2024-09-14T06:16:12.640968+0000","last_fullsized":"2024-09-14T06:16:12.640968+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":75319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":282624,"data_stored":665668,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":75300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":665668,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:16:17.730 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:16:18.037 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:16:18.038 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:16:18.057 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-14T06:16:16.386253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234748,"kb_used_data":1856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739972,"statfs":{"total":773094113280,"available":772853731328,"internally_reserved":0,"allocated":1900544,"data_stored":1783174,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.256033"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.179717+0000","last_change":"2024-09-14T06:16:12.179717+0000","last_active":"2024-09-14T06:16:12.179717+0000","last_peered":"2024-09-14T06:16:12.179717+0000","last_clean":"2024-09-14T06:16:12.179717+0000","last_became_active":"2024-09-14T06:16:10.150426+0000","last_became_peered":"2024-09-14T06:16:10.150426+0000","last_unstale":"2024-09-14T06:16:12.179717+0000","last_undegraded":"2024-09-14T06:16:12.179717+0000","last_fullsized":"2024-09-14T06:16:12.179717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:29:55.403337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010688049999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.382896+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:16:12.382896+0000","last_peered":"2024-09-14T06:16:12.382896+0000","last_clean":"2024-09-14T06:16:12.382896+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:16:12.382896+0000","last_undegraded":"2024-09-14T06:16:12.382896+0000","last_fullsized":"2024-09-14T06:16:12.382896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.180261+0000","last_change":"2024-09-14T06:16:12.180261+0000","last_active":"2024-09-14T06:16:12.180261+0000","last_peered":"2024-09-14T06:16:12.180261+0000","last_clean":"2024-09-14T06:16:12.180261+0000","last_became_active":"2024-09-14T06:16:10.147337+0000","last_became_peered":"2024-09-14T06:16:10.147337+0000","last_unstale":"2024-09-14T06:16:12.180261+0000","last_undegraded":"2024-09-14T06:16:12.180261+0000","last_fullsized":"2024-09-14T06:16:12.180261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:54:30.677581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001722187,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.162328+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:16:12.162328+0000","last_peered":"2024-09-14T06:16:12.162328+0000","last_clean":"2024-09-14T06:16:12.162328+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:16:12.162328+0000","last_undegraded":"2024-09-14T06:16:12.162328+0000","last_fullsized":"2024-09-14T06:16:12.162328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.185168+0000","last_change":"2024-09-14T06:16:12.182029+0000","last_active":"2024-09-14T06:16:12.185168+0000","last_peered":"2024-09-14T06:16:12.185168+0000","last_clean":"2024-09-14T06:16:12.185168+0000","last_became_active":"2024-09-14T06:16:10.196390+0000","last_became_peered":"2024-09-14T06:16:10.196390+0000","last_unstale":"2024-09-14T06:16:12.185168+0000","last_undegraded":"2024-09-14T06:16:12.185168+0000","last_fullsized":"2024-09-14T06:16:12.185168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:21.237087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034648000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.919743+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:16:12.919743+0000","last_peered":"2024-09-14T06:16:12.919743+0000","last_clean":"2024-09-14T06:16:12.919743+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:16:12.919743+0000","last_undegraded":"2024-09-14T06:16:12.919743+0000","last_fullsized":"2024-09-14T06:16:12.919743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.179657+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:16:12.179657+0000","last_peered":"2024-09-14T06:16:12.179657+0000","last_clean":"2024-09-14T06:16:12.179657+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:16:12.179657+0000","last_undegraded":"2024-09-14T06:16:12.179657+0000","last_fullsized":"2024-09-14T06:16:12.179657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.181967+0000","last_change":"2024-09-14T06:16:12.181967+0000","last_active":"2024-09-14T06:16:12.181967+0000","last_peered":"2024-09-14T06:16:12.181967+0000","last_clean":"2024-09-14T06:16:12.181967+0000","last_became_active":"2024-09-14T06:16:10.147507+0000","last_became_peered":"2024-09-14T06:16:10.147507+0000","last_unstale":"2024-09-14T06:16:12.181967+0000","last_undegraded":"2024-09-14T06:16:12.181967+0000","last_fullsized":"2024-09-14T06:16:12.181967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:28:01.545390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:16:12.640968+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:16:12.640968+0000","last_peered":"2024-09-14T06:16:12.640968+0000","last_clean":"2024-09-14T06:16:12.640968+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:16:12.640968+0000","last_undegraded":"2024-09-14T06:16:12.640968+0000","last_fullsized":"2024-09-14T06:16:12.640968+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":75319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":282624,"data_stored":665668,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":75300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":665668,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:16:18.058 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T06:16:18.058 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-14T06:16:18.360 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 22 2024-09-14T06:16:18.360 INFO:teuthology.orchestra.run.smithi039.stdout:fsid 038563b1-268f-461f-b6a6-4b4b41ad0236 2024-09-14T06:16:18.361 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-09-14T06:15:49.493035+0000 2024-09-14T06:16:18.361 INFO:teuthology.orchestra.run.smithi039.stdout:modified 2024-09-14T06:16:12.153642+0000 2024-09-14T06:16:18.361 INFO:teuthology.orchestra.run.smithi039.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-14T06:16:18.361 INFO:teuthology.orchestra.run.smithi039.stdout:crush_version 5 2024-09-14T06:16:18.361 INFO:teuthology.orchestra.run.smithi039.stdout:full_ratio 0.95 2024-09-14T06:16:18.361 INFO:teuthology.orchestra.run.smithi039.stdout:backfillfull_ratio 0.9 2024-09-14T06:16:18.361 INFO:teuthology.orchestra.run.smithi039.stdout:nearfull_ratio 0.85 2024-09-14T06:16:18.361 INFO:teuthology.orchestra.run.smithi039.stdout:require_min_compat_client luminous 2024-09-14T06:16:18.361 INFO:teuthology.orchestra.run.smithi039.stdout:min_compat_client jewel 2024-09-14T06:16:18.361 INFO:teuthology.orchestra.run.smithi039.stdout:require_osd_release squid 2024-09-14T06:16:18.361 INFO:teuthology.orchestra.run.smithi039.stdout:stretch_mode_enabled false 2024-09-14T06:16:18.361 INFO:teuthology.orchestra.run.smithi039.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-14T06:16:18.362 INFO:teuthology.orchestra.run.smithi039.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 22 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 4.98 2024-09-14T06:16:18.362 INFO:teuthology.orchestra.run.smithi039.stdout: removed_snaps_queue [2~1] 2024-09-14T06:16:18.362 INFO:teuthology.orchestra.run.smithi039.stdout:max_osd 8 2024-09-14T06:16:18.362 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6808/1157559201,v1:172.21.15.39:6809/1157559201] [v2:172.21.15.39:6810/1157559201,v1:172.21.15.39:6811/1157559201] exists,up c669ea76-ed19-4c12-9e56-d79d62846f95 2024-09-14T06:16:18.362 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6816/4062089212,v1:172.21.15.39:6817/4062089212] [v2:172.21.15.39:6818/4062089212,v1:172.21.15.39:6819/4062089212] exists,up 7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b 2024-09-14T06:16:18.362 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6824/1724773882,v1:172.21.15.39:6825/1724773882] [v2:172.21.15.39:6826/1724773882,v1:172.21.15.39:6827/1724773882] exists,up 9bda988f-e615-4545-849b-318c408c4c42 2024-09-14T06:16:18.362 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6800/3775031293,v1:172.21.15.39:6801/3775031293] [v2:172.21.15.39:6802/3775031293,v1:172.21.15.39:6803/3775031293] exists,up 8ea6cd69-2ef1-42ea-b339-7394bf8f4a87 2024-09-14T06:16:18.362 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6824/2632317380,v1:172.21.15.87:6825/2632317380] [v2:172.21.15.87:6826/2632317380,v1:172.21.15.87:6827/2632317380] exists,up fea45e8a-e8ea-4963-b93f-0651b84c9e74 2024-09-14T06:16:18.362 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6816/3501664149,v1:172.21.15.87:6817/3501664149] [v2:172.21.15.87:6818/3501664149,v1:172.21.15.87:6819/3501664149] exists,up b601e852-a180-477f-86fc-6a213fa866f0 2024-09-14T06:16:18.362 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6800/1395936437,v1:172.21.15.87:6801/1395936437] [v2:172.21.15.87:6802/1395936437,v1:172.21.15.87:6803/1395936437] exists,up 1113afe0-f524-41b1-8e51-eb5164602d38 2024-09-14T06:16:18.362 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6808/711112216,v1:172.21.15.87:6809/711112216] [v2:172.21.15.87:6810/711112216,v1:172.21.15.87:6811/711112216] exists,up c9162545-318b-4591-be64-cc8ace9097fe 2024-09-14T06:16:18.378 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6808/1157559201,v1:172.21.15.39:6809/1157559201] [v2:172.21.15.39:6810/1157559201,v1:172.21.15.39:6811/1157559201] exists,up c669ea76-ed19-4c12-9e56-d79d62846f95', 'osd.1 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6816/4062089212,v1:172.21.15.39:6817/4062089212] [v2:172.21.15.39:6818/4062089212,v1:172.21.15.39:6819/4062089212] exists,up 7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b', 'osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6824/1724773882,v1:172.21.15.39:6825/1724773882] [v2:172.21.15.39:6826/1724773882,v1:172.21.15.39:6827/1724773882] exists,up 9bda988f-e615-4545-849b-318c408c4c42', 'osd.3 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6800/3775031293,v1:172.21.15.39:6801/3775031293] [v2:172.21.15.39:6802/3775031293,v1:172.21.15.39:6803/3775031293] exists,up 8ea6cd69-2ef1-42ea-b339-7394bf8f4a87', 'osd.4 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6824/2632317380,v1:172.21.15.87:6825/2632317380] [v2:172.21.15.87:6826/2632317380,v1:172.21.15.87:6827/2632317380] exists,up fea45e8a-e8ea-4963-b93f-0651b84c9e74', 'osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6816/3501664149,v1:172.21.15.87:6817/3501664149] [v2:172.21.15.87:6818/3501664149,v1:172.21.15.87:6819/3501664149] exists,up b601e852-a180-477f-86fc-6a213fa866f0', 'osd.6 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6800/1395936437,v1:172.21.15.87:6801/1395936437] [v2:172.21.15.87:6802/1395936437,v1:172.21.15.87:6803/1395936437] exists,up 1113afe0-f524-41b1-8e51-eb5164602d38', 'osd.7 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6808/711112216,v1:172.21.15.87:6809/711112216] [v2:172.21.15.87:6810/711112216,v1:172.21.15.87:6811/711112216] exists,up c9162545-318b-4591-be64-cc8ace9097fe'] 2024-09-14T06:16:18.378 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-14T06:16:18.489 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:18.489 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mon.a", 2024-09-14T06:16:18.490 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster": "ceph", 2024-09-14T06:16:18.490 INFO:teuthology.orchestra.run.smithi039.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-14T06:16:18.490 INFO:teuthology.orchestra.run.smithi039.stdout: "admin_socket_mode": "", 2024-09-14T06:16:18.490 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-14T06:16:18.490 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_client_required": "cephx, none", 2024-09-14T06:16:18.490 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_cluster_required": "cephx", 2024-09-14T06:16:18.490 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_debug": "true", 2024-09-14T06:16:18.490 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-14T06:16:18.491 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-14T06:16:18.491 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_service_required": "cephx", 2024-09-14T06:16:18.491 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-14T06:16:18.491 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_supported": "cephx", 2024-09-14T06:16:18.491 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_aio": "true", 2024-09-14T06:16:18.491 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-14T06:16:18.491 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_aio_poll_ms": "250", 2024-09-14T06:16:18.491 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_aio_reap_max": "16", 2024-09-14T06:16:18.491 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_async_discard_threads": "0", 2024-09-14T06:16:18.491 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_block_size": "4096", 2024-09-14T06:16:18.491 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_debug_aio": "false", 2024-09-14T06:16:18.491 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-14T06:16:18.492 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-14T06:16:18.492 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_debug_inflight_ios": "false", 2024-09-14T06:16:18.492 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_enable_discard": "false", 2024-09-14T06:16:18.492 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_flock_retry": "3", 2024-09-14T06:16:18.492 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-14T06:16:18.492 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_inject_crash": "0", 2024-09-14T06:16:18.493 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-14T06:16:18.493 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_ioring": "false", 2024-09-14T06:16:18.493 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_ioring_hipri": "false", 2024-09-14T06:16:18.493 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-14T06:16:18.493 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-14T06:16:18.493 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-14T06:16:18.493 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-14T06:16:18.493 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-14T06:16:18.493 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-14T06:16:18.493 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_type": "", 2024-09-14T06:16:18.493 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_alloc_size": "1048576", 2024-09-14T06:16:18.493 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_allocator": "hybrid", 2024-09-14T06:16:18.494 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_buffered_io": "true", 2024-09-14T06:16:18.494 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_check_for_zeros": "false", 2024-09-14T06:16:18.494 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-14T06:16:18.494 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-14T06:16:18.494 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_compact_log_sync": "false", 2024-09-14T06:16:18.494 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-14T06:16:18.494 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-14T06:16:18.495 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-14T06:16:18.495 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-14T06:16:18.495 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_max_log_runway": "4194304", 2024-09-14T06:16:18.495 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_max_prefetch": "1048576", 2024-09-14T06:16:18.495 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_min_flush_size": "524288", 2024-09-14T06:16:18.495 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_min_log_runway": "1048576", 2024-09-14T06:16:18.495 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_replay_recovery": "false", 2024-09-14T06:16:18.495 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-14T06:16:18.495 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-14T06:16:18.495 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_sync_write": "false", 2024-09-14T06:16:18.495 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-14T06:16:18.496 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-14T06:16:18.496 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-14T06:16:18.496 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_allocation_from_file": "true", 2024-09-14T06:16:18.496 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_allocator": "hybrid", 2024-09-14T06:16:18.496 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_async_db_compaction": "true", 2024-09-14T06:16:18.496 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-14T06:16:18.496 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-14T06:16:18.496 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-14T06:16:18.496 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-14T06:16:18.497 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_bdev_label_multi": "true", 2024-09-14T06:16:18.497 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-14T06:16:18.497 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-14T06:16:18.497 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-14T06:16:18.497 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-14T06:16:18.497 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-14T06:16:18.497 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_create": "true", 2024-09-14T06:16:18.497 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_db_create": "false", 2024-09-14T06:16:18.497 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_db_path": "", 2024-09-14T06:16:18.497 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_db_size": "0", 2024-09-14T06:16:18.497 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_path": "", 2024-09-14T06:16:18.498 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_preallocate_file": "false", 2024-09-14T06:16:18.498 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_size": "107374182400", 2024-09-14T06:16:18.498 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_wal_create": "false", 2024-09-14T06:16:18.498 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_wal_path": "", 2024-09-14T06:16:18.498 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_wal_size": "100663296", 2024-09-14T06:16:18.498 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_bluefs": "true", 2024-09-14T06:16:18.498 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-14T06:16:18.498 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-14T06:16:18.498 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-14T06:16:18.498 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-14T06:16:18.498 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-14T06:16:18.498 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T06:16:18.498 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T06:16:18.498 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-14T06:16:18.498 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T06:16:18.499 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_autotune": "true", 2024-09-14T06:16:18.499 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-14T06:16:18.499 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-14T06:16:18.499 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-14T06:16:18.499 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-14T06:16:18.499 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_size": "0", 2024-09-14T06:16:18.499 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-14T06:16:18.499 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-14T06:16:18.499 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-14T06:16:18.499 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-14T06:16:18.499 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_type": "2q", 2024-09-14T06:16:18.500 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-14T06:16:18.500 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_clone_cow": "true", 2024-09-14T06:16:18.500 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-14T06:16:18.500 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-14T06:16:18.500 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-14T06:16:18.500 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-14T06:16:18.500 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-14T06:16:18.500 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-14T06:16:18.500 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-14T06:16:18.500 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_mode": "none", 2024-09-14T06:16:18.500 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-14T06:16:18.501 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_csum_type": "crc32c", 2024-09-14T06:16:18.501 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-14T06:16:18.501 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-14T06:16:18.501 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_freelist": "false", 2024-09-14T06:16:18.501 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-14T06:16:18.501 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-14T06:16:18.501 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-14T06:16:18.501 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-14T06:16:18.501 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-14T06:16:18.501 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-14T06:16:18.501 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-14T06:16:18.501 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-14T06:16:18.501 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-14T06:16:18.501 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-14T06:16:18.502 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-14T06:16:18.502 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-14T06:16:18.502 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_small_allocations": "0", 2024-09-14T06:16:18.502 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-14T06:16:18.502 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_default_buffered_read": "true", 2024-09-14T06:16:18.502 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_default_buffered_write": "false", 2024-09-14T06:16:18.502 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-14T06:16:18.502 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-14T06:16:18.502 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-14T06:16:18.502 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-14T06:16:18.502 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-14T06:16:18.503 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-14T06:16:18.503 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-14T06:16:18.503 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-14T06:16:18.503 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-14T06:16:18.503 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fail_eio": "false", 2024-09-14T06:16:18.503 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-14T06:16:18.503 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-14T06:16:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-14T06:16:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-14T06:16:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-14T06:16:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-14T06:16:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_on_mount": "false", 2024-09-14T06:16:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-14T06:16:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_on_umount": "false", 2024-09-14T06:16:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-14T06:16:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-14T06:16:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-14T06:16:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-14T06:16:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-14T06:16:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-14T06:16:18.505 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-14T06:16:18.505 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-14T06:16:18.505 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_ignore_data_csum": "false", 2024-09-14T06:16:18.505 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-14T06:16:18.505 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-14T06:16:18.505 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-14T06:16:18.505 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-14T06:16:18.505 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_log_op_age": "5.000000", 2024-09-14T06:16:18.505 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_max_alloc_size": "0", 2024-09-14T06:16:18.505 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_max_blob_size": "0", 2024-09-14T06:16:18.506 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-14T06:16:18.506 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-14T06:16:18.506 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-14T06:16:18.506 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_max_deferred_txc": "32", 2024-09-14T06:16:18.506 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_min_alloc_size": "0", 2024-09-14T06:16:18.506 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-14T06:16:18.506 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-14T06:16:18.506 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_nid_prealloc": "1024", 2024-09-14T06:16:18.506 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-14T06:16:18.506 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-14T06:16:18.506 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-14T06:16:18.506 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-14T06:16:18.506 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_retry_disk_reads": "3", 2024-09-14T06:16:18.506 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_rocksdb_cf": "true", 2024-09-14T06:16:18.506 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-14T06:16:18.507 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-14T06:16:18.507 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-14T06:16:18.507 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-14T06:16:18.507 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-14T06:16:18.507 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-14T06:16:18.507 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-14T06:16:18.507 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-14T06:16:18.507 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_spdk_mem": "512", 2024-09-14T06:16:18.507 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-14T06:16:18.507 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-14T06:16:18.507 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-14T06:16:18.507 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-14T06:16:18.508 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-14T06:16:18.508 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-14T06:16:18.508 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-14T06:16:18.508 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_tracing": "false", 2024-09-14T06:16:18.508 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-14T06:16:18.508 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-14T06:16:18.508 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-14T06:16:18.508 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-14T06:16:18.508 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-14T06:16:18.508 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-14T06:16:18.508 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-14T06:16:18.508 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-14T06:16:18.509 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-14T06:16:18.509 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_write_v2": "false", 2024-09-14T06:16:18.509 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_write_v2_random": "false", 2024-09-14T06:16:18.509 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_zero_block_detection": "false", 2024-09-14T06:16:18.509 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-14T06:16:18.509 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-14T06:16:18.509 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-14T06:16:18.509 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-14T06:16:18.509 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-14T06:16:18.509 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-14T06:16:18.509 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-14T06:16:18.509 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-14T06:16:18.510 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-14T06:16:18.510 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-14T06:16:18.510 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-14T06:16:18.510 INFO:teuthology.orchestra.run.smithi039.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-14T06:16:18.510 INFO:teuthology.orchestra.run.smithi039.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-14T06:16:18.510 INFO:teuthology.orchestra.run.smithi039.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-14T06:16:18.510 INFO:teuthology.orchestra.run.smithi039.stdout: "cephx_cluster_require_signatures": "false", 2024-09-14T06:16:18.510 INFO:teuthology.orchestra.run.smithi039.stdout: "cephx_cluster_require_version": "2", 2024-09-14T06:16:18.510 INFO:teuthology.orchestra.run.smithi039.stdout: "cephx_require_signatures": "false", 2024-09-14T06:16:18.510 INFO:teuthology.orchestra.run.smithi039.stdout: "cephx_require_version": "2", 2024-09-14T06:16:18.510 INFO:teuthology.orchestra.run.smithi039.stdout: "cephx_service_require_signatures": "false", 2024-09-14T06:16:18.510 INFO:teuthology.orchestra.run.smithi039.stdout: "cephx_service_require_version": "2", 2024-09-14T06:16:18.510 INFO:teuthology.orchestra.run.smithi039.stdout: "cephx_sign_messages": "true", 2024-09-14T06:16:18.510 INFO:teuthology.orchestra.run.smithi039.stdout: "chdir": "", 2024-09-14T06:16:18.510 INFO:teuthology.orchestra.run.smithi039.stdout: "client_acl_type": "", 2024-09-14T06:16:18.511 INFO:teuthology.orchestra.run.smithi039.stdout: "client_asio_thread_count": "2", 2024-09-14T06:16:18.511 INFO:teuthology.orchestra.run.smithi039.stdout: "client_cache_mid": "0.750000", 2024-09-14T06:16:18.511 INFO:teuthology.orchestra.run.smithi039.stdout: "client_cache_size": "16384", 2024-09-14T06:16:18.511 INFO:teuthology.orchestra.run.smithi039.stdout: "client_caps_release_delay": "5", 2024-09-14T06:16:18.511 INFO:teuthology.orchestra.run.smithi039.stdout: "client_check_pool_perm": "true", 2024-09-14T06:16:18.511 INFO:teuthology.orchestra.run.smithi039.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-14T06:16:18.511 INFO:teuthology.orchestra.run.smithi039.stdout: "client_debug_force_sync_read": "false", 2024-09-14T06:16:18.511 INFO:teuthology.orchestra.run.smithi039.stdout: "client_debug_getattr_caps": "false", 2024-09-14T06:16:18.511 INFO:teuthology.orchestra.run.smithi039.stdout: "client_debug_inject_features": "", 2024-09-14T06:16:18.511 INFO:teuthology.orchestra.run.smithi039.stdout: "client_debug_inject_tick_delay": "0", 2024-09-14T06:16:18.511 INFO:teuthology.orchestra.run.smithi039.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-14T06:16:18.511 INFO:teuthology.orchestra.run.smithi039.stdout: "client_die_on_failed_remount": "false", 2024-09-14T06:16:18.512 INFO:teuthology.orchestra.run.smithi039.stdout: "client_dirsize_rbytes": "true", 2024-09-14T06:16:18.512 INFO:teuthology.orchestra.run.smithi039.stdout: "client_force_lazyio": "false", 2024-09-14T06:16:18.512 INFO:teuthology.orchestra.run.smithi039.stdout: "client_fs": "", 2024-09-14T06:16:18.512 INFO:teuthology.orchestra.run.smithi039.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-14T06:16:18.512 INFO:teuthology.orchestra.run.smithi039.stdout: "client_inject_release_failure": "false", 2024-09-14T06:16:18.512 INFO:teuthology.orchestra.run.smithi039.stdout: "client_max_inline_size": "4096", 2024-09-14T06:16:18.512 INFO:teuthology.orchestra.run.smithi039.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-14T06:16:18.512 INFO:teuthology.orchestra.run.smithi039.stdout: "client_mds_namespace": "", 2024-09-14T06:16:18.512 INFO:teuthology.orchestra.run.smithi039.stdout: "client_metadata": "", 2024-09-14T06:16:18.512 INFO:teuthology.orchestra.run.smithi039.stdout: "client_mount_gid": "-1", 2024-09-14T06:16:18.513 INFO:teuthology.orchestra.run.smithi039.stdout: "client_mount_timeout": "300", 2024-09-14T06:16:18.513 INFO:teuthology.orchestra.run.smithi039.stdout: "client_mount_uid": "-1", 2024-09-14T06:16:18.513 INFO:teuthology.orchestra.run.smithi039.stdout: "client_mountpoint": "/", 2024-09-14T06:16:18.513 INFO:teuthology.orchestra.run.smithi039.stdout: "client_notify_timeout": "10", 2024-09-14T06:16:18.513 INFO:teuthology.orchestra.run.smithi039.stdout: "client_oc": "true", 2024-09-14T06:16:18.513 INFO:teuthology.orchestra.run.smithi039.stdout: "client_oc_max_dirty": "104857600", 2024-09-14T06:16:18.513 INFO:teuthology.orchestra.run.smithi039.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-14T06:16:18.513 INFO:teuthology.orchestra.run.smithi039.stdout: "client_oc_max_objects": "1000", 2024-09-14T06:16:18.513 INFO:teuthology.orchestra.run.smithi039.stdout: "client_oc_size": "209715200", 2024-09-14T06:16:18.513 INFO:teuthology.orchestra.run.smithi039.stdout: "client_oc_target_dirty": "8388608", 2024-09-14T06:16:18.513 INFO:teuthology.orchestra.run.smithi039.stdout: "client_permissions": "true", 2024-09-14T06:16:18.513 INFO:teuthology.orchestra.run.smithi039.stdout: "client_quota": "true", 2024-09-14T06:16:18.513 INFO:teuthology.orchestra.run.smithi039.stdout: "client_quota_df": "true", 2024-09-14T06:16:18.514 INFO:teuthology.orchestra.run.smithi039.stdout: "client_readahead_max_bytes": "0", 2024-09-14T06:16:18.514 INFO:teuthology.orchestra.run.smithi039.stdout: "client_readahead_max_periods": "4", 2024-09-14T06:16:18.514 INFO:teuthology.orchestra.run.smithi039.stdout: "client_readahead_min": "131072", 2024-09-14T06:16:18.514 INFO:teuthology.orchestra.run.smithi039.stdout: "client_reconnect_stale": "false", 2024-09-14T06:16:18.514 INFO:teuthology.orchestra.run.smithi039.stdout: "client_shutdown_timeout": "30", 2024-09-14T06:16:18.514 INFO:teuthology.orchestra.run.smithi039.stdout: "client_snapdir": ".snap", 2024-09-14T06:16:18.514 INFO:teuthology.orchestra.run.smithi039.stdout: "client_tick_interval": "1", 2024-09-14T06:16:18.514 INFO:teuthology.orchestra.run.smithi039.stdout: "client_trace": "", 2024-09-14T06:16:18.514 INFO:teuthology.orchestra.run.smithi039.stdout: "client_try_dentry_invalidate": "false", 2024-09-14T06:16:18.514 INFO:teuthology.orchestra.run.smithi039.stdout: "client_use_faked_inos": "false", 2024-09-14T06:16:18.514 INFO:teuthology.orchestra.run.smithi039.stdout: "client_use_random_mds": "false", 2024-09-14T06:16:18.514 INFO:teuthology.orchestra.run.smithi039.stdout: "clog_to_graylog": "false", 2024-09-14T06:16:18.514 INFO:teuthology.orchestra.run.smithi039.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-14T06:16:18.514 INFO:teuthology.orchestra.run.smithi039.stdout: "clog_to_graylog_port": "12201", 2024-09-14T06:16:18.514 INFO:teuthology.orchestra.run.smithi039.stdout: "clog_to_monitors": "default=true", 2024-09-14T06:16:18.515 INFO:teuthology.orchestra.run.smithi039.stdout: "clog_to_syslog": "false", 2024-09-14T06:16:18.515 INFO:teuthology.orchestra.run.smithi039.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-14T06:16:18.515 INFO:teuthology.orchestra.run.smithi039.stdout: "clog_to_syslog_level": "info", 2024-09-14T06:16:18.515 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "-", 2024-09-14T06:16:18.515 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_network": "", 2024-09-14T06:16:18.515 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_network_interface": "", 2024-09-14T06:16:18.515 INFO:teuthology.orchestra.run.smithi039.stdout: "compressor_zlib_isal": "false", 2024-09-14T06:16:18.515 INFO:teuthology.orchestra.run.smithi039.stdout: "compressor_zlib_level": "5", 2024-09-14T06:16:18.515 INFO:teuthology.orchestra.run.smithi039.stdout: "compressor_zlib_winsize": "-15", 2024-09-14T06:16:18.515 INFO:teuthology.orchestra.run.smithi039.stdout: "compressor_zstd_level": "1", 2024-09-14T06:16:18.515 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-14T06:16:18.515 INFO:teuthology.orchestra.run.smithi039.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-14T06:16:18.516 INFO:teuthology.orchestra.run.smithi039.stdout: "crimson_alien_op_num_threads": "6", 2024-09-14T06:16:18.516 INFO:teuthology.orchestra.run.smithi039.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-14T06:16:18.516 INFO:teuthology.orchestra.run.smithi039.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-14T06:16:18.516 INFO:teuthology.orchestra.run.smithi039.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-14T06:16:18.516 INFO:teuthology.orchestra.run.smithi039.stdout: "crimson_osd_stat_interval": "0", 2024-09-14T06:16:18.516 INFO:teuthology.orchestra.run.smithi039.stdout: "crimson_seastar_cpu_cores": "", 2024-09-14T06:16:18.516 INFO:teuthology.orchestra.run.smithi039.stdout: "crimson_seastar_num_threads": "0", 2024-09-14T06:16:18.516 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_location": "", 2024-09-14T06:16:18.516 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_location_hook": "", 2024-09-14T06:16:18.516 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_location_hook_timeout": "10", 2024-09-14T06:16:18.516 INFO:teuthology.orchestra.run.smithi039.stdout: "daemonize": "false", 2024-09-14T06:16:18.516 INFO:teuthology.orchestra.run.smithi039.stdout: "daos_pool": "tank", 2024-09-14T06:16:18.516 INFO:teuthology.orchestra.run.smithi039.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-14T06:16:18.516 INFO:teuthology.orchestra.run.smithi039.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-14T06:16:18.516 INFO:teuthology.orchestra.run.smithi039.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-14T06:16:18.517 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_alienstore": "0/5", 2024-09-14T06:16:18.517 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_allow_any_pool_priority": "false", 2024-09-14T06:16:18.517 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_asok": "1/5", 2024-09-14T06:16:18.517 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_asok_assert_abort": "false", 2024-09-14T06:16:18.517 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_asserts_on_shutdown": "true", 2024-09-14T06:16:18.517 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_auth": "20/20", 2024-09-14T06:16:18.517 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_bdev": "1/3", 2024-09-14T06:16:18.517 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_bluefs": "1/5", 2024-09-14T06:16:18.517 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_bluestore": "1/5", 2024-09-14T06:16:18.517 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_buffer": "0/1", 2024-09-14T06:16:18.517 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_ceph_dedup": "0/5", 2024-09-14T06:16:18.517 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_ceph_exporter": "1/5", 2024-09-14T06:16:18.517 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_cephfs_mirror": "0/5", 2024-09-14T06:16:18.517 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_cephsqlite": "0/5", 2024-09-14T06:16:18.518 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_client": "0/5", 2024-09-14T06:16:18.518 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_compressor": "1/5", 2024-09-14T06:16:18.518 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_context": "0/1", 2024-09-14T06:16:18.518 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_crimson_interrupt": "0/5", 2024-09-14T06:16:18.518 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_crush": "1/1", 2024-09-14T06:16:18.518 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_crypto": "1/5", 2024-09-14T06:16:18.518 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_cyanstore": "0/5", 2024-09-14T06:16:18.518 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_deliberately_leak_memory": "false", 2024-09-14T06:16:18.518 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_disable_randomized_ping": "false", 2024-09-14T06:16:18.518 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_dpdk": "1/5", 2024-09-14T06:16:18.518 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_eventtrace": "1/5", 2024-09-14T06:16:18.518 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_filer": "0/1", 2024-09-14T06:16:18.518 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_filestore": "1/3", 2024-09-14T06:16:18.518 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_finisher": "1/1", 2024-09-14T06:16:18.518 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_fuse": "1/5", 2024-09-14T06:16:18.519 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_heartbeat_testing_span": "0", 2024-09-14T06:16:18.519 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_heartbeatmap": "1/5", 2024-09-14T06:16:18.519 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-14T06:16:18.519 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_javaclient": "1/5", 2024-09-14T06:16:18.519 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_journal": "1/3", 2024-09-14T06:16:18.519 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_journaler": "0/5", 2024-09-14T06:16:18.519 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_kstore": "1/5", 2024-09-14T06:16:18.519 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_lockdep": "0/1", 2024-09-14T06:16:18.519 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mclock": "1/5", 2024-09-14T06:16:18.519 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mds": "1/5", 2024-09-14T06:16:18.519 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mds_balancer": "1/5", 2024-09-14T06:16:18.519 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mds_locker": "1/5", 2024-09-14T06:16:18.519 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mds_log": "1/5", 2024-09-14T06:16:18.520 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mds_log_expire": "1/5", 2024-09-14T06:16:18.520 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mds_migrator": "1/5", 2024-09-14T06:16:18.520 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mds_quiesce": "3/5", 2024-09-14T06:16:18.520 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_memstore": "1/5", 2024-09-14T06:16:18.520 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mgr": "2/5", 2024-09-14T06:16:18.520 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mgrc": "1/5", 2024-09-14T06:16:18.520 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mon": "20/20", 2024-09-14T06:16:18.520 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_monc": "0/10", 2024-09-14T06:16:18.520 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_ms": "1/1", 2024-09-14T06:16:18.520 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_none": "0/5", 2024-09-14T06:16:18.520 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_objclass": "0/5", 2024-09-14T06:16:18.521 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_objectcacher": "0/5", 2024-09-14T06:16:18.521 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_objecter": "0/1", 2024-09-14T06:16:18.521 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_optracker": "0/5", 2024-09-14T06:16:18.521 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_osd": "1/5", 2024-09-14T06:16:18.521 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_paxos": "20/20", 2024-09-14T06:16:18.521 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_perfcounter": "1/5", 2024-09-14T06:16:18.521 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_prioritycache": "1/5", 2024-09-14T06:16:18.521 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rados": "0/5", 2024-09-14T06:16:18.521 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rbd": "0/5", 2024-09-14T06:16:18.521 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rbd_mirror": "0/5", 2024-09-14T06:16:18.521 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rbd_pwl": "0/5", 2024-09-14T06:16:18.521 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rbd_replay": "0/5", 2024-09-14T06:16:18.521 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_refs": "0/0", 2024-09-14T06:16:18.521 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_reserver": "1/1", 2024-09-14T06:16:18.521 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rgw": "1/5", 2024-09-14T06:16:18.522 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rgw_access": "1/5", 2024-09-14T06:16:18.522 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rgw_datacache": "1/5", 2024-09-14T06:16:18.522 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rgw_dbstore": "1/5", 2024-09-14T06:16:18.522 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rgw_flight": "1/5", 2024-09-14T06:16:18.522 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-14T06:16:18.522 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rgw_notification": "1/5", 2024-09-14T06:16:18.522 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rgw_sync": "1/5", 2024-09-14T06:16:18.522 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rocksdb": "4/5", 2024-09-14T06:16:18.522 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore": "0/5", 2024-09-14T06:16:18.522 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_backref": "0/5", 2024-09-14T06:16:18.522 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_cache": "0/5", 2024-09-14T06:16:18.522 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_cleaner": "0/5", 2024-09-14T06:16:18.523 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_device": "0/5", 2024-09-14T06:16:18.523 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_epm": "0/5", 2024-09-14T06:16:18.523 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-14T06:16:18.523 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_journal": "0/5", 2024-09-14T06:16:18.523 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_lba": "0/5", 2024-09-14T06:16:18.523 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_odata": "0/5", 2024-09-14T06:16:18.523 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_omap": "0/5", 2024-09-14T06:16:18.523 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_onode": "0/5", 2024-09-14T06:16:18.523 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_t": "0/5", 2024-09-14T06:16:18.523 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_tm": "0/5", 2024-09-14T06:16:18.523 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_striper": "0/1", 2024-09-14T06:16:18.523 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_test": "0/5", 2024-09-14T06:16:18.523 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_throttle": "1/1", 2024-09-14T06:16:18.524 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_timer": "0/1", 2024-09-14T06:16:18.524 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_tp": "0/5", 2024-09-14T06:16:18.524 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_trace": "1/5", 2024-09-14T06:16:18.524 INFO:teuthology.orchestra.run.smithi039.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-14T06:16:18.524 INFO:teuthology.orchestra.run.smithi039.stdout: "device_failure_prediction_mode": "none", 2024-09-14T06:16:18.524 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-14T06:16:18.524 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-14T06:16:18.524 INFO:teuthology.orchestra.run.smithi039.stdout: "err_to_graylog": "false", 2024-09-14T06:16:18.524 INFO:teuthology.orchestra.run.smithi039.stdout: "err_to_journald": "false", 2024-09-14T06:16:18.524 INFO:teuthology.orchestra.run.smithi039.stdout: "err_to_stderr": "true", 2024-09-14T06:16:18.524 INFO:teuthology.orchestra.run.smithi039.stdout: "err_to_syslog": "false", 2024-09-14T06:16:18.524 INFO:teuthology.orchestra.run.smithi039.stdout: "event_tracing": "false", 2024-09-14T06:16:18.524 INFO:teuthology.orchestra.run.smithi039.stdout: "exporter_addr": "0.0.0.0", 2024-09-14T06:16:18.524 INFO:teuthology.orchestra.run.smithi039.stdout: "exporter_cert_file": "", 2024-09-14T06:16:18.525 INFO:teuthology.orchestra.run.smithi039.stdout: "exporter_http_port": "9926", 2024-09-14T06:16:18.525 INFO:teuthology.orchestra.run.smithi039.stdout: "exporter_key_file": "", 2024-09-14T06:16:18.525 INFO:teuthology.orchestra.run.smithi039.stdout: "exporter_prio_limit": "5", 2024-09-14T06:16:18.525 INFO:teuthology.orchestra.run.smithi039.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-14T06:16:18.525 INFO:teuthology.orchestra.run.smithi039.stdout: "exporter_sort_metrics": "true", 2024-09-14T06:16:18.525 INFO:teuthology.orchestra.run.smithi039.stdout: "exporter_stats_period": "5", 2024-09-14T06:16:18.525 INFO:teuthology.orchestra.run.smithi039.stdout: "fake_statfs_for_testing": "0", 2024-09-14T06:16:18.525 INFO:teuthology.orchestra.run.smithi039.stdout: "fatal_signal_handlers": "true", 2024-09-14T06:16:18.525 INFO:teuthology.orchestra.run.smithi039.stdout: "filer_max_purge_ops": "10", 2024-09-14T06:16:18.525 INFO:teuthology.orchestra.run.smithi039.stdout: "filer_max_truncate_ops": "128", 2024-09-14T06:16:18.525 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_apply_finisher_threads": "1", 2024-09-14T06:16:18.525 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_blackhole": "false", 2024-09-14T06:16:18.525 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_btrfs_clone_range": "true", 2024-09-14T06:16:18.525 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_btrfs_snap": "true", 2024-09-14T06:16:18.526 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_caller_concurrency": "10", 2024-09-14T06:16:18.526 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_collect_device_partition_information": "true", 2024-09-14T06:16:18.526 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_commit_timeout": "600.000000", 2024-09-14T06:16:18.526 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_debug_inject_read_err": "false", 2024-09-14T06:16:18.526 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_debug_omap_check": "false", 2024-09-14T06:16:18.526 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_debug_verify_split": "false", 2024-09-14T06:16:18.526 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_dump_file": "", 2024-09-14T06:16:18.526 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-14T06:16:18.526 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-14T06:16:18.526 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_fadvise": "true", 2024-09-14T06:16:18.527 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_fail_eio": "true", 2024-09-14T06:16:18.527 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_fd_cache_shards": "16", 2024-09-14T06:16:18.527 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_fd_cache_size": "128", 2024-09-14T06:16:18.527 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_fiemap": "false", 2024-09-14T06:16:18.527 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_fiemap_threshold": "4096", 2024-09-14T06:16:18.527 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-14T06:16:18.527 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-14T06:16:18.527 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_inject_stall": "0", 2024-09-14T06:16:18.527 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_journal_parallel": "false", 2024-09-14T06:16:18.527 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_journal_trailing": "false", 2024-09-14T06:16:18.527 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_journal_writeahead": "false", 2024-09-14T06:16:18.527 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_kill_at": "0", 2024-09-14T06:16:18.528 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-14T06:16:18.528 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-14T06:16:18.528 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-14T06:16:18.528 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-14T06:16:18.528 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-14T06:16:18.528 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_inline_xattrs": "0", 2024-09-14T06:16:18.528 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-14T06:16:18.528 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-14T06:16:18.528 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-14T06:16:18.528 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-14T06:16:18.528 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_xattr_value_size": "0", 2024-09-14T06:16:18.528 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-14T06:16:18.528 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-14T06:16:18.528 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-14T06:16:18.529 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_merge_threshold": "-10", 2024-09-14T06:16:18.529 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-14T06:16:18.529 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_odsync_write": "false", 2024-09-14T06:16:18.529 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_omap_backend": "rocksdb", 2024-09-14T06:16:18.529 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_omap_backend_path": "", 2024-09-14T06:16:18.529 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-14T06:16:18.529 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-14T06:16:18.529 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-14T06:16:18.529 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_op_thread_timeout": "60", 2024-09-14T06:16:18.529 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_op_threads": "2", 2024-09-14T06:16:18.529 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_punch_hole": "false", 2024-09-14T06:16:18.529 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-14T06:16:18.529 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-14T06:16:18.530 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-14T06:16:18.530 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-14T06:16:18.530 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-14T06:16:18.530 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-14T06:16:18.530 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-14T06:16:18.530 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-14T06:16:18.530 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-14T06:16:18.530 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_max_ops": "50", 2024-09-14T06:16:18.530 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-14T06:16:18.530 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_seek_data_hole": "false", 2024-09-14T06:16:18.530 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_sloppy_crc": "false", 2024-09-14T06:16:18.530 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-14T06:16:18.530 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_splice": "false", 2024-09-14T06:16:18.530 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_split_multiple": "2", 2024-09-14T06:16:18.531 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_split_rand_factor": "20", 2024-09-14T06:16:18.531 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_update_to": "1000", 2024-09-14T06:16:18.531 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-14T06:16:18.531 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-14T06:16:18.531 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-14T06:16:18.531 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-14T06:16:18.531 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-14T06:16:18.531 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-14T06:16:18.531 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_enable": "true", 2024-09-14T06:16:18.531 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-14T06:16:18.531 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-14T06:16:18.531 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-14T06:16:18.531 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-14T06:16:18.531 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-14T06:16:18.532 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-14T06:16:18.532 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_xfs_extsize": "false", 2024-09-14T06:16:18.532 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_zfs_snap": "false", 2024-09-14T06:16:18.532 INFO:teuthology.orchestra.run.smithi039.stdout: "fio_dir": "/tmp/fio", 2024-09-14T06:16:18.532 INFO:teuthology.orchestra.run.smithi039.stdout: "fsid": "038563b1-268f-461f-b6a6-4b4b41ad0236", 2024-09-14T06:16:18.532 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_allow_other": "true", 2024-09-14T06:16:18.532 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_atomic_o_trunc": "true", 2024-09-14T06:16:18.532 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_big_writes": "true", 2024-09-14T06:16:18.532 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_debug": "false", 2024-09-14T06:16:18.532 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_default_permissions": "false", 2024-09-14T06:16:18.532 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_disable_pagecache": "false", 2024-09-14T06:16:18.532 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_max_write": "0", 2024-09-14T06:16:18.532 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_multithreaded": "true", 2024-09-14T06:16:18.532 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_require_active_mds": "true", 2024-09-14T06:16:18.533 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_set_user_groups": "true", 2024-09-14T06:16:18.533 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_splice_move": "true", 2024-09-14T06:16:18.533 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_splice_read": "true", 2024-09-14T06:16:18.533 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_splice_write": "true", 2024-09-14T06:16:18.533 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-14T06:16:18.533 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_use_invalidate_cb": "true", 2024-09-14T06:16:18.533 INFO:teuthology.orchestra.run.smithi039.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-14T06:16:18.533 INFO:teuthology.orchestra.run.smithi039.stdout: "gss_target_name": "ceph", 2024-09-14T06:16:18.533 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_file": "", 2024-09-14T06:16:18.533 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_inject_failure": "0", 2024-09-14T06:16:18.533 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_interval": "5", 2024-09-14T06:16:18.534 INFO:teuthology.orchestra.run.smithi039.stdout: "host": "smithi039", 2024-09-14T06:16:18.534 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-14T06:16:18.534 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-14T06:16:18.534 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-14T06:16:18.534 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-14T06:16:18.534 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-14T06:16:18.534 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-14T06:16:18.534 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-14T06:16:18.534 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-14T06:16:18.534 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-14T06:16:18.534 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-14T06:16:18.535 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-14T06:16:18.535 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-14T06:16:18.535 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-14T06:16:18.535 INFO:teuthology.orchestra.run.smithi039.stdout: "inject_early_sigterm": "false", 2024-09-14T06:16:18.535 INFO:teuthology.orchestra.run.smithi039.stdout: "jaeger_agent_port": "6799", 2024-09-14T06:16:18.535 INFO:teuthology.orchestra.run.smithi039.stdout: "jaeger_tracing_enable": "false", 2024-09-14T06:16:18.535 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_aio": "true", 2024-09-14T06:16:18.535 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_align_min_size": "65536", 2024-09-14T06:16:18.535 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_block_align": "true", 2024-09-14T06:16:18.536 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_block_size": "4096", 2024-09-14T06:16:18.536 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_dio": "true", 2024-09-14T06:16:18.536 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_discard": "false", 2024-09-14T06:16:18.536 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_force_aio": "false", 2024-09-14T06:16:18.536 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_ignore_corruption": "false", 2024-09-14T06:16:18.536 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_max_write_bytes": "10485760", 2024-09-14T06:16:18.537 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_max_write_entries": "100", 2024-09-14T06:16:18.537 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_replay_from": "0", 2024-09-14T06:16:18.537 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-14T06:16:18.537 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-14T06:16:18.537 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-14T06:16:18.537 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-14T06:16:18.537 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_write_header_frequency": "0", 2024-09-14T06:16:18.537 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_zero_on_create": "false", 2024-09-14T06:16:18.538 INFO:teuthology.orchestra.run.smithi039.stdout: "journaler_prefetch_periods": "10", 2024-09-14T06:16:18.538 INFO:teuthology.orchestra.run.smithi039.stdout: "journaler_prezero_periods": "5", 2024-09-14T06:16:18.538 INFO:teuthology.orchestra.run.smithi039.stdout: "journaler_write_head_interval": "15", 2024-09-14T06:16:18.538 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-14T06:16:18.538 INFO:teuthology.orchestra.run.smithi039.stdout: "keyfile": "", 2024-09-14T06:16:18.538 INFO:teuthology.orchestra.run.smithi039.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-14T06:16:18.538 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_backend": "rocksdb", 2024-09-14T06:16:18.538 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_default_stripe_size": "65536", 2024-09-14T06:16:18.538 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_fsck_on_mount": "false", 2024-09-14T06:16:18.538 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-14T06:16:18.538 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_max_bytes": "67108864", 2024-09-14T06:16:18.538 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_max_ops": "512", 2024-09-14T06:16:18.538 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_nid_prealloc": "1024", 2024-09-14T06:16:18.538 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_onode_map_size": "1024", 2024-09-14T06:16:18.539 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-14T06:16:18.539 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_sync_submit_transaction": "false", 2024-09-14T06:16:18.539 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_sync_transaction": "false", 2024-09-14T06:16:18.539 INFO:teuthology.orchestra.run.smithi039.stdout: "librados_thread_count": "2", 2024-09-14T06:16:18.539 INFO:teuthology.orchestra.run.smithi039.stdout: "lockdep": "false", 2024-09-14T06:16:18.539 INFO:teuthology.orchestra.run.smithi039.stdout: "lockdep_force_backtrace": "false", 2024-09-14T06:16:18.539 INFO:teuthology.orchestra.run.smithi039.stdout: "log_coarse_timestamps": "true", 2024-09-14T06:16:18.539 INFO:teuthology.orchestra.run.smithi039.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-14T06:16:18.539 INFO:teuthology.orchestra.run.smithi039.stdout: "log_flush_on_exit": "false", 2024-09-14T06:16:18.539 INFO:teuthology.orchestra.run.smithi039.stdout: "log_graylog_host": "127.0.0.1", 2024-09-14T06:16:18.539 INFO:teuthology.orchestra.run.smithi039.stdout: "log_graylog_port": "12201", 2024-09-14T06:16:18.539 INFO:teuthology.orchestra.run.smithi039.stdout: "log_max_new": "1000", 2024-09-14T06:16:18.539 INFO:teuthology.orchestra.run.smithi039.stdout: "log_max_recent": "10000", 2024-09-14T06:16:18.539 INFO:teuthology.orchestra.run.smithi039.stdout: "log_stderr_prefix": "", 2024-09-14T06:16:18.539 INFO:teuthology.orchestra.run.smithi039.stdout: "log_stop_at_utilization": "0.970000", 2024-09-14T06:16:18.540 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": "true", 2024-09-14T06:16:18.540 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_graylog": "false", 2024-09-14T06:16:18.540 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_journald": "false", 2024-09-14T06:16:18.540 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_stderr": "false", 2024-09-14T06:16:18.540 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_syslog": "false", 2024-09-14T06:16:18.540 INFO:teuthology.orchestra.run.smithi039.stdout: "max_rotating_auth_attempts": "10", 2024-09-14T06:16:18.540 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-14T06:16:18.540 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_action_on_write_error": "1", 2024-09-14T06:16:18.540 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_alternate_name_max": "8192", 2024-09-14T06:16:18.540 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_asio_thread_count": "2", 2024-09-14T06:16:18.540 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_export_pin": "true", 2024-09-14T06:16:18.540 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_fragment_dirs": "true", 2024-09-14T06:16:18.540 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-14T06:16:18.541 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_fragment_interval": "5", 2024-09-14T06:16:18.541 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-14T06:16:18.541 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-14T06:16:18.541 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_interval": "10", 2024-09-14T06:16:18.541 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_max": "-1", 2024-09-14T06:16:18.541 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_max_until": "-1", 2024-09-14T06:16:18.541 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_merge_size": "50", 2024-09-14T06:16:18.541 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_midchunk": "0.300000", 2024-09-14T06:16:18.541 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-14T06:16:18.541 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_min_start": "0.200000", 2024-09-14T06:16:18.541 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_minchunk": "0.001000", 2024-09-14T06:16:18.541 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_mode": "0", 2024-09-14T06:16:18.541 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_need_max": "1.200000", 2024-09-14T06:16:18.541 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_need_min": "0.800000", 2024-09-14T06:16:18.541 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_overload_epochs": "2", 2024-09-14T06:16:18.542 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-14T06:16:18.542 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-14T06:16:18.542 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_split_bits": "3", 2024-09-14T06:16:18.542 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-14T06:16:18.542 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_split_size": "10000", 2024-09-14T06:16:18.542 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-14T06:16:18.542 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_target_decay": "10.000000", 2024-09-14T06:16:18.542 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-14T06:16:18.542 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_beacon_grace": "15.000000", 2024-09-14T06:16:18.542 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_beacon_interval": "4.000000", 2024-09-14T06:16:18.542 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-14T06:16:18.542 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-14T06:16:18.542 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_mid": "0.700000", 2024-09-14T06:16:18.542 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-14T06:16:18.542 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_quiesce_delay": "0", 2024-09-14T06:16:18.543 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-14T06:16:18.543 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-14T06:16:18.543 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-14T06:16:18.543 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_release_free_interval": "10", 2024-09-14T06:16:18.543 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_reservation": "0.050000", 2024-09-14T06:16:18.543 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-14T06:16:18.543 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_trim_interval": "1", 2024-09-14T06:16:18.543 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_trim_threshold": "262144", 2024-09-14T06:16:18.543 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-14T06:16:18.543 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-14T06:16:18.543 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-14T06:16:18.543 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_client_prealloc_inos": "1000", 2024-09-14T06:16:18.543 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-14T06:16:18.543 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_connect_bootstrapping": "false", 2024-09-14T06:16:18.544 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_damage_table_max_entries": "10000", 2024-09-14T06:16:18.544 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-14T06:16:18.544 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_debug_auth_pins": "false", 2024-09-14T06:16:18.544 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_debug_frag": "false", 2024-09-14T06:16:18.544 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_debug_scatterstat": "false", 2024-09-14T06:16:18.544 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_debug_subtrees": "false", 2024-09-14T06:16:18.544 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_decay_halflife": "5.000000", 2024-09-14T06:16:18.544 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_default_dir_hash": "2", 2024-09-14T06:16:18.544 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_defer_session_stale": "true", 2024-09-14T06:16:18.544 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_deny_all_reconnect": "false", 2024-09-14T06:16:18.544 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dir_keys_per_op": "16384", 2024-09-14T06:16:18.544 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dir_max_commit_size": "10", 2024-09-14T06:16:18.544 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dir_max_entries": "0", 2024-09-14T06:16:18.544 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dir_prefetch": "true", 2024-09-14T06:16:18.545 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-14T06:16:18.545 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-14T06:16:18.545 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dump_cache_on_map": "false", 2024-09-14T06:16:18.545 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-14T06:16:18.545 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-14T06:16:18.545 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_early_reply": "true", 2024-09-14T06:16:18.545 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_enable_op_tracker": "true", 2024-09-14T06:16:18.545 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_enforce_unique_name": "true", 2024-09-14T06:16:18.545 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-14T06:16:18.545 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-14T06:16:18.545 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_export_ephemeral_random": "true", 2024-09-14T06:16:18.545 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-14T06:16:18.545 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-14T06:16:18.545 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-14T06:16:18.545 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-14T06:16:18.546 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-14T06:16:18.546 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-14T06:16:18.546 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-14T06:16:18.546 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-14T06:16:18.546 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_health_summarize_threshold": "10", 2024-09-14T06:16:18.546 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-14T06:16:18.546 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-14T06:16:18.546 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_inject_health_dummy": "false", 2024-09-14T06:16:18.546 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-14T06:16:18.546 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_inject_migrator_session_race": "false", 2024-09-14T06:16:18.546 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-14T06:16:18.546 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-14T06:16:18.546 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-14T06:16:18.546 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_join_fs": "", 2024-09-14T06:16:18.547 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_journal_format": "1", 2024-09-14T06:16:18.547 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-14T06:16:18.547 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_create_at": "0", 2024-09-14T06:16:18.547 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_dirfrag_at": "0", 2024-09-14T06:16:18.547 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_export_at": "0", 2024-09-14T06:16:18.547 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_import_at": "0", 2024-09-14T06:16:18.547 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_journal_at": "0", 2024-09-14T06:16:18.547 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_journal_expire_at": "0", 2024-09-14T06:16:18.547 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_journal_replay_at": "0", 2024-09-14T06:16:18.547 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_link_at": "0", 2024-09-14T06:16:18.548 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_mdstable_at": "0", 2024-09-14T06:16:18.548 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_openc_at": "0", 2024-09-14T06:16:18.548 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_rename_at": "0", 2024-09-14T06:16:18.548 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_shutdown_at": "0", 2024-09-14T06:16:18.548 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_event_large_threshold": "524288", 2024-09-14T06:16:18.548 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_events_per_segment": "1024", 2024-09-14T06:16:18.548 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-14T06:16:18.548 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_max_events": "-1", 2024-09-14T06:16:18.548 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_max_segments": "128", 2024-09-14T06:16:18.548 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_pause": "false", 2024-09-14T06:16:18.548 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_segment_size": "0", 2024-09-14T06:16:18.548 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-14T06:16:18.548 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-14T06:16:18.548 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-14T06:16:18.548 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_trim_threshold": "128", 2024-09-14T06:16:18.549 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-14T06:16:18.549 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_warn_factor": "2.000000", 2024-09-14T06:16:18.549 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_caps_per_client": "1048576", 2024-09-14T06:16:18.549 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_completed_flushes": "100000", 2024-09-14T06:16:18.549 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_completed_requests": "100000", 2024-09-14T06:16:18.549 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_export_size": "20971520", 2024-09-14T06:16:18.549 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_file_recover": "32", 2024-09-14T06:16:18.549 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_purge_files": "64", 2024-09-14T06:16:18.549 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_purge_ops": "8192", 2024-09-14T06:16:18.549 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-14T06:16:18.549 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-14T06:16:18.549 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_snaps_per_dir": "100", 2024-09-14T06:16:18.549 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_metrics_update_interval": "2", 2024-09-14T06:16:18.549 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_min_caps_per_client": "100", 2024-09-14T06:16:18.549 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_min_caps_working_set": "10000", 2024-09-14T06:16:18.550 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-14T06:16:18.550 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_numa_node": "-1", 2024-09-14T06:16:18.550 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-14T06:16:18.550 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_op_complaint_time": "30.000000", 2024-09-14T06:16:18.550 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_op_history_duration": "600", 2024-09-14T06:16:18.550 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_op_history_size": "20", 2024-09-14T06:16:18.550 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_op_history_slow_op_size": "20", 2024-09-14T06:16:18.550 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-14T06:16:18.550 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_op_log_threshold": "5", 2024-09-14T06:16:18.550 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_ping_grace": "15", 2024-09-14T06:16:18.550 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_ping_interval": "5", 2024-09-14T06:16:18.550 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-14T06:16:18.551 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-14T06:16:18.551 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_recall_max_caps": "30000", 2024-09-14T06:16:18.551 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-14T06:16:18.551 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-14T06:16:18.551 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-14T06:16:18.551 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_recall_warning_threshold": "262144", 2024-09-14T06:16:18.551 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-14T06:16:18.551 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_replay_interval": "1.000000", 2024-09-14T06:16:18.551 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-14T06:16:18.551 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-14T06:16:18.551 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_root_ino_gid": "0", 2024-09-14T06:16:18.551 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_root_ino_uid": "0", 2024-09-14T06:16:18.552 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-14T06:16:18.552 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_scrub_stats_review_period": "1", 2024-09-14T06:16:18.552 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-14T06:16:18.552 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-14T06:16:18.552 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-14T06:16:18.552 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-14T06:16:18.552 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-14T06:16:18.552 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-14T06:16:18.552 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-14T06:16:18.552 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-14T06:16:18.552 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-14T06:16:18.552 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-14T06:16:18.552 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-14T06:16:18.553 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_shutdown_check": "0", 2024-09-14T06:16:18.553 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_skip_ino": "0", 2024-09-14T06:16:18.553 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-14T06:16:18.553 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_snap_max_uid": "4294967294", 2024-09-14T06:16:18.553 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_snap_min_uid": "0", 2024-09-14T06:16:18.553 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_snap_rstat": "false", 2024-09-14T06:16:18.553 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_standby_replay_damaged": "false", 2024-09-14T06:16:18.553 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_symlink_recovery": "true", 2024-09-14T06:16:18.553 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-14T06:16:18.553 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_thrash_exports": "0", 2024-09-14T06:16:18.553 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_thrash_fragments": "0", 2024-09-14T06:16:18.553 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_tick_interval": "5.000000", 2024-09-14T06:16:18.554 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_valgrind_exit": "false", 2024-09-14T06:16:18.554 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_verify_backtrace": "1", 2024-09-14T06:16:18.554 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_verify_scatter": "false", 2024-09-14T06:16:18.554 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-14T06:16:18.554 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_wipe_sessions": "false", 2024-09-14T06:16:18.554 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool_debug": "false", 2024-09-14T06:16:18.554 INFO:teuthology.orchestra.run.smithi039.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-14T06:16:18.554 INFO:teuthology.orchestra.run.smithi039.stdout: "memstore_device_bytes": "1073741824", 2024-09-14T06:16:18.555 INFO:teuthology.orchestra.run.smithi039.stdout: "memstore_page_set": "false", 2024-09-14T06:16:18.555 INFO:teuthology.orchestra.run.smithi039.stdout: "memstore_page_size": "65536", 2024-09-14T06:16:18.555 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_client_bytes": "134217728", 2024-09-14T06:16:18.555 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_client_messages": "512", 2024-09-14T06:16:18.555 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-14T06:16:18.555 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-14T06:16:18.555 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-14T06:16:18.555 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-14T06:16:18.555 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_disabled_modules": "", 2024-09-14T06:16:18.555 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_enable_op_tracker": "true", 2024-09-14T06:16:18.555 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-14T06:16:18.555 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_max_pg_creating": "1024", 2024-09-14T06:16:18.555 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_max_pg_num_change": "128", 2024-09-14T06:16:18.555 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_mds_bytes": "134217728", 2024-09-14T06:16:18.556 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_mds_messages": "128", 2024-09-14T06:16:18.556 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-14T06:16:18.556 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_mon_bytes": "134217728", 2024-09-14T06:16:18.556 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_mon_messages": "128", 2024-09-14T06:16:18.556 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-14T06:16:18.556 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-14T06:16:18.556 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_op_history_duration": "600", 2024-09-14T06:16:18.556 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_op_history_size": "20", 2024-09-14T06:16:18.556 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-14T06:16:18.556 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-14T06:16:18.556 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_op_log_threshold": "5", 2024-09-14T06:16:18.556 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_osd_bytes": "536870912", 2024-09-14T06:16:18.556 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_osd_messages": "8192", 2024-09-14T06:16:18.556 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_pool": "true", 2024-09-14T06:16:18.557 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-14T06:16:18.557 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_standby_modules": "true", 2024-09-14T06:16:18.557 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_stats_period": "5", 2024-09-14T06:16:18.557 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_stats_threshold": "5", 2024-09-14T06:16:18.557 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_tick_period": "2", 2024-09-14T06:16:18.557 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-14T06:16:18.557 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-14T06:16:18.557 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_allow_pool_delete": "true", 2024-09-14T06:16:18.557 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_allow_pool_size_one": "true", 2024-09-14T06:16:18.557 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_auth_validate_all_caps": "true", 2024-09-14T06:16:18.558 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-14T06:16:18.558 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-14T06:16:18.558 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_bytes": "104857600", 2024-09-14T06:16:18.558 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_directed_command_retry": "5", 2024-09-14T06:16:18.558 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-14T06:16:18.558 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-14T06:16:18.558 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-14T06:16:18.558 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-14T06:16:18.558 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_hunt_parallel": "3", 2024-09-14T06:16:18.558 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_log_interval": "1.000000", 2024-09-14T06:16:18.559 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-14T06:16:18.559 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_ping_interval": "10.000000", 2024-09-14T06:16:18.559 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-14T06:16:18.559 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_target_rank": "-1", 2024-09-14T06:16:18.559 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-14T06:16:18.559 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-14T06:16:18.559 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-14T06:16:18.559 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_level": "debug", 2024-09-14T06:16:18.559 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_file": "true", 2024-09-14T06:16:18.559 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-14T06:16:18.559 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-14T06:16:18.559 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-14T06:16:18.560 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_journald": "false", 2024-09-14T06:16:18.560 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-14T06:16:18.560 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-14T06:16:18.560 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-14T06:16:18.560 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_compact_on_bootstrap": "false", 2024-09-14T06:16:18.560 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_compact_on_start": "false", 2024-09-14T06:16:18.560 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_compact_on_trim": "true", 2024-09-14T06:16:18.560 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-14T06:16:18.560 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-14T06:16:18.560 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-14T06:16:18.560 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cpu_threads": "4", 2024-09-14T06:16:18.561 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_crush_min_required_version": "hammer", 2024-09-14T06:16:18.561 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_daemon_bytes": "419430400", 2024-09-14T06:16:18.561 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-14T06:16:18.561 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_data_avail_crit": "5", 2024-09-14T06:16:18.561 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_data_avail_warn": "5", 2024-09-14T06:16:18.561 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_data_size_warn": "16106127360", 2024-09-14T06:16:18.561 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-14T06:16:18.561 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-14T06:16:18.561 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_dump_json": "false", 2024-09-14T06:16:18.561 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-14T06:16:18.561 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_dump_transactions": "false", 2024-09-14T06:16:18.561 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_extra_checks": "false", 2024-09-14T06:16:18.562 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-14T06:16:18.562 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-14T06:16:18.562 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_no_require_reef": "false", 2024-09-14T06:16:18.562 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_no_require_squid": "false", 2024-09-14T06:16:18.562 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-14T06:16:18.562 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-14T06:16:18.562 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-14T06:16:18.562 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_down_mkfs_grace": "120", 2024-09-14T06:16:18.562 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_down_uptime_grace": "60", 2024-09-14T06:16:18.562 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_election_default_strategy": "1", 2024-09-14T06:16:18.562 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_election_timeout": "5.000000", 2024-09-14T06:16:18.562 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-14T06:16:18.562 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_elector_ping_divisor": "2", 2024-09-14T06:16:18.562 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-14T06:16:18.562 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_enable_op_tracker": "true", 2024-09-14T06:16:18.563 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_fake_pool_delete": "false", 2024-09-14T06:16:18.563 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_force_quorum_join": "false", 2024-09-14T06:16:18.563 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-14T06:16:18.563 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_globalid_prealloc": "10000", 2024-09-14T06:16:18.563 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_health_detail_to_clog": "false", 2024-09-14T06:16:18.563 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_health_log_update_period": "5", 2024-09-14T06:16:18.563 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_health_max_detail": "50", 2024-09-14T06:16:18.563 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_health_to_clog": "true", 2024-09-14T06:16:18.563 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_health_to_clog_interval": "600", 2024-09-14T06:16:18.563 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-14T06:16:18.563 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_host": "172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790],172.21.15.87", 2024-09-14T06:16:18.563 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_host_override": "", 2024-09-14T06:16:18.563 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_initial_members": "", 2024-09-14T06:16:18.563 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-14T06:16:18.563 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-14T06:16:18.564 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-14T06:16:18.564 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-14T06:16:18.564 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-14T06:16:18.564 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_lease": "5.000000", 2024-09-14T06:16:18.564 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-14T06:16:18.564 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-14T06:16:18.564 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_log_full_interval": "50", 2024-09-14T06:16:18.564 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_log_max": "10000", 2024-09-14T06:16:18.564 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_log_max_summary": "50", 2024-09-14T06:16:18.564 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-14T06:16:18.564 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_log_epochs": "500", 2024-09-14T06:16:18.564 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-14T06:16:18.564 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-14T06:16:18.565 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-14T06:16:18.565 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_osd": "10000", 2024-09-14T06:16:18.565 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_pg_per_osd": "10000", 2024-09-14T06:16:18.565 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_pool_pg_num": "65536", 2024-09-14T06:16:18.565 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-14T06:16:18.565 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-14T06:16:18.565 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mds_force_trim_to": "0", 2024-09-14T06:16:18.565 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mds_skip_sanity": "false", 2024-09-14T06:16:18.565 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_memory_autotune": "true", 2024-09-14T06:16:18.565 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_memory_target": "2147483648", 2024-09-14T06:16:18.565 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mgr_beacon_grace": "30", 2024-09-14T06:16:18.565 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-14T06:16:18.565 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mgr_digest_period": "5", 2024-09-14T06:16:18.565 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mgr_inactive_grace": "60", 2024-09-14T06:16:18.566 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-14T06:16:18.566 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-14T06:16:18.566 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_min_osdmap_epochs": "500", 2024-09-14T06:16:18.566 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-14T06:16:18.566 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-14T06:16:18.566 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_op_complaint_time": "30", 2024-09-14T06:16:18.566 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_op_history_duration": "600", 2024-09-14T06:16:18.566 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_op_history_size": "20", 2024-09-14T06:16:18.566 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_op_history_slow_op_size": "20", 2024-09-14T06:16:18.566 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-14T06:16:18.566 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_op_log_threshold": "5", 2024-09-14T06:16:18.566 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-14T06:16:18.566 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-14T06:16:18.566 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-14T06:16:18.566 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_auto_mark_in": "false", 2024-09-14T06:16:18.566 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-14T06:16:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-14T06:16:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-14T06:16:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_cache_size": "500", 2024-09-14T06:16:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-14T06:16:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-14T06:16:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-14T06:16:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_down_out_interval": "600", 2024-09-14T06:16:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-14T06:16:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-14T06:16:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_force_trim_to": "0", 2024-09-14T06:16:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-14T06:16:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-14T06:16:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-14T06:16:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-14T06:16:18.567 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-14T06:16:18.568 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-14T06:16:18.568 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-14T06:16:18.568 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_min_down_reporters": "2", 2024-09-14T06:16:18.568 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-14T06:16:18.568 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-14T06:16:18.568 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-14T06:16:18.568 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-14T06:16:18.568 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-14T06:16:18.568 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-14T06:16:18.568 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_report_timeout": "900", 2024-09-14T06:16:18.568 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-14T06:16:18.569 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-14T06:16:18.569 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-14T06:16:18.569 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-14T06:16:18.569 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-14T06:16:18.569 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-14T06:16:18.569 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-14T06:16:18.569 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-14T06:16:18.569 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-14T06:16:18.569 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_pg_stuck_threshold": "60", 2024-09-14T06:16:18.569 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-14T06:16:18.569 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-14T06:16:18.569 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-14T06:16:18.569 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-14T06:16:18.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-14T06:16:18.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-14T06:16:18.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_probe_timeout": "2.000000", 2024-09-14T06:16:18.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_reweight_max_change": "0.050000", 2024-09-14T06:16:18.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_reweight_max_osds": "4", 2024-09-14T06:16:18.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-14T06:16:18.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-14T06:16:18.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-14T06:16:18.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-14T06:16:18.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-14T06:16:18.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_scrub_interval": "86400", 2024-09-14T06:16:18.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_scrub_max_keys": "100", 2024-09-14T06:16:18.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_scrub_timeout": "300", 2024-09-14T06:16:18.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_session_timeout": "300", 2024-09-14T06:16:18.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_smart_report_timeout": "5", 2024-09-14T06:16:18.571 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_stat_smooth_intervals": "6", 2024-09-14T06:16:18.571 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-14T06:16:18.571 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_stretch_pool_min_size": "2", 2024-09-14T06:16:18.571 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_stretch_pool_size": "4", 2024-09-14T06:16:18.571 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-14T06:16:18.571 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-14T06:16:18.571 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_sync_debug": "false", 2024-09-14T06:16:18.571 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-14T06:16:18.571 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-14T06:16:18.571 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_sync_provider_kill_at": "0", 2024-09-14T06:16:18.571 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_sync_requester_kill_at": "0", 2024-09-14T06:16:18.571 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_sync_timeout": "60.000000", 2024-09-14T06:16:18.572 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_target_pg_per_osd": "100", 2024-09-14T06:16:18.572 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_tick_interval": "5", 2024-09-14T06:16:18.572 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_timecheck_interval": "300.000000", 2024-09-14T06:16:18.572 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-14T06:16:18.572 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_use_min_delay_socket": "false", 2024-09-14T06:16:18.572 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_older_version_delay": "604800", 2024-09-14T06:16:18.572 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-14T06:16:18.572 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-14T06:16:18.572 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-14T06:16:18.572 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-14T06:16:18.572 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-14T06:16:18.573 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-14T06:16:18.573 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-14T06:16:18.573 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_misplaced": "false", 2024-09-14T06:16:18.573 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-14T06:16:18.573 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_older_version": "true", 2024-09-14T06:16:18.573 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-14T06:16:18.573 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-14T06:16:18.573 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-14T06:16:18.573 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-14T06:16:18.573 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-14T06:16:18.573 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-14T06:16:18.573 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-14T06:16:18.573 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-14T06:16:18.573 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-14T06:16:18.574 INFO:teuthology.orchestra.run.smithi039.stdout: "monmap": "", 2024-09-14T06:16:18.574 INFO:teuthology.orchestra.run.smithi039.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T06:16:18.574 INFO:teuthology.orchestra.run.smithi039.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-14T06:16:18.574 INFO:teuthology.orchestra.run.smithi039.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-14T06:16:18.574 INFO:teuthology.orchestra.run.smithi039.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T06:16:18.574 INFO:teuthology.orchestra.run.smithi039.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-14T06:16:18.574 INFO:teuthology.orchestra.run.smithi039.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-14T06:16:18.574 INFO:teuthology.orchestra.run.smithi039.stdout: "motr_tracing_enabled": "false", 2024-09-14T06:16:18.574 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_op_threads": "3", 2024-09-14T06:16:18.574 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-14T06:16:18.574 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_cm": "false", 2024-09-14T06:16:18.574 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_device_name": "", 2024-09-14T06:16:18.574 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_dscp": "96", 2024-09-14T06:16:18.574 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-14T06:16:18.574 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-14T06:16:18.574 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_local_gid": "", 2024-09-14T06:16:18.575 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-14T06:16:18.575 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_port_num": "1", 2024-09-14T06:16:18.575 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-14T06:16:18.575 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-14T06:16:18.575 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-14T06:16:18.575 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-14T06:16:18.575 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_sl": "3", 2024-09-14T06:16:18.575 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_support_srq": "true", 2024-09-14T06:16:18.575 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_type": "ib", 2024-09-14T06:16:18.575 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_reap_threshold": "5", 2024-09-14T06:16:18.575 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_before_connect": "false", 2024-09-14T06:16:18.575 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_ipv4": "true", 2024-09-14T06:16:18.575 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_ipv6": "false", 2024-09-14T06:16:18.575 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_msgr1": "true", 2024-09-14T06:16:18.575 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_msgr2": "true", 2024-09-14T06:16:18.576 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_port_max": "7568", 2024-09-14T06:16:18.576 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_port_min": "6800", 2024-09-14T06:16:18.576 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-14T06:16:18.576 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_retry_count": "3", 2024-09-14T06:16:18.576 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_retry_delay": "5", 2024-09-14T06:16:18.576 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_blackhole_client": "false", 2024-09-14T06:16:18.576 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_blackhole_mds": "false", 2024-09-14T06:16:18.576 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_blackhole_mgr": "false", 2024-09-14T06:16:18.576 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_blackhole_mon": "false", 2024-09-14T06:16:18.576 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_blackhole_osd": "false", 2024-09-14T06:16:18.576 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_client_mode": "crc secure", 2024-09-14T06:16:18.576 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-14T06:16:18.576 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_cluster_mode": "crc secure", 2024-09-14T06:16:18.576 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_cluster_type": "", 2024-09-14T06:16:18.576 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_compress_secure": "false", 2024-09-14T06:16:18.577 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_connection_idle_timeout": "900", 2024-09-14T06:16:18.577 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_connection_ready_timeout": "10", 2024-09-14T06:16:18.577 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_crc_data": "true", 2024-09-14T06:16:18.577 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_crc_header": "true", 2024-09-14T06:16:18.577 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_die_on_bad_msg": "false", 2024-09-14T06:16:18.577 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_die_on_bug": "true", 2024-09-14T06:16:18.577 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_die_on_old_message": "true", 2024-09-14T06:16:18.577 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_die_on_skipped_message": "false", 2024-09-14T06:16:18.577 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-14T06:16:18.577 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-14T06:16:18.577 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_coremask": "0xF", 2024-09-14T06:16:18.577 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-14T06:16:18.577 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-14T06:16:18.577 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_enable_tso": "true", 2024-09-14T06:16:18.577 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-14T06:16:18.578 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-14T06:16:18.578 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_hugepages": "", 2024-09-14T06:16:18.578 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-14T06:16:18.578 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-14T06:16:18.578 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_lro": "true", 2024-09-14T06:16:18.579 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_memory_channel": "4", 2024-09-14T06:16:18.579 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-14T06:16:18.579 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_pmd": "", 2024-09-14T06:16:18.579 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_port_id": "0", 2024-09-14T06:16:18.579 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-14T06:16:18.579 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-14T06:16:18.579 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dump_on_send": "false", 2024-09-14T06:16:18.579 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_initial_backoff": "0.200000", 2024-09-14T06:16:18.579 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_inject_delay_max": "1.000000", 2024-09-14T06:16:18.579 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-14T06:16:18.579 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_inject_delay_type": "", 2024-09-14T06:16:18.579 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-14T06:16:18.579 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_inject_network_congestion": "0", 2024-09-14T06:16:18.580 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_inject_socket_failures": "5000", 2024-09-14T06:16:18.580 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_learn_addr_from_peer": "true", 2024-09-14T06:16:18.580 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_max_accept_failures": "4", 2024-09-14T06:16:18.580 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_max_backoff": "15.000000", 2024-09-14T06:16:18.580 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_mon_client_mode": "secure crc", 2024-09-14T06:16:18.580 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-14T06:16:18.580 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_mon_service_mode": "secure crc", 2024-09-14T06:16:18.580 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_osd_compress_min_size": "1024", 2024-09-14T06:16:18.580 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_osd_compress_mode": "none", 2024-09-14T06:16:18.580 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-14T06:16:18.580 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-14T06:16:18.580 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_pq_min_cost": "65536", 2024-09-14T06:16:18.580 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_public_type": "", 2024-09-14T06:16:18.580 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_service_mode": "crc secure", 2024-09-14T06:16:18.581 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_tcp_listen_backlog": "512", 2024-09-14T06:16:18.581 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_tcp_nodelay": "true", 2024-09-14T06:16:18.581 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-14T06:16:18.581 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_tcp_rcvbuf": "0", 2024-09-14T06:16:18.581 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-14T06:16:18.581 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_type": "async+posix", 2024-09-14T06:16:18.581 INFO:teuthology.orchestra.run.smithi039.stdout: "no_config_file": "false", 2024-09-14T06:16:18.581 INFO:teuthology.orchestra.run.smithi039.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-14T06:16:18.581 INFO:teuthology.orchestra.run.smithi039.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-14T06:16:18.581 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter_completion_locks_per_session": "32", 2024-09-14T06:16:18.581 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-14T06:16:18.581 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-14T06:16:18.582 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter_inflight_ops": "1024", 2024-09-14T06:16:18.582 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-14T06:16:18.582 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-14T06:16:18.582 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter_tick_interval": "5.000000", 2024-09-14T06:16:18.582 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter_timeout": "10.000000", 2024-09-14T06:16:18.582 INFO:teuthology.orchestra.run.smithi039.stdout: "objectstore_blackhole": "false", 2024-09-14T06:16:18.582 INFO:teuthology.orchestra.run.smithi039.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-14T06:16:18.582 INFO:teuthology.orchestra.run.smithi039.stdout: "openssl_engine_opts": "", 2024-09-14T06:16:18.582 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_agent_delay_time": "5.000000", 2024-09-14T06:16:18.582 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_agent_hist_halflife": "1000", 2024-09-14T06:16:18.582 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_agent_max_low_ops": "2", 2024-09-14T06:16:18.582 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_agent_max_ops": "4", 2024-09-14T06:16:18.582 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-14T06:16:18.582 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-14T06:16:18.583 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_agent_slop": "0.020000", 2024-09-14T06:16:18.583 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-14T06:16:18.583 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-14T06:16:18.583 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_asio_thread_count": "2", 2024-09-14T06:16:18.583 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_async_recovery_min_cost": "100", 2024-09-14T06:16:18.583 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-14T06:16:18.583 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-14T06:16:18.583 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_backfill_scan_max": "512", 2024-09-14T06:16:18.583 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_backfill_scan_min": "64", 2024-09-14T06:16:18.583 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_backoff_on_degraded": "false", 2024-09-14T06:16:18.583 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_backoff_on_peering": "false", 2024-09-14T06:16:18.583 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_backoff_on_unfound": "true", 2024-09-14T06:16:18.583 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_beacon_report_interval": "300", 2024-09-14T06:16:18.583 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bench_duration": "30", 2024-09-14T06:16:18.583 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-14T06:16:18.584 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bench_max_block_size": "67108864", 2024-09-14T06:16:18.584 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-14T06:16:18.584 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_blkin_trace_all": "false", 2024-09-14T06:16:18.584 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-14T06:16:18.584 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-14T06:16:18.584 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-14T06:16:18.584 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-14T06:16:18.584 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_check_for_log_corruption": "false", 2024-09-14T06:16:18.584 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-14T06:16:18.584 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T06:16:18.584 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-14T06:16:18.584 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T06:16:18.584 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_class_update_on_start": "true", 2024-09-14T06:16:18.584 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_client_message_cap": "256", 2024-09-14T06:16:18.585 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_client_message_size_cap": "524288000", 2024-09-14T06:16:18.585 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_client_op_priority": "63", 2024-09-14T06:16:18.585 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_client_watch_timeout": "30", 2024-09-14T06:16:18.585 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_command_max_records": "256", 2024-09-14T06:16:18.585 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_compact_on_start": "false", 2024-09-14T06:16:18.585 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-14T06:16:18.585 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-14T06:16:18.585 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-14T06:16:18.585 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_crush_update_on_start": "true", 2024-09-14T06:16:18.585 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_crush_update_weight_set": "true", 2024-09-14T06:16:18.585 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-14T06:16:18.585 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-14T06:16:18.585 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-14T06:16:18.586 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-14T06:16:18.586 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-14T06:16:18.586 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_feed_pullee": "-1", 2024-09-14T06:16:18.586 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-14T06:16:18.586 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-14T06:16:18.586 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-14T06:16:18.586 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_misdirected_ops": "false", 2024-09-14T06:16:18.586 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_no_acting_change": "false", 2024-09-14T06:16:18.586 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_no_purge_strays": "false", 2024-09-14T06:16:18.586 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_op_order": "false", 2024-09-14T06:16:18.586 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-14T06:16:18.586 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-14T06:16:18.586 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-14T06:16:18.587 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-14T06:16:18.587 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_shutdown": "false", 2024-09-14T06:16:18.587 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-14T06:16:18.587 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-14T06:16:18.587 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_trim_objects": "false", 2024-09-14T06:16:18.587 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-14T06:16:18.587 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-14T06:16:18.587 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_verify_snaps": "false", 2024-09-14T06:16:18.587 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-14T06:16:18.587 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-14T06:16:18.587 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_deep_scrub_keys": "1024", 2024-09-14T06:16:18.587 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-14T06:16:18.587 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-14T06:16:18.587 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-14T06:16:18.587 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_deep_scrub_stride": "524288", 2024-09-14T06:16:18.588 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-14T06:16:18.588 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-14T06:16:18.588 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_default_notify_timeout": "30", 2024-09-14T06:16:18.588 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_delete_sleep": "0.000000", 2024-09-14T06:16:18.588 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-14T06:16:18.588 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-14T06:16:18.588 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-14T06:16:18.588 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_discard_disconnected_ops": "true", 2024-09-14T06:16:18.588 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_ec_partial_reads": "true", 2024-09-14T06:16:18.588 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_enable_op_tracker": "true", 2024-09-14T06:16:18.588 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-14T06:16:18.588 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-14T06:16:18.588 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-14T06:16:18.589 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-14T06:16:18.589 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_fast_info": "true", 2024-09-14T06:16:18.589 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_fast_shutdown": "true", 2024-09-14T06:16:18.589 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-14T06:16:18.589 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-14T06:16:18.589 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-14T06:16:18.589 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-14T06:16:18.589 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-14T06:16:18.589 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_function_tracing": "false", 2024-09-14T06:16:18.589 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_heartbeat_grace": "20", 2024-09-14T06:16:18.590 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_heartbeat_interval": "6", 2024-09-14T06:16:18.590 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-14T06:16:18.590 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_heartbeat_min_peers": "10", 2024-09-14T06:16:18.590 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_heartbeat_min_size": "2000", 2024-09-14T06:16:18.590 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_heartbeat_stale": "600", 2024-09-14T06:16:18.590 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-14T06:16:18.590 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_hit_set_max_size": "100000", 2024-09-14T06:16:18.590 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_hit_set_min_size": "1000", 2024-09-14T06:16:18.590 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-14T06:16:18.590 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-14T06:16:18.590 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-14T06:16:18.590 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-14T06:16:18.590 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-14T06:16:18.591 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-14T06:16:18.591 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_journal_size": "5120", 2024-09-14T06:16:18.591 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_kill_backfill_at": "0", 2024-09-14T06:16:18.591 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-14T06:16:18.591 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_map_cache_size": "50", 2024-09-14T06:16:18.591 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_map_dedup": "true", 2024-09-14T06:16:18.591 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_map_message_max": "40", 2024-09-14T06:16:18.591 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-14T06:16:18.591 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_map_share_max_epochs": "40", 2024-09-14T06:16:18.591 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_attr_name_len": "100", 2024-09-14T06:16:18.591 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_attr_size": "0", 2024-09-14T06:16:18.591 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_backfills": "1", 2024-09-14T06:16:18.592 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_markdown_count": "5", 2024-09-14T06:16:18.592 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_markdown_period": "600", 2024-09-14T06:16:18.592 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_object_name_len": "2048", 2024-09-14T06:16:18.592 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_object_namespace_len": "256", 2024-09-14T06:16:18.592 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_object_size": "134217728", 2024-09-14T06:16:18.592 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-14T06:16:18.592 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-14T06:16:18.592 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_pg_blocked_by": "16", 2024-09-14T06:16:18.592 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_pg_log_entries": "10000", 2024-09-14T06:16:18.592 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-14T06:16:18.592 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_pgls": "1024", 2024-09-14T06:16:18.592 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_push_cost": "8388608", 2024-09-14T06:16:18.592 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_push_objects": "10", 2024-09-14T06:16:18.593 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_scrubs": "3", 2024-09-14T06:16:18.593 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-14T06:16:18.593 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_trimming_pgs": "2", 2024-09-14T06:16:18.593 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_write_op_reply_len": "64", 2024-09-14T06:16:18.593 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_write_size": "90", 2024-09-14T06:16:18.593 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-14T06:16:18.593 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-14T06:16:18.593 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-14T06:16:18.593 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-14T06:16:18.593 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-14T06:16:18.593 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-14T06:16:18.594 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-14T06:16:18.594 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-14T06:16:18.594 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_profile": "balanced", 2024-09-14T06:16:18.594 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-14T06:16:18.594 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-14T06:16:18.594 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-14T06:16:18.594 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-14T06:16:18.594 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-14T06:16:18.594 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-14T06:16:18.594 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-14T06:16:18.594 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-14T06:16:18.594 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-14T06:16:18.594 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-14T06:16:18.595 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-14T06:16:18.595 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_memory_base": "805306368", 2024-09-14T06:16:18.595 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_memory_cache_min": "134217728", 2024-09-14T06:16:18.595 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-14T06:16:18.595 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-14T06:16:18.595 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_memory_target": "4294967296", 2024-09-14T06:16:18.595 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_memory_target_autotune": "false", 2024-09-14T06:16:18.595 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-14T06:16:18.595 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_min_pg_log_entries": "250", 2024-09-14T06:16:18.595 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_min_recovery_priority": "0", 2024-09-14T06:16:18.596 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-14T06:16:18.596 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-14T06:16:18.596 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mon_report_interval": "5", 2024-09-14T06:16:18.596 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-14T06:16:18.596 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-14T06:16:18.596 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_num_cache_shards": "32", 2024-09-14T06:16:18.596 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_num_op_tracker_shard": "32", 2024-09-14T06:16:18.596 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_numa_auto_affinity": "true", 2024-09-14T06:16:18.597 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_numa_node": "-1", 2024-09-14T06:16:18.597 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_numa_prefer_iface": "true", 2024-09-14T06:16:18.597 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-14T06:16:18.597 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_objecter_finishers": "1", 2024-09-14T06:16:18.597 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_objectstore": "bluestore", 2024-09-14T06:16:18.597 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_objectstore_fuse": "false", 2024-09-14T06:16:18.598 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-14T06:16:18.598 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_objectstore_tracing": "false", 2024-09-14T06:16:18.598 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_complaint_time": "30.000000", 2024-09-14T06:16:18.598 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_history_duration": "600", 2024-09-14T06:16:18.598 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_history_size": "20", 2024-09-14T06:16:18.598 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_history_slow_op_size": "20", 2024-09-14T06:16:18.598 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-14T06:16:18.598 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_log_threshold": "5", 2024-09-14T06:16:18.599 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_num_shards": "0", 2024-09-14T06:16:18.599 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_num_shards_hdd": "5", 2024-09-14T06:16:18.599 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_num_shards_ssd": "8", 2024-09-14T06:16:18.599 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-14T06:16:18.599 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-14T06:16:18.599 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-14T06:16:18.599 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-14T06:16:18.599 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_pq_min_cost": "65536", 2024-09-14T06:16:18.599 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-14T06:16:18.599 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_queue_cut_off": "high", 2024-09-14T06:16:18.600 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-14T06:16:18.600 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_thread_timeout": "15", 2024-09-14T06:16:18.600 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_open_classes_on_start": "true", 2024-09-14T06:16:18.600 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_os_flags": "0", 2024-09-14T06:16:18.600 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_peering_op_priority": "255", 2024-09-14T06:16:18.600 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_delete_cost": "1048576", 2024-09-14T06:16:18.600 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_delete_priority": "5", 2024-09-14T06:16:18.600 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-14T06:16:18.600 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-14T06:16:18.600 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-14T06:16:18.600 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_log_trim_max": "10000", 2024-09-14T06:16:18.601 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_log_trim_min": "100", 2024-09-14T06:16:18.601 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-14T06:16:18.601 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-14T06:16:18.601 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-14T06:16:18.601 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-14T06:16:18.601 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-14T06:16:18.601 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-14T06:16:18.601 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-14T06:16:18.601 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-14T06:16:18.602 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-14T06:16:18.602 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-14T06:16:18.602 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_crimson": "false", 2024-09-14T06:16:18.602 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-14T06:16:18.602 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-14T06:16:18.602 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-14T06:16:18.602 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-14T06:16:18.602 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-14T06:16:18.602 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-14T06:16:18.602 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-14T06:16:18.603 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-14T06:16:18.603 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_flags": "0", 2024-09-14T06:16:18.603 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-14T06:16:18.603 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_min_size": "0", 2024-09-14T06:16:18.603 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-14T06:16:18.603 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_pg_num": "32", 2024-09-14T06:16:18.604 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_pgp_num": "0", 2024-09-14T06:16:18.604 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-14T06:16:18.604 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_read_ratio": "70", 2024-09-14T06:16:18.604 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_size": "2", 2024-09-14T06:16:18.604 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_type": "replicated", 2024-09-14T06:16:18.604 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-14T06:16:18.604 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-14T06:16:18.604 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_push_per_object_cost": "1000", 2024-09-14T06:16:18.604 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-14T06:16:18.604 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recover_clone_overlap": "true", 2024-09-14T06:16:18.604 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-14T06:16:18.604 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_cost": "20971520", 2024-09-14T06:16:18.604 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-14T06:16:18.605 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_max_active": "0", 2024-09-14T06:16:18.605 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-14T06:16:18.605 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-14T06:16:18.605 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-14T06:16:18.605 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-14T06:16:18.605 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_max_single_start": "1", 2024-09-14T06:16:18.605 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_op_priority": "3", 2024-09-14T06:16:18.605 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-14T06:16:18.605 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_priority": "5", 2024-09-14T06:16:18.605 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-14T06:16:18.605 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_sleep": "0.000000", 2024-09-14T06:16:18.605 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-14T06:16:18.605 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-14T06:16:18.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-14T06:16:18.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_repair_during_recovery": "false", 2024-09-14T06:16:18.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_requested_scrub_priority": "120", 2024-09-14T06:16:18.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-14T06:16:18.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-14T06:16:18.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_auto_repair": "false", 2024-09-14T06:16:18.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-14T06:16:18.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-14T06:16:18.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_begin_hour": "0", 2024-09-14T06:16:18.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_begin_week_day": "0", 2024-09-14T06:16:18.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_chunk_max": "25", 2024-09-14T06:16:18.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_chunk_min": "5", 2024-09-14T06:16:18.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_cost": "52428800", 2024-09-14T06:16:18.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-14T06:16:18.607 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_during_recovery": "false", 2024-09-14T06:16:18.607 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_end_hour": "0", 2024-09-14T06:16:18.607 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_end_week_day": "0", 2024-09-14T06:16:18.607 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_event_cost": "4096", 2024-09-14T06:16:18.607 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-14T06:16:18.607 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-14T06:16:18.607 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_invalid_stats": "true", 2024-09-14T06:16:18.607 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-14T06:16:18.607 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-14T06:16:18.607 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_max_preemptions": "5", 2024-09-14T06:16:18.607 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-14T06:16:18.607 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_priority": "5", 2024-09-14T06:16:18.607 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_sleep": "0.000000", 2024-09-14T06:16:18.608 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-14T06:16:18.608 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-14T06:16:18.608 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-14T06:16:18.608 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-14T06:16:18.608 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_skip_data_digest": "false", 2024-09-14T06:16:18.608 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_smart_report_timeout": "5", 2024-09-14T06:16:18.608 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_snap_trim_cost": "1048576", 2024-09-14T06:16:18.608 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_snap_trim_priority": "5", 2024-09-14T06:16:18.608 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-14T06:16:18.608 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-14T06:16:18.608 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-14T06:16:18.608 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-14T06:16:18.609 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-14T06:16:18.609 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-14T06:16:18.609 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-14T06:16:18.609 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_target_transaction_size": "30", 2024-09-14T06:16:18.609 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-14T06:16:18.609 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-14T06:16:18.609 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-14T06:16:18.609 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-14T06:16:18.609 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-14T06:16:18.609 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-14T06:16:18.610 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-14T06:16:18.610 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-14T06:16:18.610 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-14T06:16:18.610 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-14T06:16:18.610 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tracing": "false", 2024-09-14T06:16:18.610 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_use_stale_snap": "false", 2024-09-14T06:16:18.610 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-14T06:16:18.610 INFO:teuthology.orchestra.run.smithi039.stdout: "osdc_blkin_trace_all": "false", 2024-09-14T06:16:18.610 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_kill_at": "0", 2024-09-14T06:16:18.611 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_max_join_drift": "10", 2024-09-14T06:16:18.611 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_min": "500", 2024-09-14T06:16:18.611 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_min_wait": "0.050000", 2024-09-14T06:16:18.611 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_propose_interval": "1.000000", 2024-09-14T06:16:18.611 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_service_trim_max": "500", 2024-09-14T06:16:18.611 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-14T06:16:18.611 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_service_trim_min": "250", 2024-09-14T06:16:18.611 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_stash_full_interval": "25", 2024-09-14T06:16:18.611 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_trim_max": "500", 2024-09-14T06:16:18.611 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_trim_min": "250", 2024-09-14T06:16:18.611 INFO:teuthology.orchestra.run.smithi039.stdout: "perf": "true", 2024-09-14T06:16:18.611 INFO:teuthology.orchestra.run.smithi039.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-14T06:16:18.611 INFO:teuthology.orchestra.run.smithi039.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-14T06:16:18.611 INFO:teuthology.orchestra.run.smithi039.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-14T06:16:18.611 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "-", 2024-09-14T06:16:18.612 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrv": "", 2024-09-14T06:16:18.612 INFO:teuthology.orchestra.run.smithi039.stdout: "public_bind_addr": "-", 2024-09-14T06:16:18.612 INFO:teuthology.orchestra.run.smithi039.stdout: "public_network": "", 2024-09-14T06:16:18.612 INFO:teuthology.orchestra.run.smithi039.stdout: "public_network_interface": "", 2024-09-14T06:16:18.612 INFO:teuthology.orchestra.run.smithi039.stdout: "qat_compressor_busy_polling": "false", 2024-09-14T06:16:18.612 INFO:teuthology.orchestra.run.smithi039.stdout: "qat_compressor_enabled": "false", 2024-09-14T06:16:18.612 INFO:teuthology.orchestra.run.smithi039.stdout: "qat_compressor_session_max_number": "256", 2024-09-14T06:16:18.612 INFO:teuthology.orchestra.run.smithi039.stdout: "rados_mon_op_timeout": "0", 2024-09-14T06:16:18.612 INFO:teuthology.orchestra.run.smithi039.stdout: "rados_osd_op_timeout": "0", 2024-09-14T06:16:18.613 INFO:teuthology.orchestra.run.smithi039.stdout: "rados_tracing": "false", 2024-09-14T06:16:18.613 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_atime_update_interval": "60", 2024-09-14T06:16:18.613 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-14T06:16:18.613 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_balance_parent_reads": "false", 2024-09-14T06:16:18.613 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_balance_snap_reads": "false", 2024-09-14T06:16:18.613 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_blkin_trace_all": "false", 2024-09-14T06:16:18.613 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-14T06:16:18.613 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-14T06:16:18.613 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache": "true", 2024-09-14T06:16:18.613 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-14T06:16:18.613 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-14T06:16:18.613 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-14T06:16:18.613 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-14T06:16:18.614 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache_policy": "writearound", 2024-09-14T06:16:18.614 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache_size": "33554432", 2024-09-14T06:16:18.614 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-14T06:16:18.614 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-14T06:16:18.614 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_clone_copy_on_read": "false", 2024-09-14T06:16:18.614 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_compression_hint": "none", 2024-09-14T06:16:18.614 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_concurrent_management_ops": "10", 2024-09-14T06:16:18.614 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-14T06:16:18.614 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_clone_format": "auto", 2024-09-14T06:16:18.614 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_data_pool": "", 2024-09-14T06:16:18.614 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_features": "61", 2024-09-14T06:16:18.614 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_format": "2", 2024-09-14T06:16:18.614 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_map_options": "", 2024-09-14T06:16:18.614 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_order": "22", 2024-09-14T06:16:18.615 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_pool": "rbd", 2024-09-14T06:16:18.615 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-14T06:16:18.615 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_stripe_count": "0", 2024-09-14T06:16:18.615 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_stripe_unit": "0", 2024-09-14T06:16:18.615 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-14T06:16:18.615 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-14T06:16:18.615 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-14T06:16:18.615 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_enable_alloc_hint": "true", 2024-09-14T06:16:18.615 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-14T06:16:18.615 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_io_scheduler": "simple", 2024-09-14T06:16:18.615 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-14T06:16:18.615 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-14T06:16:18.615 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-14T06:16:18.615 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-14T06:16:18.615 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-14T06:16:18.616 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-14T06:16:18.616 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-14T06:16:18.616 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-14T06:16:18.616 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-14T06:16:18.616 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_order": "24", 2024-09-14T06:16:18.616 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_pool": "", 2024-09-14T06:16:18.616 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_splay_width": "4", 2024-09-14T06:16:18.616 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_localize_parent_reads": "false", 2024-09-14T06:16:18.616 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_localize_snap_reads": "false", 2024-09-14T06:16:18.616 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-14T06:16:18.616 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-14T06:16:18.616 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-14T06:16:18.616 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-14T06:16:18.617 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-14T06:16:18.617 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-14T06:16:18.617 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-14T06:16:18.617 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-14T06:16:18.617 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-14T06:16:18.617 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-14T06:16:18.617 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-14T06:16:18.617 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-14T06:16:18.617 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-14T06:16:18.617 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-14T06:16:18.617 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-14T06:16:18.617 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-14T06:16:18.617 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-14T06:16:18.617 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-14T06:16:18.618 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-14T06:16:18.618 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-14T06:16:18.618 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-14T06:16:18.618 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-14T06:16:18.618 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-14T06:16:18.618 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-14T06:16:18.618 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-14T06:16:18.618 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-14T06:16:18.618 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-14T06:16:18.618 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-14T06:16:18.618 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-14T06:16:18.618 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-14T06:16:18.618 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-14T06:16:18.618 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mtime_update_interval": "60", 2024-09-14T06:16:18.619 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_non_blocking_aio": "true", 2024-09-14T06:16:18.619 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_op_thread_timeout": "60", 2024-09-14T06:16:18.619 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_op_threads": "1", 2024-09-14T06:16:18.619 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_parent_cache_enabled": "false", 2024-09-14T06:16:18.619 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-14T06:16:18.619 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-14T06:16:18.619 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-14T06:16:18.619 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_plugins": "", 2024-09-14T06:16:18.619 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_bps_burst": "0", 2024-09-14T06:16:18.619 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-14T06:16:18.619 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_bps_limit": "0", 2024-09-14T06:16:18.619 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_exclude_ops": "0", 2024-09-14T06:16:18.619 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_iops_burst": "0", 2024-09-14T06:16:18.619 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-14T06:16:18.619 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_iops_limit": "0", 2024-09-14T06:16:18.620 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-14T06:16:18.620 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-14T06:16:18.620 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-14T06:16:18.620 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-14T06:16:18.620 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-14T06:16:18.620 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-14T06:16:18.620 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-14T06:16:18.620 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-14T06:16:18.620 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-14T06:16:18.620 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-14T06:16:18.620 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-14T06:16:18.621 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-14T06:16:18.621 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-14T06:16:18.621 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-14T06:16:18.621 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_read_from_replica_policy": "default", 2024-09-14T06:16:18.621 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-14T06:16:18.621 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-14T06:16:18.621 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-14T06:16:18.621 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-14T06:16:18.621 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_skip_partial_discard": "true", 2024-09-14T06:16:18.622 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-14T06:16:18.622 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_tracing": "false", 2024-09-14T06:16:18.622 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_validate_names": "true", 2024-09-14T06:16:18.622 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_validate_pool": "true", 2024-09-14T06:16:18.622 INFO:teuthology.orchestra.run.smithi039.stdout: "restapi_base_url": "", 2024-09-14T06:16:18.622 INFO:teuthology.orchestra.run.smithi039.stdout: "restapi_log_level": "", 2024-09-14T06:16:18.622 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-14T06:16:18.622 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-14T06:16:18.622 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_acl_grants_max_num": "100", 2024-09-14T06:16:18.622 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_admin_entry": "admin", 2024-09-14T06:16:18.622 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-14T06:16:18.622 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_asio_assert_yielding": "false", 2024-09-14T06:16:18.622 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_backend_store": "rados", 2024-09-14T06:16:18.622 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_barbican_url": "", 2024-09-14T06:16:18.622 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_beast_enable_async": "true", 2024-09-14T06:16:18.622 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_bucket_counters_cache": "false", 2024-09-14T06:16:18.623 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-14T06:16:18.623 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-14T06:16:18.623 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-14T06:16:18.623 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-14T06:16:18.623 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-14T06:16:18.623 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-14T06:16:18.623 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-14T06:16:18.623 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-14T06:16:18.623 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_cache_enabled": "true", 2024-09-14T06:16:18.623 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_cache_expiry_interval": "900", 2024-09-14T06:16:18.624 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_cache_lru_size": "25000", 2024-09-14T06:16:18.624 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_config_store": "rados", 2024-09-14T06:16:18.624 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_content_length_compat": "false", 2024-09-14T06:16:18.624 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_copy_obj_progress": "true", 2024-09-14T06:16:18.624 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-14T06:16:18.624 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_cors_rules_max_num": "100", 2024-09-14T06:16:18.624 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_cross_domain_policy": "", 2024-09-14T06:16:18.624 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-14T06:16:18.624 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_kmip_addr": "", 2024-09-14T06:16:18.624 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-14T06:16:18.625 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-14T06:16:18.625 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-14T06:16:18.625 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-14T06:16:18.625 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_kmip_password": "", 2024-09-14T06:16:18.625 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-14T06:16:18.625 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_kmip_username": "", 2024-09-14T06:16:18.625 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_require_ssl": "true", 2024-09-14T06:16:18.625 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-14T06:16:18.625 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-14T06:16:18.625 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-14T06:16:18.625 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-14T06:16:18.625 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-14T06:16:18.626 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-14T06:16:18.626 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-14T06:16:18.626 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-14T06:16:18.626 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-14T06:16:18.626 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-14T06:16:18.626 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-14T06:16:18.626 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-14T06:16:18.626 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-14T06:16:18.626 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-14T06:16:18.626 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-14T06:16:18.626 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_addr": "", 2024-09-14T06:16:18.626 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_auth": "token", 2024-09-14T06:16:18.626 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_namespace": "", 2024-09-14T06:16:18.626 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_prefix": "", 2024-09-14T06:16:18.626 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-14T06:16:18.627 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-14T06:16:18.627 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-14T06:16:18.627 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-14T06:16:18.627 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_token_file": "", 2024-09-14T06:16:18.627 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-14T06:16:18.627 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_curl_buffersize": "524288", 2024-09-14T06:16:18.627 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-14T06:16:18.627 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_curl_low_speed_time": "300", 2024-09-14T06:16:18.627 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-14T06:16:18.627 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-14T06:16:18.627 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-14T06:16:18.627 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-14T06:16:18.627 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-14T06:16:18.627 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-14T06:16:18.627 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-14T06:16:18.627 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-14T06:16:18.628 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-14T06:16:18.628 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-14T06:16:18.628 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-14T06:16:18.628 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-14T06:16:18.628 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-14T06:16:18.628 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-14T06:16:18.628 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-14T06:16:18.628 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-14T06:16:18.628 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-14T06:16:18.628 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-14T06:16:18.628 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-14T06:16:18.628 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_data_log_changes_size": "1000", 2024-09-14T06:16:18.629 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_data_log_num_shards": "128", 2024-09-14T06:16:18.629 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-14T06:16:18.629 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_data_log_window": "30", 2024-09-14T06:16:18.629 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-14T06:16:18.629 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-14T06:16:18.629 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-14T06:16:18.629 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-14T06:16:18.629 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-14T06:16:18.629 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-14T06:16:18.629 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-14T06:16:18.629 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-14T06:16:18.629 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-14T06:16:18.629 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-14T06:16:18.630 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-14T06:16:18.630 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-14T06:16:18.630 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-14T06:16:18.630 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_disable_s3select": "false", 2024-09-14T06:16:18.630 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-14T06:16:18.630 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-14T06:16:18.630 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-14T06:16:18.630 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-14T06:16:18.630 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-14T06:16:18.630 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-14T06:16:18.630 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-14T06:16:18.630 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-14T06:16:18.630 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-14T06:16:18.630 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-14T06:16:18.630 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-14T06:16:18.630 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-14T06:16:18.631 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dns_name": "", 2024-09-14T06:16:18.631 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dns_s3website_name": "", 2024-09-14T06:16:18.631 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dynamic_resharding": "true", 2024-09-14T06:16:18.631 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-14T06:16:18.631 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-14T06:16:18.631 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-14T06:16:18.631 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enable_gc_threads": "true", 2024-09-14T06:16:18.631 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enable_lc_threads": "true", 2024-09-14T06:16:18.631 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enable_mdsearch": "true", 2024-09-14T06:16:18.631 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enable_ops_log": "false", 2024-09-14T06:16:18.631 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enable_quota_threads": "true", 2024-09-14T06:16:18.631 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enable_static_website": "false", 2024-09-14T06:16:18.632 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enable_usage_log": "false", 2024-09-14T06:16:18.632 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enforce_swift_acls": "true", 2024-09-14T06:16:18.632 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_exit_timeout_secs": "120", 2024-09-14T06:16:18.632 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_expose_bucket": "false", 2024-09-14T06:16:18.632 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_extended_http_attrs": "", 2024-09-14T06:16:18.632 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_filter": "none", 2024-09-14T06:16:18.632 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-14T06:16:18.632 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_frontends": "beast port=7480", 2024-09-14T06:16:18.632 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-14T06:16:18.632 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_max_deferred": "50", 2024-09-14T06:16:18.632 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-14T06:16:18.632 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_max_objs": "32", 2024-09-14T06:16:18.632 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-14T06:16:18.633 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-14T06:16:18.633 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-14T06:16:18.633 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-14T06:16:18.633 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_processor_period": "3600", 2024-09-14T06:16:18.633 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-14T06:16:18.633 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-14T06:16:18.633 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-14T06:16:18.633 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-14T06:16:18.633 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_init_timeout": "300", 2024-09-14T06:16:18.633 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-14T06:16:18.633 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-14T06:16:18.633 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_kafka_connection_idle": "300", 2024-09-14T06:16:18.633 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-14T06:16:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-14T06:16:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-14T06:16:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-14T06:16:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-14T06:16:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_admin_domain": "", 2024-09-14T06:16:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_admin_password": "", 2024-09-14T06:16:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_admin_password_path": "", 2024-09-14T06:16:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_admin_project": "", 2024-09-14T06:16:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_admin_tenant": "", 2024-09-14T06:16:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_admin_token": "", 2024-09-14T06:16:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_admin_token_path": "", 2024-09-14T06:16:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_admin_user": "", 2024-09-14T06:16:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_barbican_domain": "", 2024-09-14T06:16:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_barbican_password": "", 2024-09-14T06:16:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_barbican_project": "", 2024-09-14T06:16:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-14T06:16:18.635 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_barbican_user": "", 2024-09-14T06:16:18.635 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-14T06:16:18.635 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-14T06:16:18.635 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-14T06:16:18.635 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-14T06:16:18.635 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-14T06:16:18.635 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_url": "", 2024-09-14T06:16:18.635 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-14T06:16:18.635 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lc_debug_interval": "-1", 2024-09-14T06:16:18.635 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lc_lock_max_time": "90", 2024-09-14T06:16:18.635 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lc_max_objs": "32", 2024-09-14T06:16:18.635 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lc_max_rules": "1000", 2024-09-14T06:16:18.635 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lc_max_worker": "3", 2024-09-14T06:16:18.635 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-14T06:16:18.635 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lc_thread_delay": "0", 2024-09-14T06:16:18.636 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-14T06:16:18.636 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ldap_dnattr": "uid", 2024-09-14T06:16:18.636 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-14T06:16:18.636 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ldap_searchfilter": "", 2024-09-14T06:16:18.636 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-14T06:16:18.636 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ldap_uri": "", 2024-09-14T06:16:18.636 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-14T06:16:18.636 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-14T06:16:18.636 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-14T06:16:18.636 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-14T06:16:18.636 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_log_http_headers": "", 2024-09-14T06:16:18.636 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-14T06:16:18.636 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-14T06:16:18.637 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_log_object_name_utc": "false", 2024-09-14T06:16:18.637 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-14T06:16:18.637 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-14T06:16:18.637 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_attr_name_len": "0", 2024-09-14T06:16:18.637 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_attr_size": "0", 2024-09-14T06:16:18.637 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-14T06:16:18.637 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_chunk_size": "4194304", 2024-09-14T06:16:18.637 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-14T06:16:18.637 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-14T06:16:18.637 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-14T06:16:18.637 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_listing_results": "1000", 2024-09-14T06:16:18.637 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_notify_retries": "10", 2024-09-14T06:16:18.638 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-14T06:16:18.638 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_put_param_size": "1048576", 2024-09-14T06:16:18.638 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_put_size": "5368709120", 2024-09-14T06:16:18.638 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_slo_entries": "1000", 2024-09-14T06:16:18.638 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_md_log_max_shards": "64", 2024-09-14T06:16:18.638 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-14T06:16:18.638 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-14T06:16:18.638 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-14T06:16:18.638 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-14T06:16:18.638 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_mp_lock_max_time": "600", 2024-09-14T06:16:18.638 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-14T06:16:18.638 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-14T06:16:18.638 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-14T06:16:18.639 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-14T06:16:18.639 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-14T06:16:18.639 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-14T06:16:18.639 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-14T06:16:18.639 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-14T06:16:18.639 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_max_gc": "300", 2024-09-14T06:16:18.639 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-14T06:16:18.639 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-14T06:16:18.639 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-14T06:16:18.639 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-14T06:16:18.639 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-14T06:16:18.639 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-14T06:16:18.639 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-14T06:16:18.639 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_num_async_rados_threads": "32", 2024-09-14T06:16:18.639 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_num_control_oids": "8", 2024-09-14T06:16:18.640 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_numa_node": "-1", 2024-09-14T06:16:18.640 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-14T06:16:18.640 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-14T06:16:18.640 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_objexp_chunk_size": "100", 2024-09-14T06:16:18.640 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_objexp_gc_interval": "600", 2024-09-14T06:16:18.640 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-14T06:16:18.640 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-14T06:16:18.640 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-14T06:16:18.640 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_op_thread_timeout": "600", 2024-09-14T06:16:18.640 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_op_tracing": "false", 2024-09-14T06:16:18.640 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_opa_token": "", 2024-09-14T06:16:18.640 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_opa_url": "", 2024-09-14T06:16:18.640 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_opa_verify_ssl": "true", 2024-09-14T06:16:18.640 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-14T06:16:18.641 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-14T06:16:18.641 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ops_log_rados": "false", 2024-09-14T06:16:18.641 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ops_log_socket_path": "", 2024-09-14T06:16:18.641 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-14T06:16:18.641 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-14T06:16:18.641 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-14T06:16:18.641 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_period_push_interval": "2.000000", 2024-09-14T06:16:18.641 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-14T06:16:18.641 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-14T06:16:18.641 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-14T06:16:18.641 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-14T06:16:18.641 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_posix_cache_lanes": "3", 2024-09-14T06:16:18.641 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-14T06:16:18.642 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-14T06:16:18.642 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_posix_cache_partitions": "3", 2024-09-14T06:16:18.642 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-14T06:16:18.642 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_print_continue": "true", 2024-09-14T06:16:18.642 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-14T06:16:18.642 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-14T06:16:18.642 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-14T06:16:18.642 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-14T06:16:18.642 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-14T06:16:18.642 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_rados_tracing": "false", 2024-09-14T06:16:18.643 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_realm": "", 2024-09-14T06:16:18.643 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_realm_id": "", 2024-09-14T06:16:18.643 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-14T06:16:18.643 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_region": "", 2024-09-14T06:16:18.643 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-14T06:16:18.643 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-14T06:16:18.643 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-14T06:16:18.643 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-14T06:16:18.643 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_request_uri": "", 2024-09-14T06:16:18.643 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_reshard_batch_size": "64", 2024-09-14T06:16:18.643 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-14T06:16:18.643 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-14T06:16:18.643 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_reshard_max_aio": "128", 2024-09-14T06:16:18.643 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_reshard_num_logs": "16", 2024-09-14T06:16:18.644 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-14T06:16:18.644 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-14T06:16:18.644 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_reshard_thread_interval": "600", 2024-09-14T06:16:18.644 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-14T06:16:18.644 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_resolve_cname": "false", 2024-09-14T06:16:18.644 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-14T06:16:18.644 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_run_sync_thread": "true", 2024-09-14T06:16:18.644 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-14T06:16:18.644 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-14T06:16:18.644 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-14T06:16:18.644 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-14T06:16:18.644 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-14T06:16:18.644 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-14T06:16:18.645 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-14T06:16:18.645 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-14T06:16:18.645 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-14T06:16:18.645 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_scheduler_type": "throttler", 2024-09-14T06:16:18.645 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_script_uri": "", 2024-09-14T06:16:18.645 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_service_provider_name": "", 2024-09-14T06:16:18.645 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-14T06:16:18.645 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sts_client_id": "", 2024-09-14T06:16:18.645 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sts_client_secret": "", 2024-09-14T06:16:18.645 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sts_entry": "sts", 2024-09-14T06:16:18.645 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sts_key": "sts", 2024-09-14T06:16:18.645 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-14T06:16:18.645 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sts_min_session_duration": "900", 2024-09-14T06:16:18.645 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sts_token_introspection_url": "", 2024-09-14T06:16:18.645 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_account_in_url": "false", 2024-09-14T06:16:18.646 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_auth_entry": "auth", 2024-09-14T06:16:18.646 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_auth_url": "", 2024-09-14T06:16:18.646 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_custom_header": "", 2024-09-14T06:16:18.646 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-14T06:16:18.646 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_need_stats": "true", 2024-09-14T06:16:18.646 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_tenant_name": "", 2024-09-14T06:16:18.646 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_token_expiration": "86400", 2024-09-14T06:16:18.646 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_url": "", 2024-09-14T06:16:18.646 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_url_prefix": "swift", 2024-09-14T06:16:18.646 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-14T06:16:18.646 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-14T06:16:18.646 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-14T06:16:18.646 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_lease_period": "120", 2024-09-14T06:16:18.647 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-14T06:16:18.647 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-14T06:16:18.647 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-14T06:16:18.647 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-14T06:16:18.647 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-14T06:16:18.647 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-14T06:16:18.647 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-14T06:16:18.647 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-14T06:16:18.647 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-14T06:16:18.647 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_thread_pool_size": "512", 2024-09-14T06:16:18.647 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-14T06:16:18.647 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-14T06:16:18.647 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-14T06:16:18.647 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-14T06:16:18.648 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_torrent_comment": "", 2024-09-14T06:16:18.648 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_torrent_createby": "", 2024-09-14T06:16:18.648 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_torrent_encoding": "", 2024-09-14T06:16:18.648 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_torrent_flag": "false", 2024-09-14T06:16:18.648 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-14T06:16:18.648 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_torrent_origin": "", 2024-09-14T06:16:18.648 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-14T06:16:18.648 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_torrent_tracker": "", 2024-09-14T06:16:18.648 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_trust_forwarded_https": "false", 2024-09-14T06:16:18.648 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-14T06:16:18.648 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-14T06:16:18.648 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_usage_max_shards": "32", 2024-09-14T06:16:18.648 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_usage_max_user_shards": "1", 2024-09-14T06:16:18.649 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_use_opa_authz": "false", 2024-09-14T06:16:18.649 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_counters_cache": "false", 2024-09-14T06:16:18.649 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-14T06:16:18.649 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-14T06:16:18.649 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-14T06:16:18.649 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_max_buckets": "1000", 2024-09-14T06:16:18.649 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_policies_max_num": "100", 2024-09-14T06:16:18.649 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-14T06:16:18.649 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-14T06:16:18.649 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-14T06:16:18.649 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-14T06:16:18.649 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_unique_email": "true", 2024-09-14T06:16:18.649 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_verify_ssl": "true", 2024-09-14T06:16:18.649 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-14T06:16:18.650 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_zone": "", 2024-09-14T06:16:18.650 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_zone_id": "", 2024-09-14T06:16:18.650 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-14T06:16:18.650 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_zonegroup": "", 2024-09-14T06:16:18.650 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_zonegroup_id": "", 2024-09-14T06:16:18.650 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-14T06:16:18.650 INFO:teuthology.orchestra.run.smithi039.stdout: "rgwlc_auto_session_clear": "true", 2024-09-14T06:16:18.650 INFO:teuthology.orchestra.run.smithi039.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-14T06:16:18.650 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_block_size": "4096", 2024-09-14T06:16:18.650 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-14T06:16:18.650 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-14T06:16:18.650 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-14T06:16:18.650 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-14T06:16:18.650 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-14T06:16:18.650 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cache_size": "536870912", 2024-09-14T06:16:18.651 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-14T06:16:18.651 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-14T06:16:18.651 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-14T06:16:18.651 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-14T06:16:18.651 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-14T06:16:18.651 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-14T06:16:18.651 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-14T06:16:18.651 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-14T06:16:18.651 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_index_type": "binary_search", 2024-09-14T06:16:18.651 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-14T06:16:18.651 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-14T06:16:18.651 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_partition_filters": "false", 2024-09-14T06:16:18.651 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_perf": "false", 2024-09-14T06:16:18.652 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-14T06:16:18.652 INFO:teuthology.orchestra.run.smithi039.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-14T06:16:18.652 INFO:teuthology.orchestra.run.smithi039.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-14T06:16:18.652 INFO:teuthology.orchestra.run.smithi039.stdout: "run_dir": "/var/run/ceph", 2024-09-14T06:16:18.652 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_block_create": "true", 2024-09-14T06:16:18.652 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_cache_lru_size": "67108864", 2024-09-14T06:16:18.652 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-14T06:16:18.652 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-14T06:16:18.652 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_default_max_object_size": "16777216", 2024-09-14T06:16:18.652 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-14T06:16:18.652 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_device_size": "53687091200", 2024-09-14T06:16:18.653 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-14T06:16:18.653 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_full_integrity_check": "false", 2024-09-14T06:16:18.653 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_journal_batch_capacity": "16", 2024-09-14T06:16:18.653 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-14T06:16:18.653 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-14T06:16:18.653 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-14T06:16:18.653 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_main_device_type": "SSD", 2024-09-14T06:16:18.653 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-14T06:16:18.653 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-14T06:16:18.653 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-14T06:16:18.654 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-14T06:16:18.654 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-14T06:16:18.654 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-14T06:16:18.654 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_segment_size": "67108864", 2024-09-14T06:16:18.654 INFO:teuthology.orchestra.run.smithi039.stdout: "set_keepcaps": "false", 2024-09-14T06:16:18.654 INFO:teuthology.orchestra.run.smithi039.stdout: "setgroup": "", 2024-09-14T06:16:18.654 INFO:teuthology.orchestra.run.smithi039.stdout: "setuser": "", 2024-09-14T06:16:18.654 INFO:teuthology.orchestra.run.smithi039.stdout: "setuser_match_path": "", 2024-09-14T06:16:18.654 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-14T06:16:18.654 INFO:teuthology.orchestra.run.smithi039.stdout: "thp": "false", 2024-09-14T06:16:18.655 INFO:teuthology.orchestra.run.smithi039.stdout: "threadpool_default_timeout": "60", 2024-09-14T06:16:18.655 INFO:teuthology.orchestra.run.smithi039.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-14T06:16:18.655 INFO:teuthology.orchestra.run.smithi039.stdout: "throttler_perf_counter": "true", 2024-09-14T06:16:18.655 INFO:teuthology.orchestra.run.smithi039.stdout: "tmp_dir": "/tmp", 2024-09-14T06:16:18.655 INFO:teuthology.orchestra.run.smithi039.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-14T06:16:18.655 INFO:teuthology.orchestra.run.smithi039.stdout: "uadk_compressor_enabled": "false", 2024-09-14T06:16:18.655 INFO:teuthology.orchestra.run.smithi039.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-14T06:16:18.655 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:18.657 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-14T06:16:18.802 INFO:teuthology.orchestra.run.smithi039.stdout:mon.a: {} 2024-09-14T06:16:18.803 INFO:teuthology.orchestra.run.smithi039.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T06:16:18.816 INFO:teuthology.orchestra.run.smithi039.stdout:mon.b: {} 2024-09-14T06:16:18.816 INFO:teuthology.orchestra.run.smithi039.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T06:16:18.825 INFO:teuthology.orchestra.run.smithi039.stdout:mon.c: {} 2024-09-14T06:16:18.826 INFO:teuthology.orchestra.run.smithi039.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T06:16:18.841 DEBUG:teuthology.orchestra.run.smithi039:> type ceph-objectstore-tool 2024-09-14T06:16:18.845 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T06:16:18.846 DEBUG:teuthology.orchestra.run.smithi087:> type ceph-objectstore-tool 2024-09-14T06:16:18.851 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T06:16:18.851 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-14T06:16:18.861 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-14T06:16:18.861 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-14T06:16:18.862 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:16:18.862 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:16:18.862 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:16:18.862 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-14T06:16:18.862 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-14T06:16:18.863 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-14T06:16:18.863 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:16:18.863 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-14T06:16:18.864 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:16:18.864 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-14T06:16:18.864 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:16:18.865 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-14T06:16:18.887 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:18.884+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:mapped writes DISABLED 2024-09-14T06:16:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:mapped reads DISABLED 2024-09-14T06:16:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:Seed set to 3485 2024-09-14T06:16:18.965 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:18.960+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:19.003 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:19.003 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:19.003 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:19.003 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:19.018 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:16:19.066 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:19.060+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:19.096 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:19.097 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:19.114 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:19.114 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:19.130 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:19.130 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:19.135 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:19.136 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:19.148 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:19.148 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:19.149 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:16:19.167 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:19.164+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:19.173 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:19.174 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:19.192 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:19.193 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:19.226 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:19.226 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:19.236 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-14T06:16:19.232+0000 7f9bd5666640 -1 mon.a@0(leader).osd e23 definitely_dead 0 2024-09-14T06:16:19.250 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:19.250 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:19.283 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:19.283 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:19.283 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:19.280+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:19.295 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:16:19.369 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:19.364+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:19.458 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:19.458 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:19.470 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:19.468+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:19.473 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:16:19.571 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:19.572+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:19.588 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:19.588 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:19.602 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:16:19.671 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:19.668+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:19.719 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:19.719 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:19.735 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:16:19.773 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:19.768+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:19.874 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:19.868+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:19.898 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:19.898 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:19.912 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:16:20.002 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:20.000+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:20.075 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:20.076+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:20.080 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:20.080 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:20.093 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:16:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:marked out osd.2. 2024-09-14T06:16:20.163 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-09-14T06:16:20.180 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:20.172+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:20.266 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:20.267 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:20.277 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:20.272+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:20.279 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:16:20.378 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:20.376+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:20.445 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:20.445 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:20.459 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:16:20.480 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:20.476+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:20.581 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:20.580+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:20.582 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:20.582 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:20.596 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:16:20.681 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:20.676+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:20.724 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:20.724 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:20.738 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:16:20.786 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:20.784+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:20.871 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:20.871 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:20.883 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:20.880+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:20.885 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:16:20.984 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:20.980+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:21.046 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:21.046 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:21.061 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:16:21.086 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:21.080+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:21.187 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:21.184+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:21.231 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:21.232 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:21.246 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:16:21.266 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:16:21.314 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:21.312+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:21.389 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:21.384+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:21.411 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:21.411 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:21.423 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:21.424 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:21.425 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:16:21.441 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:21.441 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:21.457 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:21.457 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:21.474 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:21.474 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:21.489 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:21.484+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:21.495 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:21.495 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:21.513 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:21.513 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:21.532 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:21.533 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:21.552 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:21.596 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:21.596 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:21.597 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:16:21.597 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:16:21.597 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:21.612 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:16:21.615 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:21.612+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:21.691 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:21.688+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:21.731 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:21.731 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:16:21.731 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:16:21.731 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:16:21.731 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.731 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:22 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'12, mlcod=17'8)", 2024-09-14T06:16:21.731 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.677941+0000", 2024-09-14T06:16:21.731 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 17.053134973999999, 2024-09-14T06:16:21.731 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003676669, 2024-09-14T06:16:21.731 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.731 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.731 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.732 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.732 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.732 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.732 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.677941+0000", 2024-09-14T06:16:21.732 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.732 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.732 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.732 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.732 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.677941+0000", 2024-09-14T06:16:21.732 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.732 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.732 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.733 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.733 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.677943+0000", 2024-09-14T06:16:21.733 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.844e-06 2024-09-14T06:16:21.733 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.733 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.733 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.733 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.678078+0000", 2024-09-14T06:16:21.733 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013481499999999999 2024-09-14T06:16:21.733 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.733 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.733 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.733 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.678104+0000", 2024-09-14T06:16:21.733 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6597e-05 2024-09-14T06:16:21.733 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.734 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.734 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.734 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.678119+0000", 2024-09-14T06:16:21.734 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.502e-05 2024-09-14T06:16:21.734 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.734 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.734 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.734 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.678215+0000", 2024-09-14T06:16:21.734 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5501999999999995e-05 2024-09-14T06:16:21.734 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.734 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.734 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.734 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.678265+0000", 2024-09-14T06:16:21.735 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0182999999999997e-05 2024-09-14T06:16:21.735 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.735 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.735 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.735 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.681513+0000", 2024-09-14T06:16:21.735 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032476720000000001 2024-09-14T06:16:21.735 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.735 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.735 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.735 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.681618+0000", 2024-09-14T06:16:21.735 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000105036 2024-09-14T06:16:21.735 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.735 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.735 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.736 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.736 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.736 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:23 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'13, mlcod=17'8)", 2024-09-14T06:16:21.736 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.678623+0000", 2024-09-14T06:16:21.736 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 17.052453392, 2024-09-14T06:16:21.736 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033827169999999999, 2024-09-14T06:16:21.736 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.736 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.736 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.736 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.736 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.736 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.736 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.678623+0000", 2024-09-14T06:16:21.737 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.737 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.737 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.737 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.737 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.678623+0000", 2024-09-14T06:16:21.737 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.737 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.737 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.737 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.737 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.678623+0000", 2024-09-14T06:16:21.737 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1999999999999998e-07 2024-09-14T06:16:21.737 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.737 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.737 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.737 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.678729+0000", 2024-09-14T06:16:21.738 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00010524200000000001 2024-09-14T06:16:21.738 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.738 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.738 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.738 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.678761+0000", 2024-09-14T06:16:21.738 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2604000000000002e-05 2024-09-14T06:16:21.738 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.738 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.738 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.738 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.678771+0000", 2024-09-14T06:16:21.738 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9660000000000008e-06 2024-09-14T06:16:21.738 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.738 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.738 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.739 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.679046+0000", 2024-09-14T06:16:21.739 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00027508099999999999 2024-09-14T06:16:21.739 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.739 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.739 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.739 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.679086+0000", 2024-09-14T06:16:21.739 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0080000000000003e-05 2024-09-14T06:16:21.739 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.739 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.739 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.739 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.681956+0000", 2024-09-14T06:16:21.739 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.002869234 2024-09-14T06:16:21.739 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.739 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.740 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.740 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.682005+0000", 2024-09-14T06:16:21.740 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9589999999999998e-05 2024-09-14T06:16:21.740 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.740 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.740 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.740 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.740 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.740 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:25 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'15, mlcod=17'9)", 2024-09-14T06:16:21.740 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.680930+0000", 2024-09-14T06:16:21.740 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 17.050146186999999, 2024-09-14T06:16:21.740 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040262800000000001, 2024-09-14T06:16:21.740 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.740 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.741 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.741 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.741 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.741 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.741 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.680930+0000", 2024-09-14T06:16:21.741 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.741 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.741 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.741 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.741 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.680930+0000", 2024-09-14T06:16:21.741 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.741 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.741 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.741 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.742 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.680931+0000", 2024-09-14T06:16:21.742 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6470000000000001e-06 2024-09-14T06:16:21.742 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.742 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.742 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.742 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.680943+0000", 2024-09-14T06:16:21.742 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1457000000000001e-05 2024-09-14T06:16:21.742 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.742 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.742 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.742 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.680975+0000", 2024-09-14T06:16:21.742 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1952999999999997e-05 2024-09-14T06:16:21.742 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.742 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.743 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.743 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.680994+0000", 2024-09-14T06:16:21.743 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8746e-05 2024-09-14T06:16:21.743 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.743 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.743 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.743 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.681123+0000", 2024-09-14T06:16:21.743 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000128999 2024-09-14T06:16:21.743 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.743 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.743 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.743 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.681185+0000", 2024-09-14T06:16:21.743 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2372999999999994e-05 2024-09-14T06:16:21.743 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.744 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.744 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.744 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.684872+0000", 2024-09-14T06:16:21.744 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003687025 2024-09-14T06:16:21.744 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.744 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.744 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.744 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.684956+0000", 2024-09-14T06:16:21.744 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:16:21.744 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.744 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.744 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.744 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.744 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.745 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:26 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'16, mlcod=17'14)", 2024-09-14T06:16:21.745 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.686211+0000", 2024-09-14T06:16:21.745 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 17.044864574000002, 2024-09-14T06:16:21.745 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047643770000000002, 2024-09-14T06:16:21.745 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.745 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.745 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.745 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.745 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.745 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.745 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.686211+0000", 2024-09-14T06:16:21.745 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.745 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.746 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.746 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.746 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.686211+0000", 2024-09-14T06:16:21.746 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.746 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.746 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.746 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.746 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.686213+0000", 2024-09-14T06:16:21.746 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6869999999999999e-06 2024-09-14T06:16:21.746 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.746 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.746 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.746 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.686224+0000", 2024-09-14T06:16:21.746 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1039999999999999e-05 2024-09-14T06:16:21.746 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.747 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.747 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.747 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.686253+0000", 2024-09-14T06:16:21.747 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8668999999999999e-05 2024-09-14T06:16:21.747 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.747 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.747 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.747 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.686267+0000", 2024-09-14T06:16:21.747 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3917000000000001e-05 2024-09-14T06:16:21.747 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.747 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.747 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.747 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.686383+0000", 2024-09-14T06:16:21.748 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000116396 2024-09-14T06:16:21.748 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.748 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.748 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.748 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.686418+0000", 2024-09-14T06:16:21.748 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5145999999999999e-05 2024-09-14T06:16:21.748 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.748 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.748 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.748 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.690886+0000", 2024-09-14T06:16:21.748 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044679819999999997 2024-09-14T06:16:21.748 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.749 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.749 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.749 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.690976+0000", 2024-09-14T06:16:21.749 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.954e-05 2024-09-14T06:16:21.749 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.749 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.749 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.749 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.749 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.749 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:40 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'22, mlcod=17'19)", 2024-09-14T06:16:21.749 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.777169+0000", 2024-09-14T06:16:21.749 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 16.953907292, 2024-09-14T06:16:21.749 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.004163941, 2024-09-14T06:16:21.750 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.750 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.750 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.750 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.750 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.750 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.750 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.777169+0000", 2024-09-14T06:16:21.750 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.750 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.750 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.750 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.750 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.777169+0000", 2024-09-14T06:16:21.750 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.751 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.751 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.751 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.751 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.777170+0000", 2024-09-14T06:16:21.751 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6390000000000001e-06 2024-09-14T06:16:21.751 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.751 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.751 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.751 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.777209+0000", 2024-09-14T06:16:21.751 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9053e-05 2024-09-14T06:16:21.751 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.751 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.752 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.752 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.777233+0000", 2024-09-14T06:16:21.752 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3819999999999999e-05 2024-09-14T06:16:21.752 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.752 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.752 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.752 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.777249+0000", 2024-09-14T06:16:21.752 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.552e-05 2024-09-14T06:16:21.752 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.752 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.752 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.752 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.777343+0000", 2024-09-14T06:16:21.752 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3826000000000006e-05 2024-09-14T06:16:21.752 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.753 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.753 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.753 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.777378+0000", 2024-09-14T06:16:21.753 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5243000000000001e-05 2024-09-14T06:16:21.753 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.753 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.753 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.753 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.781234+0000", 2024-09-14T06:16:21.753 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003856431 2024-09-14T06:16:21.753 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.753 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.753 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.753 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.781333+0000", 2024-09-14T06:16:21.753 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.8408999999999999e-05 2024-09-14T06:16:21.754 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.754 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.754 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.754 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.754 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.754 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:41 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'23, mlcod=17'19)", 2024-09-14T06:16:21.754 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.777720+0000", 2024-09-14T06:16:21.754 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 16.953355797, 2024-09-14T06:16:21.754 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0079088760000000004, 2024-09-14T06:16:21.754 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.754 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.754 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.754 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.754 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.754 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.755 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.777720+0000", 2024-09-14T06:16:21.755 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.755 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.755 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.755 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.755 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.777720+0000", 2024-09-14T06:16:21.755 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.755 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.755 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.755 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.755 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.777722+0000", 2024-09-14T06:16:21.755 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.407e-06 2024-09-14T06:16:21.755 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.755 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.755 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.756 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.777770+0000", 2024-09-14T06:16:21.756 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8412999999999999e-05 2024-09-14T06:16:21.756 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.756 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.756 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.756 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.777787+0000", 2024-09-14T06:16:21.756 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7003e-05 2024-09-14T06:16:21.756 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.756 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.756 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.756 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.777800+0000", 2024-09-14T06:16:21.756 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.289e-05 2024-09-14T06:16:21.756 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.756 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.757 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.757 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.778111+0000", 2024-09-14T06:16:21.757 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000311284 2024-09-14T06:16:21.757 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.757 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.757 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.757 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.778149+0000", 2024-09-14T06:16:21.757 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8269999999999998e-05 2024-09-14T06:16:21.757 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.757 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.757 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.757 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.785510+0000", 2024-09-14T06:16:21.757 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0073604229999999996 2024-09-14T06:16:21.757 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.757 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.757 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.758 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.785629+0000", 2024-09-14T06:16:21.758 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00011918599999999999 2024-09-14T06:16:21.758 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.758 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.758 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.758 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.758 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.758 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:42 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'24, mlcod=17'19)", 2024-09-14T06:16:21.758 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.778337+0000", 2024-09-14T06:16:21.758 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 16.952738667999999, 2024-09-14T06:16:21.758 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0073910829999999997, 2024-09-14T06:16:21.758 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.758 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.758 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.758 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.759 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.759 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.759 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.778337+0000", 2024-09-14T06:16:21.759 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.759 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.759 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.759 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.759 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.778337+0000", 2024-09-14T06:16:21.759 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.759 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.759 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.759 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.759 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.778339+0000", 2024-09-14T06:16:21.759 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3540000000000001e-06 2024-09-14T06:16:21.760 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.760 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.760 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.760 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.778349+0000", 2024-09-14T06:16:21.760 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0396e-05 2024-09-14T06:16:21.760 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.760 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.760 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.760 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.778457+0000", 2024-09-14T06:16:21.760 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00010803599999999999 2024-09-14T06:16:21.760 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.760 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.760 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.760 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.778480+0000", 2024-09-14T06:16:21.760 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3065999999999999e-05 2024-09-14T06:16:21.761 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.761 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.761 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.761 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.779222+0000", 2024-09-14T06:16:21.761 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00074136799999999995 2024-09-14T06:16:21.761 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.761 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.761 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.761 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.779262+0000", 2024-09-14T06:16:21.761 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0633000000000002e-05 2024-09-14T06:16:21.761 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.761 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.761 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.761 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.785637+0000", 2024-09-14T06:16:21.762 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0063744470000000001 2024-09-14T06:16:21.762 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.762 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.762 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.762 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.785728+0000", 2024-09-14T06:16:21.762 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1782999999999999e-05 2024-09-14T06:16:21.762 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.762 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.762 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.762 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.762 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.762 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:43 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'25, mlcod=17'19)", 2024-09-14T06:16:21.762 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.778803+0000", 2024-09-14T06:16:21.762 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 16.952272555, 2024-09-14T06:16:21.763 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0070142090000000004, 2024-09-14T06:16:21.763 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.763 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.763 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.763 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.763 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.763 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.763 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.778803+0000", 2024-09-14T06:16:21.763 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.763 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.763 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.763 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.763 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.778803+0000", 2024-09-14T06:16:21.763 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.764 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.764 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.764 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.764 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.778805+0000", 2024-09-14T06:16:21.764 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3400000000000001e-06 2024-09-14T06:16:21.764 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.764 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.764 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.764 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.778817+0000", 2024-09-14T06:16:21.764 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2527000000000001e-05 2024-09-14T06:16:21.764 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.764 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.764 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.764 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.778841+0000", 2024-09-14T06:16:21.765 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4165999999999999e-05 2024-09-14T06:16:21.765 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.765 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.765 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.765 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.778857+0000", 2024-09-14T06:16:21.765 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5837000000000001e-05 2024-09-14T06:16:21.765 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.765 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.765 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.765 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.779842+0000", 2024-09-14T06:16:21.765 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00098503499999999999 2024-09-14T06:16:21.765 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.765 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.765 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.766 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.779880+0000", 2024-09-14T06:16:21.766 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7784000000000003e-05 2024-09-14T06:16:21.766 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.766 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.766 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.766 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.785735+0000", 2024-09-14T06:16:21.766 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0058544679999999998 2024-09-14T06:16:21.766 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.766 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.766 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.766 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.785818+0000", 2024-09-14T06:16:21.766 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3052000000000004e-05 2024-09-14T06:16:21.766 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.766 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.767 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.767 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.767 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.767 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:44 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'26, mlcod=17'20)", 2024-09-14T06:16:21.767 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.779511+0000", 2024-09-14T06:16:21.767 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 16.951564600000001, 2024-09-14T06:16:21.767 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0063804869999999998, 2024-09-14T06:16:21.767 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.767 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.767 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.767 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.767 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.767 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.767 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.779511+0000", 2024-09-14T06:16:21.767 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.768 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.768 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.768 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.768 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.779511+0000", 2024-09-14T06:16:21.768 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.768 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.768 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.768 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.768 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.779513+0000", 2024-09-14T06:16:21.768 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8169999999999999e-06 2024-09-14T06:16:21.768 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.768 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.768 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.768 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.779564+0000", 2024-09-14T06:16:21.769 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0442000000000003e-05 2024-09-14T06:16:21.769 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.769 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.769 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.769 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.779596+0000", 2024-09-14T06:16:21.769 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2480000000000001e-05 2024-09-14T06:16:21.769 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.769 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.769 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.769 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.779613+0000", 2024-09-14T06:16:21.769 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6443000000000002e-05 2024-09-14T06:16:21.769 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.769 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.770 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.770 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.780473+0000", 2024-09-14T06:16:21.770 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00086078700000000001 2024-09-14T06:16:21.770 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.770 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.770 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.770 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.780513+0000", 2024-09-14T06:16:21.770 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9573000000000002e-05 2024-09-14T06:16:21.770 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.770 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.770 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.770 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.785823+0000", 2024-09-14T06:16:21.770 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0053102519999999997 2024-09-14T06:16:21.770 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.771 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.771 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.771 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.785892+0000", 2024-09-14T06:16:21.771 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8693000000000001e-05 2024-09-14T06:16:21.771 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.771 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.771 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.771 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.771 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.771 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:45 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'27, mlcod=17'20)", 2024-09-14T06:16:21.771 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.780029+0000", 2024-09-14T06:16:21.771 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 16.951046820999998, 2024-09-14T06:16:21.771 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0077786299999999999, 2024-09-14T06:16:21.771 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.771 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.772 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.772 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.772 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.772 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.772 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.780029+0000", 2024-09-14T06:16:21.772 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.772 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.772 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.772 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.772 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.780029+0000", 2024-09-14T06:16:21.772 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.772 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.772 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.772 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.772 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.780031+0000", 2024-09-14T06:16:21.772 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4470000000000001e-06 2024-09-14T06:16:21.773 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.773 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.773 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.773 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.780042+0000", 2024-09-14T06:16:21.773 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1039999999999999e-05 2024-09-14T06:16:21.773 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.773 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.773 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.773 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.780067+0000", 2024-09-14T06:16:21.773 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5456000000000002e-05 2024-09-14T06:16:21.773 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.773 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.773 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.773 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.780084+0000", 2024-09-14T06:16:21.774 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6520000000000001e-05 2024-09-14T06:16:21.774 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.774 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.774 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.774 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.781461+0000", 2024-09-14T06:16:21.774 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00137718 2024-09-14T06:16:21.774 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.774 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.774 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.774 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.781497+0000", 2024-09-14T06:16:21.774 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6579000000000003e-05 2024-09-14T06:16:21.774 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.774 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.774 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.774 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.787702+0000", 2024-09-14T06:16:21.775 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.006204205 2024-09-14T06:16:21.775 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.775 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.775 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.775 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.787808+0000", 2024-09-14T06:16:21.775 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000106203 2024-09-14T06:16:21.775 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.775 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.775 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.775 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.775 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.775 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:46 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'28, mlcod=17'20)", 2024-09-14T06:16:21.775 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.780453+0000", 2024-09-14T06:16:21.775 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 16.950623286999999, 2024-09-14T06:16:21.775 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.007523414, 2024-09-14T06:16:21.775 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.776 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.776 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.776 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.776 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.776 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.776 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.780453+0000", 2024-09-14T06:16:21.776 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.776 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.776 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.776 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.776 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.780453+0000", 2024-09-14T06:16:21.776 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.776 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.776 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.776 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.777 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.780454+0000", 2024-09-14T06:16:21.777 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4360000000000001e-06 2024-09-14T06:16:21.777 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.777 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.777 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.777 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.780464+0000", 2024-09-14T06:16:21.777 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0144e-05 2024-09-14T06:16:21.777 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.777 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.777 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.777 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.780481+0000", 2024-09-14T06:16:21.777 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6688999999999999e-05 2024-09-14T06:16:21.777 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.777 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.777 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.778 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.780521+0000", 2024-09-14T06:16:21.778 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9647000000000002e-05 2024-09-14T06:16:21.778 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.778 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.778 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.778 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.782098+0000", 2024-09-14T06:16:21.778 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001577328 2024-09-14T06:16:21.778 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.778 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.778 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.778 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.782130+0000", 2024-09-14T06:16:21.778 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2385999999999998e-05 2024-09-14T06:16:21.778 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.778 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.778 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.778 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.787818+0000", 2024-09-14T06:16:21.779 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056877489999999998 2024-09-14T06:16:21.779 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.779 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.779 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.779 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.787976+0000", 2024-09-14T06:16:21.779 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000158035 2024-09-14T06:16:21.779 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.779 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.779 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.779 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.779 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.779 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:47 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'29, mlcod=17'20)", 2024-09-14T06:16:21.779 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.780998+0000", 2024-09-14T06:16:21.779 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 16.950078079000001, 2024-09-14T06:16:21.779 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0070700219999999996, 2024-09-14T06:16:21.780 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.780 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.780 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.780 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.780 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.780 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.780 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.780998+0000", 2024-09-14T06:16:21.780 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.780 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.780 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.780 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.780 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.780998+0000", 2024-09-14T06:16:21.780 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.780 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.781 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.781 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.781 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.781000+0000", 2024-09-14T06:16:21.781 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6700000000000001e-06 2024-09-14T06:16:21.781 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.781 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.781 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.781 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.781082+0000", 2024-09-14T06:16:21.781 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2644000000000004e-05 2024-09-14T06:16:21.781 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.781 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.781 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.781 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.781107+0000", 2024-09-14T06:16:21.781 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4566000000000002e-05 2024-09-14T06:16:21.782 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.782 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.782 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.782 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.781120+0000", 2024-09-14T06:16:21.782 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.309e-05 2024-09-14T06:16:21.782 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.782 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.782 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.782 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.782808+0000", 2024-09-14T06:16:21.782 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0016883759999999999 2024-09-14T06:16:21.782 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.782 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.782 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.782 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.782842+0000", 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3694e-05 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.787983+0000", 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0051409000000000003 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.788068+0000", 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5081999999999996e-05 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.783 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'20)", 2024-09-14T06:16:21.784 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.781521+0000", 2024-09-14T06:16:21.784 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 16.949554892999998, 2024-09-14T06:16:21.784 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0066364789999999998, 2024-09-14T06:16:21.784 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.784 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.784 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.784 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.784 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.784 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.784 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.781521+0000", 2024-09-14T06:16:21.784 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.784 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.784 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.784 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.785 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.781521+0000", 2024-09-14T06:16:21.785 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.785 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.785 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.785 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.785 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.781522+0000", 2024-09-14T06:16:21.785 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3540000000000001e-06 2024-09-14T06:16:21.785 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.785 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.785 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.785 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.781559+0000", 2024-09-14T06:16:21.785 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6726e-05 2024-09-14T06:16:21.785 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.785 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.786 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.786 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.781587+0000", 2024-09-14T06:16:21.786 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8124e-05 2024-09-14T06:16:21.786 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.786 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.786 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.786 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.781604+0000", 2024-09-14T06:16:21.786 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6365999999999999e-05 2024-09-14T06:16:21.786 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.786 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.786 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.786 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784118+0000", 2024-09-14T06:16:21.786 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025142900000000002 2024-09-14T06:16:21.786 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.787 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.787 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.787 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784150+0000", 2024-09-14T06:16:21.787 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2564000000000002e-05 2024-09-14T06:16:21.787 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.787 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.787 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.787 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.788075+0000", 2024-09-14T06:16:21.787 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039245620000000004 2024-09-14T06:16:21.787 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.787 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.787 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.787 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.788158+0000", 2024-09-14T06:16:21.787 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2492999999999998e-05 2024-09-14T06:16:21.788 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.788 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.788 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.788 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.788 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.788 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'21)", 2024-09-14T06:16:21.788 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.782160+0000", 2024-09-14T06:16:21.788 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 16.948916320999999, 2024-09-14T06:16:21.788 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0060996160000000004, 2024-09-14T06:16:21.788 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.788 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.788 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.788 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.788 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.789 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.789 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.782160+0000", 2024-09-14T06:16:21.789 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.789 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.789 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.789 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.789 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.782160+0000", 2024-09-14T06:16:21.789 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.789 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.789 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.789 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.789 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.782161+0000", 2024-09-14T06:16:21.789 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.513e-06 2024-09-14T06:16:21.789 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.789 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.790 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.790 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.782173+0000", 2024-09-14T06:16:21.790 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.145e-05 2024-09-14T06:16:21.790 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.790 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.790 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.790 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.782195+0000", 2024-09-14T06:16:21.790 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2189999999999999e-05 2024-09-14T06:16:21.790 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.790 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.790 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.790 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.782215+0000", 2024-09-14T06:16:21.790 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9993999999999999e-05 2024-09-14T06:16:21.790 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.791 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.791 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.791 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784708+0000", 2024-09-14T06:16:21.791 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024930429999999999 2024-09-14T06:16:21.791 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.791 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.791 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.791 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784765+0000", 2024-09-14T06:16:21.791 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.677e-05 2024-09-14T06:16:21.791 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.791 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.791 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.791 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.788163+0000", 2024-09-14T06:16:21.791 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003398444 2024-09-14T06:16:21.792 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.792 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.792 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.792 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.788259+0000", 2024-09-14T06:16:21.792 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6212000000000007e-05 2024-09-14T06:16:21.792 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.792 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.792 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.792 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.792 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.792 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'21)", 2024-09-14T06:16:21.792 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.784080+0000", 2024-09-14T06:16:21.792 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 16.946995949000002, 2024-09-14T06:16:21.793 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042677899999999996, 2024-09-14T06:16:21.793 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.793 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.793 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.793 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.793 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.793 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.793 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784080+0000", 2024-09-14T06:16:21.793 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.793 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.793 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.793 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.793 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784080+0000", 2024-09-14T06:16:21.793 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.794 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.794 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.794 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.794 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784082+0000", 2024-09-14T06:16:21.794 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.75e-06 2024-09-14T06:16:21.794 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.794 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.794 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.794 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784328+0000", 2024-09-14T06:16:21.794 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024582100000000002 2024-09-14T06:16:21.794 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.794 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.794 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.794 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784349+0000", 2024-09-14T06:16:21.795 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1590000000000002e-05 2024-09-14T06:16:21.795 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.795 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.795 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.795 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784375+0000", 2024-09-14T06:16:21.795 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5743000000000001e-05 2024-09-14T06:16:21.795 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.795 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.795 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.795 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.785995+0000", 2024-09-14T06:16:21.795 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0016202740000000001 2024-09-14T06:16:21.796 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.796 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.796 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.796 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.786032+0000", 2024-09-14T06:16:21.796 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6310000000000003e-05 2024-09-14T06:16:21.796 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.796 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.796 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.796 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.788268+0000", 2024-09-14T06:16:21.796 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0022362200000000001 2024-09-14T06:16:21.796 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.796 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.796 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.796 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.788348+0000", 2024-09-14T06:16:21.797 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0081999999999997e-05 2024-09-14T06:16:21.797 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.797 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.797 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.797 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.797 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.797 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'21)", 2024-09-14T06:16:21.797 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.784404+0000", 2024-09-14T06:16:21.797 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 16.946671678000001, 2024-09-14T06:16:21.797 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.004579797, 2024-09-14T06:16:21.797 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.797 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.797 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.797 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.797 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.798 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.798 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784404+0000", 2024-09-14T06:16:21.798 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.798 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.798 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.798 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.798 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784404+0000", 2024-09-14T06:16:21.798 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.798 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.798 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.798 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.798 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784405+0000", 2024-09-14T06:16:21.798 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6700000000000002e-07 2024-09-14T06:16:21.798 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.798 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.799 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.799 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784412+0000", 2024-09-14T06:16:21.799 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.5529999999999998e-06 2024-09-14T06:16:21.799 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.799 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.799 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.799 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784428+0000", 2024-09-14T06:16:21.799 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5917e-05 2024-09-14T06:16:21.799 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.799 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.799 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.799 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784436+0000", 2024-09-14T06:16:21.799 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1329999999999999e-06 2024-09-14T06:16:21.799 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.800 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.800 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.800 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.786969+0000", 2024-09-14T06:16:21.800 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025334899999999998 2024-09-14T06:16:21.800 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.800 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.800 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.800 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.787028+0000", 2024-09-14T06:16:21.800 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8746000000000003e-05 2024-09-14T06:16:21.800 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.800 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.800 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.800 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.788886+0000", 2024-09-14T06:16:21.800 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001857542 2024-09-14T06:16:21.801 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.801 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.801 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.801 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.788984+0000", 2024-09-14T06:16:21.801 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.8548999999999994e-05 2024-09-14T06:16:21.801 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.801 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.801 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.801 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.801 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.801 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'21)", 2024-09-14T06:16:21.801 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.784451+0000", 2024-09-14T06:16:21.801 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 16.946625112, 2024-09-14T06:16:21.801 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0050513240000000003, 2024-09-14T06:16:21.802 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.802 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.802 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.802 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.802 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.802 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.802 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784451+0000", 2024-09-14T06:16:21.802 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.802 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.802 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.802 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.802 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784451+0000", 2024-09-14T06:16:21.802 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.803 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.803 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.803 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.803 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784451+0000", 2024-09-14T06:16:21.803 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0699999999999998e-07 2024-09-14T06:16:21.803 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.803 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.803 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.803 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784474+0000", 2024-09-14T06:16:21.803 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2376000000000001e-05 2024-09-14T06:16:21.803 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.803 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.803 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.803 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784503+0000", 2024-09-14T06:16:21.803 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9037e-05 2024-09-14T06:16:21.804 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.804 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.804 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.804 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.784513+0000", 2024-09-14T06:16:21.804 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0606e-05 2024-09-14T06:16:21.804 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.804 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.804 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.804 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.788433+0000", 2024-09-14T06:16:21.804 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039201899999999996 2024-09-14T06:16:21.804 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.804 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.804 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.805 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.788530+0000", 2024-09-14T06:16:21.805 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6158999999999999e-05 2024-09-14T06:16:21.805 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.805 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.805 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.805 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.789440+0000", 2024-09-14T06:16:21.805 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00091005000000000003 2024-09-14T06:16:21.805 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.805 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.805 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.805 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.789502+0000", 2024-09-14T06:16:21.805 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2498999999999999e-05 2024-09-14T06:16:21.805 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.805 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.806 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.806 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.806 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.806 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:53 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'35, mlcod=17'33)", 2024-09-14T06:16:21.806 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.790489+0000", 2024-09-14T06:16:21.806 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 16.940586839000002, 2024-09-14T06:16:21.806 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041219689999999996, 2024-09-14T06:16:21.806 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.806 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.806 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.806 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.806 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.806 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.806 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.790489+0000", 2024-09-14T06:16:21.807 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.807 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.807 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.807 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.807 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.790489+0000", 2024-09-14T06:16:21.807 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.807 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.807 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.807 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.807 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.790491+0000", 2024-09-14T06:16:21.807 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8840000000000001e-06 2024-09-14T06:16:21.807 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.807 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.807 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.807 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.790598+0000", 2024-09-14T06:16:21.808 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000106999 2024-09-14T06:16:21.808 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.808 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.808 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.808 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.790628+0000", 2024-09-14T06:16:21.808 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9899e-05 2024-09-14T06:16:21.808 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.808 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.808 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.808 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.790652+0000", 2024-09-14T06:16:21.808 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4130000000000001e-05 2024-09-14T06:16:21.808 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.808 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.808 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.809 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.790746+0000", 2024-09-14T06:16:21.809 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3826000000000006e-05 2024-09-14T06:16:21.809 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.809 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.809 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.809 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.790782+0000", 2024-09-14T06:16:21.809 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6473000000000001e-05 2024-09-14T06:16:21.809 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.809 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.809 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.809 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.794511+0000", 2024-09-14T06:16:21.809 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0037289689999999999 2024-09-14T06:16:21.809 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.809 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.810 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.810 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.794611+0000", 2024-09-14T06:16:21.810 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9789000000000003e-05 2024-09-14T06:16:21.810 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.810 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.810 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.810 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.810 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.810 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:54 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'36, mlcod=17'33)", 2024-09-14T06:16:21.810 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.790832+0000", 2024-09-14T06:16:21.810 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 16.940243831, 2024-09-14T06:16:21.810 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039101199999999996, 2024-09-14T06:16:21.810 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.810 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.811 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.811 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.811 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.811 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.811 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.790832+0000", 2024-09-14T06:16:21.811 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.811 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.811 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.811 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.811 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.790832+0000", 2024-09-14T06:16:21.811 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.811 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.811 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.811 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.811 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.790833+0000", 2024-09-14T06:16:21.812 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.176e-06 2024-09-14T06:16:21.812 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.812 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.812 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.812 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.790927+0000", 2024-09-14T06:16:21.812 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3616000000000006e-05 2024-09-14T06:16:21.812 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.812 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.812 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.812 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.790943+0000", 2024-09-14T06:16:21.812 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5722999999999998e-05 2024-09-14T06:16:21.812 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.812 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.812 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.813 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.790952+0000", 2024-09-14T06:16:21.813 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2399999999999996e-06 2024-09-14T06:16:21.813 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.813 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.813 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.813 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.791430+0000", 2024-09-14T06:16:21.813 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00047763299999999999 2024-09-14T06:16:21.813 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.813 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.813 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.813 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.791489+0000", 2024-09-14T06:16:21.813 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9333000000000003e-05 2024-09-14T06:16:21.813 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.813 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.814 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.816 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.794630+0000", 2024-09-14T06:16:21.816 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0031410660000000001 2024-09-14T06:16:21.816 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.817 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.817 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.817 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.794742+0000", 2024-09-14T06:16:21.817 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000112333 2024-09-14T06:16:21.817 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.817 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.817 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.817 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.817 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.817 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4170.0:55 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'37, mlcod=17'33)", 2024-09-14T06:16:21.817 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:04.791301+0000", 2024-09-14T06:16:21.817 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 16.939774475, 2024-09-14T06:16:21.817 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0035366859999999998, 2024-09-14T06:16:21.817 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:21.817 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:21.818 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:21.818 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:21.818 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.818 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:21.818 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.791301+0000", 2024-09-14T06:16:21.818 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.818 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.818 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.818 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:21.818 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.791301+0000", 2024-09-14T06:16:21.818 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:21.818 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.818 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.818 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:21.818 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.791303+0000", 2024-09-14T06:16:21.819 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2929999999999999e-06 2024-09-14T06:16:21.819 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.819 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.819 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:21.819 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.791462+0000", 2024-09-14T06:16:21.819 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015971599999999999 2024-09-14T06:16:21.819 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.819 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.819 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:21.819 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.791513+0000", 2024-09-14T06:16:21.819 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0076e-05 2024-09-14T06:16:21.819 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.819 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.819 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:21.820 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.791529+0000", 2024-09-14T06:16:21.820 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.632e-05 2024-09-14T06:16:21.820 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.820 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.820 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:21.820 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.792469+0000", 2024-09-14T06:16:21.820 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00093965200000000002 2024-09-14T06:16:21.820 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.820 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.820 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:21.820 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.792552+0000", 2024-09-14T06:16:21.820 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3473000000000005e-05 2024-09-14T06:16:21.820 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.820 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.821 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:21.821 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.794746+0000", 2024-09-14T06:16:21.821 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0021940839999999998 2024-09-14T06:16:21.821 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:21.821 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:21.821 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:21.821 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:04.794838+0000", 2024-09-14T06:16:21.821 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2071999999999995e-05 2024-09-14T06:16:21.821 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.821 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.821 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.821 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:21.821 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:21.821 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:21.823 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:16:21.823 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:21.820+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:21.924 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:21.920+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:21.993 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:21.993 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:21.993 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:16:21.993 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:22.007 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:16:22.050 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:22.048+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:22.126 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:22.120+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:22.168 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:22.169 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:22.169 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:16:22.169 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:16:22.169 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:22.182 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:16:22.198 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:16:22.243 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:22.240+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:22.309 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:22.310 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:16:22.310 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:16:22.310 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:16:22.310 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.310 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4214.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T06:16:22.310 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:11.166477+0000", 2024-09-14T06:16:22.310 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 11.145024286, 2024-09-14T06:16:22.310 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0021449770000000002, 2024-09-14T06:16:22.310 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:22.310 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:22.310 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:22.310 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:22.310 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.311 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:22.311 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.166477+0000", 2024-09-14T06:16:22.311 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:22.311 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.311 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.311 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:22.311 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.166477+0000", 2024-09-14T06:16:22.311 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:22.311 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.311 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.311 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:22.311 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.166477+0000", 2024-09-14T06:16:22.311 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4e-07 2024-09-14T06:16:22.312 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.312 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.312 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:22.312 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.166488+0000", 2024-09-14T06:16:22.312 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0823e-05 2024-09-14T06:16:22.312 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.312 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.312 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:22.312 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.166533+0000", 2024-09-14T06:16:22.312 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4699e-05 2024-09-14T06:16:22.312 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.312 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.312 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:22.313 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.166576+0000", 2024-09-14T06:16:22.313 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3319999999999999e-05 2024-09-14T06:16:22.313 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.313 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.313 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:22.313 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.166735+0000", 2024-09-14T06:16:22.313 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000158386 2024-09-14T06:16:22.313 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.313 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.313 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:22.313 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.166790+0000", 2024-09-14T06:16:22.313 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.5612999999999998e-05 2024-09-14T06:16:22.313 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.314 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.314 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:22.314 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.168535+0000", 2024-09-14T06:16:22.314 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0017443700000000001 2024-09-14T06:16:22.314 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.314 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.314 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:22.314 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.168622+0000", 2024-09-14T06:16:22.314 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6926000000000001e-05 2024-09-14T06:16:22.314 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:22.314 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:22.314 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:22.314 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.314 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.315 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4214.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-14T06:16:22.315 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:12.181799+0000", 2024-09-14T06:16:22.315 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.129702112, 2024-09-14T06:16:22.315 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001894539, 2024-09-14T06:16:22.315 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:22.315 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:22.315 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:22.315 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:22.315 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.315 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:22.315 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.181799+0000", 2024-09-14T06:16:22.315 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:22.315 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.315 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.316 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:22.316 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.181799+0000", 2024-09-14T06:16:22.316 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:22.316 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.316 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.316 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:22.316 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.181800+0000", 2024-09-14T06:16:22.316 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1999999999999998e-07 2024-09-14T06:16:22.316 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.316 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.316 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:22.316 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.181817+0000", 2024-09-14T06:16:22.317 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7255999999999999e-05 2024-09-14T06:16:22.317 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.317 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.317 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:22.317 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.181856+0000", 2024-09-14T06:16:22.317 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.943e-05 2024-09-14T06:16:22.317 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.317 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.317 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:22.317 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.181885+0000", 2024-09-14T06:16:22.317 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.845e-05 2024-09-14T06:16:22.317 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.317 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.318 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:22.318 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.182010+0000", 2024-09-14T06:16:22.318 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000125623 2024-09-14T06:16:22.318 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.318 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.318 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:22.318 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.182062+0000", 2024-09-14T06:16:22.318 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1702000000000001e-05 2024-09-14T06:16:22.318 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.318 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.318 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:22.318 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.183584+0000", 2024-09-14T06:16:22.318 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001522135 2024-09-14T06:16:22.318 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.318 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.319 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:22.319 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.183693+0000", 2024-09-14T06:16:22.319 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000109023 2024-09-14T06:16:22.319 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:22.319 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:22.319 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:22.319 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:22.319 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:22.319 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:22.324 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:16:22.327 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:22.324+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:22.428 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:22.424+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:22.445 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:22.445 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:22.445 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:16:22.446 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:22.460 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:16:22.529 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:22.524+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:22.577 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:22.577 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:22.577 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:16:22.578 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:16:22.578 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:22.591 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:16:22.630 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:22.628+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:22.712 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:22.712 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:16:22.712 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:16:22.712 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:16:22.713 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.713 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:13 3.6 e26/23 3:676c166b:::rbd_object_map.11528c180b7:head v 26'1, mlcod=0'0)", 2024-09-14T06:16:22.713 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:22.215716+0000", 2024-09-14T06:16:22.713 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.498774563, 2024-09-14T06:16:22.713 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00078508199999999999, 2024-09-14T06:16:22.713 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:22.713 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:22.713 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:22.713 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:22.713 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.713 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:22.713 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.215716+0000", 2024-09-14T06:16:22.713 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:22.714 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.714 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.714 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:22.714 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.215716+0000", 2024-09-14T06:16:22.714 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:22.714 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.714 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.714 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:22.714 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.215717+0000", 2024-09-14T06:16:22.714 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.68e-07 2024-09-14T06:16:22.714 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.714 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.714 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:22.714 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.215723+0000", 2024-09-14T06:16:22.714 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:16:22.715 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.715 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.715 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:22.715 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.215736+0000", 2024-09-14T06:16:22.715 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2480000000000001e-05 2024-09-14T06:16:22.715 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.715 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.715 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:22.715 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.215745+0000", 2024-09-14T06:16:22.715 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2329999999999995e-06 2024-09-14T06:16:22.715 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.715 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.715 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:22.715 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.215833+0000", 2024-09-14T06:16:22.716 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8448000000000004e-05 2024-09-14T06:16:22.716 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.716 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.716 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:22.716 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.215862+0000", 2024-09-14T06:16:22.716 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8456999999999999e-05 2024-09-14T06:16:22.716 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.716 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.716 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:22.716 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.216464+0000", 2024-09-14T06:16:22.716 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00060177400000000002 2024-09-14T06:16:22.716 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:22.717 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:22.717 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:22.717 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.216501+0000", 2024-09-14T06:16:22.717 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7588e-05 2024-09-14T06:16:22.717 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:22.717 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:22.717 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:22.717 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:22.717 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:22.717 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:22.726 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:16:22.731 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:22.728+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:22.831 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:22.832+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:22.862 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:22.863 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:22.863 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:22.863 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:22.878 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:16:22.932 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:22.928+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:23.007 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:23.007 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:23.007 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:23.007 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:23.007 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:23.020 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:16:23.034 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:23.036+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:23.134 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:23.132+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:23.184 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:23.185 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:23.185 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:23.185 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:23.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.185 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:10 3.1c e26/23 3:3831950c:::rbd_directory:head v 26'1, mlcod=0'0)", 2024-09-14T06:16:23.185 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:22.204352+0000", 2024-09-14T06:16:23.185 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.97922192600000002, 2024-09-14T06:16:23.185 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0015404419999999999, 2024-09-14T06:16:23.185 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:23.185 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:23.185 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:23.186 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:23.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:23.186 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.204352+0000", 2024-09-14T06:16:23.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:23.186 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.204352+0000", 2024-09-14T06:16:23.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:23.186 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.204352+0000", 2024-09-14T06:16:23.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9100000000000003e-07 2024-09-14T06:16:23.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:23.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.204362+0000", 2024-09-14T06:16:23.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4280000000000002e-06 2024-09-14T06:16:23.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:23.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.204384+0000", 2024-09-14T06:16:23.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2532999999999999e-05 2024-09-14T06:16:23.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:23.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.204396+0000", 2024-09-14T06:16:23.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2013e-05 2024-09-14T06:16:23.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.188 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:23.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.204530+0000", 2024-09-14T06:16:23.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013320000000000001 2024-09-14T06:16:23.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.188 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:23.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.204585+0000", 2024-09-14T06:16:23.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5704000000000001e-05 2024-09-14T06:16:23.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.189 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:23.189 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.205801+0000", 2024-09-14T06:16:23.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001215317 2024-09-14T06:16:23.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.189 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:23.189 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.205892+0000", 2024-09-14T06:16:23.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1456e-05 2024-09-14T06:16:23.189 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.189 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:23.189 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.190 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:12 3.e 3:77456488:::rbd_header.11528c180b7:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T06:16:23.190 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:22.209238+0000", 2024-09-14T06:16:23.190 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.974335807, 2024-09-14T06:16:23.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030347970000000001, 2024-09-14T06:16:23.190 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:23.190 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:23.190 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:23.190 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:23.190 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:23.190 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:23.190 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 12 2024-09-14T06:16:23.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.190 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:23.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:23.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.209238+0000", 2024-09-14T06:16:23.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:23.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.209238+0000", 2024-09-14T06:16:23.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:23.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.209241+0000", 2024-09-14T06:16:23.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7439999999999999e-06 2024-09-14T06:16:23.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:23.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.209260+0000", 2024-09-14T06:16:23.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9703000000000001e-05 2024-09-14T06:16:23.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:23.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.209292+0000", 2024-09-14T06:16:23.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1752999999999999e-05 2024-09-14T06:16:23.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:23.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.209308+0000", 2024-09-14T06:16:23.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5897e-05 2024-09-14T06:16:23.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:23.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.209437+0000", 2024-09-14T06:16:23.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012912899999999999 2024-09-14T06:16:23.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.194 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:23.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.209814+0000", 2024-09-14T06:16:23.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00037658 2024-09-14T06:16:23.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.194 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:23.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.210400+0000", 2024-09-14T06:16:23.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00058675899999999996 2024-09-14T06:16:23.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.194 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:23.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.211690+0000", 2024-09-14T06:16:23.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0012894829999999999 2024-09-14T06:16:23.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.195 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:23.195 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.212044+0000", 2024-09-14T06:16:23.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00035423699999999998 2024-09-14T06:16:23.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:23.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.212142+0000", 2024-09-14T06:16:23.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8152999999999994e-05 2024-09-14T06:16:23.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:23.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.212273+0000", 2024-09-14T06:16:23.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000130359 2024-09-14T06:16:23.196 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.196 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:23.196 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.197 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:12 3.e e26/24 ondisk, result = 0)", 2024-09-14T06:16:23.197 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:22.211842+0000", 2024-09-14T06:16:23.197 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.97173158299999995, 2024-09-14T06:16:23.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00044996899999999999, 2024-09-14T06:16:23.197 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:23.197 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:23.197 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:23.197 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:23.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:23.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.211842+0000", 2024-09-14T06:16:23.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:23.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.211842+0000", 2024-09-14T06:16:23.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:23.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.211844+0000", 2024-09-14T06:16:23.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.703e-06 2024-09-14T06:16:23.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.199 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:23.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.211856+0000", 2024-09-14T06:16:23.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2397e-05 2024-09-14T06:16:23.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.199 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:23.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.211880+0000", 2024-09-14T06:16:23.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4076000000000001e-05 2024-09-14T06:16:23.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.199 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:23.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.211893+0000", 2024-09-14T06:16:23.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2957e-05 2024-09-14T06:16:23.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:23.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.212018+0000", 2024-09-14T06:16:23.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012474200000000001 2024-09-14T06:16:23.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:23.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.212035+0000", 2024-09-14T06:16:23.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7056000000000001e-05 2024-09-14T06:16:23.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.212292+0000", 2024-09-14T06:16:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00025703800000000001 2024-09-14T06:16:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:16 3.e 3:77456488:::rbd_header.11528c180b7:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-14T06:16:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:22.224574+0000", 2024-09-14T06:16:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.95899984699999996, 2024-09-14T06:16:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00049576199999999996, 2024-09-14T06:16:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:23.202 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:23.202 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:23.202 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:23.202 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 16 2024-09-14T06:16:23.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.202 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:23.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:23.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.224574+0000", 2024-09-14T06:16:23.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:23.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.224574+0000", 2024-09-14T06:16:23.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:23.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.224576+0000", 2024-09-14T06:16:23.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6699999999999998e-06 2024-09-14T06:16:23.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:23.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.224590+0000", 2024-09-14T06:16:23.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3086e-05 2024-09-14T06:16:23.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:23.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.224613+0000", 2024-09-14T06:16:23.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3408e-05 2024-09-14T06:16:23.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:23.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.224625+0000", 2024-09-14T06:16:23.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2109999999999999e-05 2024-09-14T06:16:23.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:23.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.224713+0000", 2024-09-14T06:16:23.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8420999999999999e-05 2024-09-14T06:16:23.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:23.205 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.224814+0000", 2024-09-14T06:16:23.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000100549 2024-09-14T06:16:23.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.205 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:23.205 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225070+0000", 2024-09-14T06:16:23.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00025551799999999999 2024-09-14T06:16:23.205 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.205 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:23.205 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.205 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:17 3.e 3:77456488:::rbd_header.11528c180b7:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-14T06:16:23.205 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:22.225246+0000", 2024-09-14T06:16:23.206 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.95832719700000002, 2024-09-14T06:16:23.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00028764200000000001, 2024-09-14T06:16:23.206 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:23.206 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:23.206 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:23.206 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:23.206 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:23.206 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:23.206 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 17 2024-09-14T06:16:23.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.206 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:23.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:23.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225246+0000", 2024-09-14T06:16:23.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:23.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225246+0000", 2024-09-14T06:16:23.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:23.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225248+0000", 2024-09-14T06:16:23.208 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2920000000000001e-06 2024-09-14T06:16:23.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.208 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.208 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:23.208 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225254+0000", 2024-09-14T06:16:23.208 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7819999999999996e-06 2024-09-14T06:16:23.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.208 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.208 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:23.208 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225270+0000", 2024-09-14T06:16:23.208 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5668e-05 2024-09-14T06:16:23.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:23.209 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225278+0000", 2024-09-14T06:16:23.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6029999999999999e-06 2024-09-14T06:16:23.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:23.209 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225342+0000", 2024-09-14T06:16:23.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3807999999999996e-05 2024-09-14T06:16:23.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:23.209 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225412+0000", 2024-09-14T06:16:23.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.046e-05 2024-09-14T06:16:23.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.210 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.210 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:23.210 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225534+0000", 2024-09-14T06:16:23.210 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012202899999999999 2024-09-14T06:16:23.210 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.210 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:23.210 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.210 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.210 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:18 3.e 3:77456488:::rbd_header.11528c180b7:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-14T06:16:23.210 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:22.225691+0000", 2024-09-14T06:16:23.210 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.95788284199999996, 2024-09-14T06:16:23.211 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00030323600000000002, 2024-09-14T06:16:23.211 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:23.211 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:23.211 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:23.211 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:23.211 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:23.211 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:23.211 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 18 2024-09-14T06:16:23.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.211 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:23.211 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.211 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:23.211 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225691+0000", 2024-09-14T06:16:23.211 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.212 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.212 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:23.212 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225691+0000", 2024-09-14T06:16:23.212 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.212 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.212 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:23.212 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225692+0000", 2024-09-14T06:16:23.212 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.78e-07 2024-09-14T06:16:23.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.212 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.212 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:23.212 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225698+0000", 2024-09-14T06:16:23.213 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3169999999999998e-06 2024-09-14T06:16:23.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.213 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.213 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:23.213 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225712+0000", 2024-09-14T06:16:23.213 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4207e-05 2024-09-14T06:16:23.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.213 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.213 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:23.213 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225720+0000", 2024-09-14T06:16:23.213 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9230000000000002e-06 2024-09-14T06:16:23.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.213 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.213 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:23.213 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225784+0000", 2024-09-14T06:16:23.214 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3887999999999995e-05 2024-09-14T06:16:23.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.214 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.214 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:23.214 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225851+0000", 2024-09-14T06:16:23.214 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6981000000000001e-05 2024-09-14T06:16:23.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.214 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.214 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:23.214 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.225994+0000", 2024-09-14T06:16:23.214 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00014294199999999999 2024-09-14T06:16:23.214 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.214 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:23.214 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.215 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:19 3.e 3:77456488:::rbd_header.11528c180b7:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-14T06:16:23.215 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:22.226143+0000", 2024-09-14T06:16:23.215 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.95743087900000001, 2024-09-14T06:16:23.215 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027979399999999998, 2024-09-14T06:16:23.215 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:23.215 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:23.215 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:23.215 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:23.215 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:23.215 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:23.215 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 19 2024-09-14T06:16:23.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.216 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:23.216 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.216 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:23.216 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.226143+0000", 2024-09-14T06:16:23.216 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.216 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.216 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:23.216 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.226143+0000", 2024-09-14T06:16:23.216 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.216 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.217 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:23.217 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.226144+0000", 2024-09-14T06:16:23.217 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0130000000000001e-06 2024-09-14T06:16:23.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.217 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.217 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:23.217 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.226149+0000", 2024-09-14T06:16:23.217 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6189999999999999e-06 2024-09-14T06:16:23.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.217 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.217 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:23.217 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.226163+0000", 2024-09-14T06:16:23.217 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3203999999999999e-05 2024-09-14T06:16:23.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.218 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.218 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:23.218 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.226170+0000", 2024-09-14T06:16:23.218 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6669999999999996e-06 2024-09-14T06:16:23.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.218 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.218 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:23.218 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.226230+0000", 2024-09-14T06:16:23.218 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9487000000000002e-05 2024-09-14T06:16:23.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.218 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.218 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:23.218 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.226293+0000", 2024-09-14T06:16:23.218 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2773000000000004e-05 2024-09-14T06:16:23.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.219 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.219 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:23.219 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.226423+0000", 2024-09-14T06:16:23.219 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013003099999999999 2024-09-14T06:16:23.219 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.219 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:23.219 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.219 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.219 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:20 3.e 3:77456488:::rbd_header.11528c180b7:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-14T06:16:23.219 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:22.227020+0000", 2024-09-14T06:16:23.219 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.95655394100000002, 2024-09-14T06:16:23.219 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000552208, 2024-09-14T06:16:23.219 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:23.220 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:23.220 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:23.220 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:23.220 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:23.220 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:23.220 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 20 2024-09-14T06:16:23.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.220 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:23.220 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.220 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:23.220 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227020+0000", 2024-09-14T06:16:23.220 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.220 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.221 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:23.221 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227020+0000", 2024-09-14T06:16:23.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.221 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.221 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:23.221 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227021+0000", 2024-09-14T06:16:23.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.198e-06 2024-09-14T06:16:23.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.221 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.221 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:23.221 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227042+0000", 2024-09-14T06:16:23.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0993e-05 2024-09-14T06:16:23.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.222 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:23.222 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227059+0000", 2024-09-14T06:16:23.222 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.738e-05 2024-09-14T06:16:23.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.222 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:23.222 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227068+0000", 2024-09-14T06:16:23.222 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7569999999999993e-06 2024-09-14T06:16:23.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.222 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:23.222 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227133+0000", 2024-09-14T06:16:23.222 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4826999999999994e-05 2024-09-14T06:16:23.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.223 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:23.223 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227208+0000", 2024-09-14T06:16:23.223 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5434999999999999e-05 2024-09-14T06:16:23.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.223 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:23.223 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227572+0000", 2024-09-14T06:16:23.223 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036361800000000001 2024-09-14T06:16:23.223 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.223 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:23.223 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.224 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:21 3.e 3:77456488:::rbd_header.11528c180b7:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-14T06:16:23.224 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:22.227740+0000", 2024-09-14T06:16:23.224 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.95583317400000001, 2024-09-14T06:16:23.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00039225399999999998, 2024-09-14T06:16:23.224 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:23.224 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:23.224 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:23.224 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:23.224 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:23.224 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:23.224 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 21 2024-09-14T06:16:23.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.224 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:23.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.225 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:23.225 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227740+0000", 2024-09-14T06:16:23.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.225 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.225 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:23.225 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227740+0000", 2024-09-14T06:16:23.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.225 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.225 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:23.225 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227741+0000", 2024-09-14T06:16:23.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5099999999999998e-07 2024-09-14T06:16:23.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.225 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.226 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:23.226 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227748+0000", 2024-09-14T06:16:23.226 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3090000000000001e-06 2024-09-14T06:16:23.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.226 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.226 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:23.226 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227762+0000", 2024-09-14T06:16:23.226 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3803e-05 2024-09-14T06:16:23.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.226 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.226 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:23.226 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227779+0000", 2024-09-14T06:16:23.226 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7850999999999999e-05 2024-09-14T06:16:23.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.226 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227853+0000", 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4053000000000004e-05 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.227906+0000", 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2765e-05 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.228133+0000", 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000226522 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.228 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:22 3.e 3:77456488:::rbd_header.11528c180b7:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-14T06:16:23.228 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:22.228291+0000", 2024-09-14T06:16:23.228 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.95528212400000001, 2024-09-14T06:16:23.228 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00025391900000000002, 2024-09-14T06:16:23.228 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:23.228 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:23.228 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:23.228 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:23.228 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:23.228 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:23.228 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 22 2024-09-14T06:16:23.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.228 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:23.229 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.229 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:23.229 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.228291+0000", 2024-09-14T06:16:23.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.229 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.229 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:23.229 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.228291+0000", 2024-09-14T06:16:23.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.229 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.229 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:23.229 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.228293+0000", 2024-09-14T06:16:23.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.277e-06 2024-09-14T06:16:23.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.230 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:23.230 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.228299+0000", 2024-09-14T06:16:23.230 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8669999999999999e-06 2024-09-14T06:16:23.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.230 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:23.230 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.228314+0000", 2024-09-14T06:16:23.230 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5466000000000001e-05 2024-09-14T06:16:23.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.230 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:23.230 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.228321+0000", 2024-09-14T06:16:23.230 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9140000000000002e-06 2024-09-14T06:16:23.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:23.231 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.228381+0000", 2024-09-14T06:16:23.231 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9701000000000003e-05 2024-09-14T06:16:23.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:23.231 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.228433+0000", 2024-09-14T06:16:23.231 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2626000000000001e-05 2024-09-14T06:16:23.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:23.231 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.228545+0000", 2024-09-14T06:16:23.231 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000112068 2024-09-14T06:16:23.232 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.232 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:23.232 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.232 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:24 3.e 3:77456488:::rbd_header.11528c180b7:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-14T06:16:23.232 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:22.229342+0000", 2024-09-14T06:16:23.232 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.95423190199999997, 2024-09-14T06:16:23.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000246526, 2024-09-14T06:16:23.232 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:23.232 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:23.232 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:23.232 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:23.232 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:23.232 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:23.233 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 24 2024-09-14T06:16:23.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.233 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:23.233 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.233 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:23.233 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229342+0000", 2024-09-14T06:16:23.233 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.233 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.233 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:23.233 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229342+0000", 2024-09-14T06:16:23.233 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.233 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.234 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:23.234 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229343+0000", 2024-09-14T06:16:23.234 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3859999999999999e-06 2024-09-14T06:16:23.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.234 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.234 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:23.234 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229351+0000", 2024-09-14T06:16:23.234 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4769999999999997e-06 2024-09-14T06:16:23.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.234 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.234 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:23.234 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229368+0000", 2024-09-14T06:16:23.234 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6948000000000001e-05 2024-09-14T06:16:23.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.235 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:23.235 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229376+0000", 2024-09-14T06:16:23.235 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9309999999999997e-06 2024-09-14T06:16:23.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.235 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:23.235 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229435+0000", 2024-09-14T06:16:23.235 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8103999999999998e-05 2024-09-14T06:16:23.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.235 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:23.235 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229486+0000", 2024-09-14T06:16:23.235 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1743999999999998e-05 2024-09-14T06:16:23.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.236 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.236 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:23.236 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229588+0000", 2024-09-14T06:16:23.236 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010193599999999999 2024-09-14T06:16:23.236 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.236 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:23.236 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.236 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.236 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:25 3.e 3:77456488:::rbd_header.11528c180b7:head [watch watch cookie 140175317418864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T06:16:23.236 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:22.229873+0000", 2024-09-14T06:16:23.236 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.95370055799999998, 2024-09-14T06:16:23.236 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001598318, 2024-09-14T06:16:23.237 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:23.237 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:23.237 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:23.237 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:23.237 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:23.237 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:23.237 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 25 2024-09-14T06:16:23.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.237 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:23.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.237 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:23.237 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229873+0000", 2024-09-14T06:16:23.237 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:23.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229873+0000", 2024-09-14T06:16:23.238 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.238 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:23.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229874+0000", 2024-09-14T06:16:23.238 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1090000000000001e-06 2024-09-14T06:16:23.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.238 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:23.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229880+0000", 2024-09-14T06:16:23.238 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6400000000000002e-06 2024-09-14T06:16:23.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.239 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:23.239 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229895+0000", 2024-09-14T06:16:23.239 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5485000000000002e-05 2024-09-14T06:16:23.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.239 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:23.239 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229903+0000", 2024-09-14T06:16:23.239 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3599999999999998e-06 2024-09-14T06:16:23.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.239 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:23.239 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229959+0000", 2024-09-14T06:16:23.239 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6450000000000003e-05 2024-09-14T06:16:23.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.240 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:23.240 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.230011+0000", 2024-09-14T06:16:23.240 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1711e-05 2024-09-14T06:16:23.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.240 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.240 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:23.240 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.230126+0000", 2024-09-14T06:16:23.240 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000115259 2024-09-14T06:16:23.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.240 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.240 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:23.240 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.230975+0000", 2024-09-14T06:16:23.240 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000849108 2024-09-14T06:16:23.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.241 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.241 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:23.241 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.231285+0000", 2024-09-14T06:16:23.241 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00030994900000000001 2024-09-14T06:16:23.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.241 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.241 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:23.241 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.231367+0000", 2024-09-14T06:16:23.241 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1592000000000001e-05 2024-09-14T06:16:23.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.241 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.241 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:23.242 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.231471+0000", 2024-09-14T06:16:23.242 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010465499999999999 2024-09-14T06:16:23.242 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.242 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:23.242 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.242 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:25 3.e e26/24 ondisk, result = 0)", 2024-09-14T06:16:23.242 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:22.231165+0000", 2024-09-14T06:16:23.242 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.95240873599999998, 2024-09-14T06:16:23.242 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031610299999999998, 2024-09-14T06:16:23.242 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:23.242 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:23.242 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:23.243 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:23.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:23.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.231165+0000", 2024-09-14T06:16:23.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:23.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.231165+0000", 2024-09-14T06:16:23.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:23.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:23.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.231166+0000", 2024-09-14T06:16:23.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2529999999999999e-06 2024-09-14T06:16:23.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.244 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.244 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:23.244 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.231172+0000", 2024-09-14T06:16:23.244 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4949999999999999e-06 2024-09-14T06:16:23.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.244 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.244 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:23.244 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.231187+0000", 2024-09-14T06:16:23.244 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5327000000000001e-05 2024-09-14T06:16:23.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.244 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.244 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:23.244 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.231195+0000", 2024-09-14T06:16:23.245 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7640000000000005e-06 2024-09-14T06:16:23.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.245 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.245 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:23.245 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.231262+0000", 2024-09-14T06:16:23.245 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7492999999999999e-05 2024-09-14T06:16:23.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.245 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.245 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:23.245 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.231277+0000", 2024-09-14T06:16:23.245 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4421e-05 2024-09-14T06:16:23.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:23.245 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:23.245 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:23.245 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.231481+0000", 2024-09-14T06:16:23.246 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020435000000000001 2024-09-14T06:16:23.246 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.246 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:23.246 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.246 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:23.246 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:23.246 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:23.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:16:23.248 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-09-14T06:16:23.249 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:23.244+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:23.348 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:23.344+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:23.413 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:23.425 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:23.425 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:23.425 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:23.426 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:16:23.450 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:23.448+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:23.551 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:23.548+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:23.571 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:16:23.590 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:23.590 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:23.590 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:23.590 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:23.590 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:23.604 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:16:23.652 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:23.648+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:23.707 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:23.707 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:23.724 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:23.724 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:23.740 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:23.740 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:23.752 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:23.752+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:23.757 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:23.757 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:23.761 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:23.761 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:23.774 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:16:23.778 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:23.778 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:23.798 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:23.820 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:23.820 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:23.841 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:23.841 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:23.854 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:23.852+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:23.889 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:23.889 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:23.901 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:16:23.954 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:23.952+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:24.055 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:24.052+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:24.060 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:24.060 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:24.076 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:16:24.135 INFO:teuthology.orchestra.run.smithi039.stdout:skipping zero size read 2024-09-14T06:16:24.135 INFO:teuthology.orchestra.run.smithi039.stdout:skipping zero size read 2024-09-14T06:16:24.135 INFO:teuthology.orchestra.run.smithi039.stdout:truncating to largest ever: 0x5703771 2024-09-14T06:16:24.141 INFO:teuthology.orchestra.run.smithi039.stdout:7 trunc from 0x0 to 0x5703771 2024-09-14T06:16:24.155 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:24.152+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:24.172 INFO:teuthology.orchestra.run.smithi039.stdout:8 write 0xdc9e2e9 thru 0xdcae28a (0xffa2 bytes) 2024-09-14T06:16:24.180 INFO:teuthology.orchestra.run.smithi039.stdout:9 punch from 0x1f3674b to 0x1f3e260, (0x7b15 bytes) 2024-09-14T06:16:24.180 INFO:teuthology.orchestra.run.smithi039.stdout:10 punch from 0x8d0af54 to 0x8d185c5, (0xd671 bytes) 2024-09-14T06:16:24.180 INFO:teuthology.orchestra.run.smithi039.stdout:12 punch from 0x3fdbe03 to 0x3fdf5ac, (0x37a9 bytes) 2024-09-14T06:16:24.180 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:24.180 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:24.180 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:24.193 INFO:teuthology.orchestra.run.smithi039.stdout:18 trunc from 0xdcae28b to 0x34051cd 2024-09-14T06:16:24.215 INFO:teuthology.orchestra.run.smithi039.stdout:19 punch from 0x1ed70a to 0x1f7722, (0xa018 bytes) 2024-09-14T06:16:24.215 INFO:teuthology.orchestra.run.smithi039.stdout:20 read 0x508df9 thru 0x50f368 (0x6570 bytes) 2024-09-14T06:16:24.215 INFO:teuthology.orchestra.run.smithi039.stdout:21 write 0xff6cd thru 0x101fad (0x28e1 bytes) 2024-09-14T06:16:24.221 INFO:teuthology.orchestra.run.smithi039.stdout:23 read 0x2bad3bd thru 0x2bbc9cb (0xf60f bytes) 2024-09-14T06:16:24.222 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:24.222 INFO:teuthology.orchestra.run.smithi039.stdout:29 punch from 0x17367bc to 0x17458eb, (0xf12f bytes) 2024-09-14T06:16:24.224 INFO:teuthology.orchestra.run.smithi039.stdout:30 trunc from 0x34051cd to 0x1cc5f2b 2024-09-14T06:16:24.234 INFO:teuthology.orchestra.run.smithi039.stdout:31 write 0x5c2601 thru 0x5c2f96 (0x996 bytes) 2024-09-14T06:16:24.242 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:24.242 INFO:teuthology.orchestra.run.smithi039.stdout:truncating to largest ever: 0x69ff4bb 2024-09-14T06:16:24.250 INFO:teuthology.orchestra.run.smithi039.stdout:33 trunc from 0x1cc5f2b to 0x69ff4bb 2024-09-14T06:16:24.256 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:24.252+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:24.260 INFO:teuthology.orchestra.run.smithi039.stdout:34 read 0x4aea1ca thru 0x4af8fa2 (0xedd9 bytes) 2024-09-14T06:16:24.260 INFO:teuthology.orchestra.run.smithi039.stdout:35 read 0x158b5d thru 0x16579a (0xcc3e bytes) 2024-09-14T06:16:24.261 INFO:teuthology.orchestra.run.smithi039.stdout:36 read 0x184d704 thru 0x1858cde (0xb5db bytes) 2024-09-14T06:16:24.261 INFO:teuthology.orchestra.run.smithi039.stdout:37 punch from 0xc5409c to 0xc60897, (0xc7fb bytes) 2024-09-14T06:16:24.261 INFO:teuthology.orchestra.run.smithi039.stdout:38 punch from 0x15a4c3b to 0x15a7d0c, (0x30d1 bytes) 2024-09-14T06:16:24.261 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:24.262 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:24.262 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:24.280 INFO:teuthology.orchestra.run.smithi039.stdout:42 write 0xba0e86a thru 0xba1d28d (0xea24 bytes) 2024-09-14T06:16:24.282 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:16:24.287 INFO:teuthology.orchestra.run.smithi039.stdout:43 write 0x1f8dd3f thru 0x1f9c3dc (0xe69e bytes) 2024-09-14T06:16:24.295 INFO:teuthology.orchestra.run.smithi039.stdout:44 writesame 0xcff1f5 thru 0xd06b41 data_size 0xb(0x794d bytes) 2024-09-14T06:16:24.303 INFO:teuthology.orchestra.run.smithi039.stdout:46 punch from 0x7b0c5d9 to 0x7b0e362, (0x1d89 bytes) 2024-09-14T06:16:24.304 INFO:teuthology.orchestra.run.smithi039.stdout:truncating to largest ever: 0xa7b8619 2024-09-14T06:16:24.305 INFO:teuthology.orchestra.run.smithi039.stdout:47 trunc from 0xba1d28e to 0xa7b8619 2024-09-14T06:16:24.325 INFO:teuthology.orchestra.run.smithi039.stdout:truncating to largest ever: 0xed19f00 2024-09-14T06:16:24.331 INFO:teuthology.orchestra.run.smithi039.stdout:48 trunc from 0xa7b8619 to 0xed19f00 2024-09-14T06:16:24.341 INFO:teuthology.orchestra.run.smithi039.stdout:53 write 0x6ed8fd thru 0x6f5b63 (0x8267 bytes) 2024-09-14T06:16:24.344 INFO:teuthology.orchestra.run.smithi039.stdout:54 write 0x317fae5 thru 0x318be4a (0xc366 bytes) 2024-09-14T06:16:24.351 INFO:teuthology.orchestra.run.smithi039.stdout:55 read 0x502cdad thru 0x502e723 (0x1977 bytes) 2024-09-14T06:16:24.351 INFO:teuthology.orchestra.run.smithi039.stdout:56 writesame 0xe865afb thru 0xe875a03 data_size 0x7(0xff09 bytes) 2024-09-14T06:16:24.357 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:24.352+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:24.370 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:24.370 INFO:teuthology.orchestra.run.smithi039.stdout:59 write 0x130d9c7 thru 0x131a42c (0xca66 bytes) 2024-09-14T06:16:24.378 INFO:teuthology.orchestra.run.smithi039.stdout:60 punch from 0xa99ab88 to 0xa99b968, (0xde0 bytes) 2024-09-14T06:16:24.378 INFO:teuthology.orchestra.run.smithi039.stdout:61 read 0xe8c758c thru 0xe8d2b90 (0xb605 bytes) 2024-09-14T06:16:24.379 INFO:teuthology.orchestra.run.smithi039.stdout:62 read 0x24876f thru 0x251e4e (0x96e0 bytes) 2024-09-14T06:16:24.379 INFO:teuthology.orchestra.run.smithi039.stdout:63 punch from 0x9a0228f to 0x9a0d5ad, (0xb31e bytes) 2024-09-14T06:16:24.381 INFO:teuthology.orchestra.run.smithi039.stdout:66 trunc from 0xed19f00 to 0xdccbe95 2024-09-14T06:16:24.416 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:24.416 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:24.425 INFO:teuthology.orchestra.run.smithi039.stdout:67 trunc from 0xdccbe95 to 0x395d514 2024-09-14T06:16:24.433 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:16:24.436 INFO:teuthology.orchestra.run.smithi039.stdout:68 read 0x26cc2b8 thru 0x26d6186 (0x9ecf bytes) 2024-09-14T06:16:24.436 INFO:teuthology.orchestra.run.smithi039.stdout:69 write 0x17b2a5e thru 0x17c2656 (0xfbf9 bytes) 2024-09-14T06:16:24.443 INFO:teuthology.orchestra.run.smithi039.stdout:70 read 0x2b9f91b thru 0x2ba255c (0x2c42 bytes) 2024-09-14T06:16:24.457 INFO:teuthology.orchestra.run.smithi039.stdout:72 writesame 0x72f22ce thru 0x72f9026 data_size 0x1f(0x6d59 bytes) 2024-09-14T06:16:24.458 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:24.452+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:24.474 INFO:teuthology.orchestra.run.smithi039.stdout:73 write 0xea01922 thru 0xea0bdc5 (0xa4a4 bytes) 2024-09-14T06:16:24.493 INFO:teuthology.orchestra.run.smithi039.stdout:75 trunc from 0xea0bdc6 to 0x8f10a14 2024-09-14T06:16:24.513 INFO:teuthology.orchestra.run.smithi039.stdout:76 writesame 0xfd275 thru 0xffaa2 data_size 0x25(0x282e bytes) 2024-09-14T06:16:24.516 INFO:teuthology.orchestra.run.smithi039.stdout:78 writesame 0x8052bdc thru 0x8058130 data_size 0x11(0x5555 bytes) 2024-09-14T06:16:24.539 INFO:teuthology.orchestra.run.smithi039.stdout:79 writesame 0xe0b30c3 thru 0xe0c0160 data_size 0x8a(0xd09e bytes) 2024-09-14T06:16:24.540 INFO:teuthology.orchestra.run.smithi039.stdout:80 read 0x37ab88b thru 0x37b89ed (0xd163 bytes) 2024-09-14T06:16:24.540 INFO:teuthology.orchestra.run.smithi039.stdout:81 write 0x1c48e3d thru 0x1c58695 (0xf859 bytes) 2024-09-14T06:16:24.544 INFO:teuthology.orchestra.run.smithi039.stdout:83 read 0x3bcedad thru 0x3bd4867 (0x5abb bytes) 2024-09-14T06:16:24.544 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:24.545 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:24.545 INFO:teuthology.orchestra.run.smithi039.stdout:87 read 0x2f21f7f thru 0x2f27eed (0x5f6f bytes) 2024-09-14T06:16:24.545 INFO:teuthology.orchestra.run.smithi039.stdout:88 read 0x78e0fce thru 0x78edefd (0xcf30 bytes) 2024-09-14T06:16:24.545 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:24.545 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:24.545 INFO:teuthology.orchestra.run.smithi039.stdout:92 write 0x977d9e thru 0x97b6e0 (0x3943 bytes) 2024-09-14T06:16:24.558 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:24.556+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:24.565 INFO:teuthology.orchestra.run.smithi039.stdout:93 trunc from 0xe0c0161 to 0x73c69e7 2024-09-14T06:16:24.570 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:24.571 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:24.584 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:16:24.589 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:24.589 INFO:teuthology.orchestra.run.smithi039.stdout:95 read 0x6ea2b01 thru 0x6ea533e (0x283e bytes) 2024-09-14T06:16:24.606 INFO:teuthology.orchestra.run.smithi039.stdout:96 write 0xd20d464 thru 0xd21b521 (0xe0be bytes) 2024-09-14T06:16:24.613 INFO:teuthology.orchestra.run.smithi039.stdout:98 punch from 0x45c51fd to 0x45d161e, (0xc421 bytes) 2024-09-14T06:16:24.613 INFO:teuthology.orchestra.run.smithi039.stdout:100 write 0x6b68d1 thru 0x6bb054 (0x4784 bytes) 2024-09-14T06:16:24.616 INFO:teuthology.orchestra.run.smithi039.stdout:103 write 0x862f3ca thru 0x863e012 (0xec49 bytes) 2024-09-14T06:16:24.621 INFO:teuthology.orchestra.run.smithi039.stdout:104 read 0x6a80da4 thru 0x6a8c110 (0xb36d bytes) 2024-09-14T06:16:24.621 INFO:teuthology.orchestra.run.smithi039.stdout:106 punch from 0x6d69d93 to 0x6d71b0a, (0x7d77 bytes) 2024-09-14T06:16:24.621 INFO:teuthology.orchestra.run.smithi039.stdout:107 punch from 0xb1029ad to 0xb102b0a, (0x15d bytes) 2024-09-14T06:16:24.622 INFO:teuthology.orchestra.run.smithi039.stdout:108 trunc from 0xd21b522 to 0xde866e8 2024-09-14T06:16:24.630 INFO:teuthology.orchestra.run.smithi039.stdout:109 punch from 0x30a988 to 0x312e7b, (0x84f3 bytes) 2024-09-14T06:16:24.642 INFO:teuthology.orchestra.run.smithi039.stdout:110 trunc from 0xde866e8 to 0x554c0ea 2024-09-14T06:16:24.659 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:24.656+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:24.661 INFO:teuthology.orchestra.run.smithi039.stdout:111 read 0x5381777 thru 0x5382082 (0x90c bytes) 2024-09-14T06:16:24.661 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:24.661 INFO:teuthology.orchestra.run.smithi039.stdout:113 read 0x4bd7de7 thru 0x4be653a (0xe754 bytes) 2024-09-14T06:16:24.685 INFO:teuthology.orchestra.run.smithi039.stdout:114 writesame 0xded0bc2 thru 0xded1f4f data_size 0x1(0x138e bytes) 2024-09-14T06:16:24.688 INFO:teuthology.orchestra.run.smithi039.stdout:115 trunc from 0xded1f50 to 0x91729d4 2024-09-14T06:16:24.698 INFO:teuthology.orchestra.run.smithi039.stdout:116 write 0x422485c thru 0x422d6b0 (0x8e55 bytes) 2024-09-14T06:16:24.718 INFO:teuthology.orchestra.run.smithi039.stdout:117 writesame 0xa928f53 thru 0xa931ae9 data_size 0x7(0x8b97 bytes) 2024-09-14T06:16:24.718 INFO:teuthology.orchestra.run.smithi039.stdout:118 punch from 0x493e2d to 0x497aad, (0x3c80 bytes) 2024-09-14T06:16:24.725 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:24.725 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:24.725 INFO:teuthology.orchestra.run.smithi039.stdout:119 trunc from 0xa931aea to 0x5c1a20d 2024-09-14T06:16:24.735 INFO:teuthology.orchestra.run.smithi039.stdout:120 write 0x31a413f thru 0x31ac3cb (0x828d bytes) 2024-09-14T06:16:24.740 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:16:24.750 INFO:teuthology.orchestra.run.smithi039.stdout:121 write 0x658ae1b thru 0x658e374 (0x355a bytes) 2024-09-14T06:16:24.758 INFO:teuthology.orchestra.run.smithi039.stdout:124 trunc from 0x658e375 to 0x5320569 2024-09-14T06:16:24.760 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:24.756+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:24.792 INFO:teuthology.orchestra.run.smithi039.stdout:125 trunc from 0x5320569 to 0xc7d484c 2024-09-14T06:16:24.801 INFO:teuthology.orchestra.run.smithi039.stdout:126 write 0x8fa6483 thru 0x8fb1e64 (0xb9e2 bytes) 2024-09-14T06:16:24.808 INFO:teuthology.orchestra.run.smithi039.stdout:127 read 0xa2f396e thru 0xa300534 (0xcbc7 bytes) 2024-09-14T06:16:24.811 INFO:teuthology.orchestra.run.smithi039.stdout:128 trunc from 0xc7d484c to 0xdf49427 2024-09-14T06:16:24.821 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:24.821 INFO:teuthology.orchestra.run.smithi039.stdout:130 writesame 0xd6ed1df thru 0xd6eecc0 data_size 0x3e(0x1ae2 bytes) 2024-09-14T06:16:24.827 INFO:teuthology.orchestra.run.smithi039.stdout:131 write 0xd2ca0f2 thru 0xd2d8273 (0xe182 bytes) 2024-09-14T06:16:24.842 INFO:teuthology.orchestra.run.smithi039.stdout:132 trunc from 0xdf49427 to 0x9ad10fe 2024-09-14T06:16:24.860 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:24.856+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:24.861 INFO:teuthology.orchestra.run.smithi039.stdout:134 read 0x911b084 thru 0x9124bd8 (0x9b55 bytes) 2024-09-14T06:16:24.861 INFO:teuthology.orchestra.run.smithi039.stdout:136 read 0x4a2f563 thru 0x4a34703 (0x51a1 bytes) 2024-09-14T06:16:24.866 INFO:teuthology.orchestra.run.smithi039.stdout:137 trunc from 0x9ad10fe to 0xbb6f1db 2024-09-14T06:16:24.876 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:24.876 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:24.877 INFO:teuthology.orchestra.run.smithi039.stdout:138 write 0x36480cc thru 0x36524d6 (0xa40b bytes) 2024-09-14T06:16:24.891 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:16:24.895 INFO:teuthology.orchestra.run.smithi039.stdout:139 write 0xbb88ff3 thru 0xbb93600 (0xa60e bytes) 2024-09-14T06:16:24.901 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:24.901 INFO:teuthology.orchestra.run.smithi039.stdout:141 read 0xb9ab2d5 thru 0xb9ba41d (0xf149 bytes) 2024-09-14T06:16:24.917 INFO:teuthology.orchestra.run.smithi039.stdout:144 write 0xd105e37 thru 0xd108556 (0x2720 bytes) 2024-09-14T06:16:24.922 INFO:teuthology.orchestra.run.smithi039.stdout:145 writesame 0x2b8e7c4 thru 0x2b9da40 data_size 0x1(0xf27d bytes) 2024-09-14T06:16:24.971 INFO:teuthology.orchestra.run.smithi039.stdout:146 write 0x8cd2b18 thru 0x8cde8b9 (0xbda2 bytes) 2024-09-14T06:16:24.974 INFO:teuthology.orchestra.run.smithi039.stdout:147 write 0x1a64e7a thru 0x1a69a59 (0x4be0 bytes) 2024-09-14T06:16:24.977 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:24.976+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:24.978 INFO:teuthology.orchestra.run.smithi039.stdout:148 read 0x7f3e8b9 thru 0x7f43571 (0x4cb9 bytes) 2024-09-14T06:16:24.978 INFO:teuthology.orchestra.run.smithi039.stdout:150 punch from 0x663f0b5 to 0x664ecc0, (0xfc0b bytes) 2024-09-14T06:16:24.978 INFO:teuthology.orchestra.run.smithi039.stdout:151 write 0x183b137 thru 0x1844427 (0x92f1 bytes) 2024-09-14T06:16:24.981 INFO:teuthology.orchestra.run.smithi039.stdout:152 read 0x3df892f thru 0x3dfcfb4 (0x4686 bytes) 2024-09-14T06:16:24.981 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:24.981 INFO:teuthology.orchestra.run.smithi039.stdout:154 read 0xb6df9bd thru 0xb6e7f92 (0x85d6 bytes) 2024-09-14T06:16:24.981 INFO:teuthology.orchestra.run.smithi039.stdout:156 read 0x5f8a58 thru 0x6089e5 (0xff8e bytes) 2024-09-14T06:16:24.982 INFO:teuthology.orchestra.run.smithi039.stdout:157 write 0x442eea4 thru 0x443c4f4 (0xd651 bytes) 2024-09-14T06:16:24.987 INFO:teuthology.orchestra.run.smithi039.stdout:158 read 0x6611504 thru 0x6619d47 (0x8844 bytes) 2024-09-14T06:16:24.988 INFO:teuthology.orchestra.run.smithi039.stdout:160 read 0x4ac3d4f thru 0x4ac8f28 (0x51da bytes) 2024-09-14T06:16:24.988 INFO:teuthology.orchestra.run.smithi039.stdout:161 read 0x6c67a83 thru 0x6c6ba80 (0x3ffe bytes) 2024-09-14T06:16:24.988 INFO:teuthology.orchestra.run.smithi039.stdout:162 write 0x2c59181 thru 0x2c65d6c (0xcbec bytes) 2024-09-14T06:16:24.994 INFO:teuthology.orchestra.run.smithi039.stdout:165 writesame 0x3693b9b thru 0x369d977 data_size 0x13(0x9ddd bytes) 2024-09-14T06:16:24.997 INFO:teuthology.orchestra.run.smithi039.stdout:166 write 0xd42082 thru 0xd47363 (0x52e2 bytes) 2024-09-14T06:16:25.000 INFO:teuthology.orchestra.run.smithi039.stdout:168 read 0x55875cf thru 0x558f044 (0x7a76 bytes) 2024-09-14T06:16:25.001 INFO:teuthology.orchestra.run.smithi039.stdout:170 read 0x6afa40b thru 0x6b08ad9 (0xe6cf bytes) 2024-09-14T06:16:25.001 INFO:teuthology.orchestra.run.smithi039.stdout:171 write 0x2a8561c thru 0x2a8f525 (0x9f0a bytes) 2024-09-14T06:16:25.003 INFO:teuthology.orchestra.run.smithi039.stdout:172 clone 1 order 21 su 16384 sc 12 2024-09-14T06:16:25.060 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:25.061 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:25.062 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:25.060+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:25.074 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:16:25.163 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:25.160+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:25.198 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:16:25.198 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:16:25.198 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-14T06:16:25.199 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-14T06:16:25.240 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:25.240 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:25.249 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:16:25.255 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:16:25.301 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:25.300+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:25.364 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:25.360+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:25.373 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:25.373 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:25.387 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:16:25.464 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:25.460+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:25.467 INFO:teuthology.orchestra.run.smithi039.stdout:174 write 0xdad9468 thru 0xdade9cf (0x5568 bytes) 2024-09-14T06:16:25.474 INFO:teuthology.orchestra.run.smithi039.stdout:175 punch from 0x9453dd to 0x950a0e, (0xb631 bytes) 2024-09-14T06:16:25.474 INFO:teuthology.orchestra.run.smithi039.stdout:176 writesame 0x5fc1579 thru 0x5fcd86c data_size 0x6(0xc2f4 bytes) 2024-09-14T06:16:25.484 INFO:teuthology.orchestra.run.smithi039.stdout:177 punch from 0x33bb320 to 0x33c2133, (0x6e13 bytes) 2024-09-14T06:16:25.484 INFO:teuthology.orchestra.run.smithi039.stdout:178 write 0xe6fa94 thru 0xe7307f (0x35ec bytes) 2024-09-14T06:16:25.515 INFO:teuthology.orchestra.run.smithi039.stdout:179 trunc from 0xdade9d0 to 0x9a68de6 2024-09-14T06:16:25.547 INFO:teuthology.orchestra.run.smithi039.stdout:180 writesame 0x1890d99 thru 0x18968df data_size 0x1(0x5b47 bytes) 2024-09-14T06:16:25.559 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:25.559 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:25.565 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:25.560+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:25.574 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:16:25.585 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-14T06:16:25.580+0000 7f9bd5666640 -1 mon.a@0(leader).osd e28 definitely_dead 0 2024-09-14T06:16:25.595 INFO:teuthology.orchestra.run.smithi039.stdout:181 writesame 0x9c8e283 thru 0x9c8f030 data_size 0x2(0xdae bytes) 2024-09-14T06:16:25.597 INFO:teuthology.orchestra.run.smithi039.stdout:182 read 0x6b46982 thru 0x6b4d1b6 (0x6835 bytes) 2024-09-14T06:16:25.598 INFO:teuthology.orchestra.run.smithi039.stdout:183 writesame 0x30e67e0 thru 0x30f1193 data_size 0x2(0xa9b4 bytes) 2024-09-14T06:16:25.634 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:25.634 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:25.640 INFO:teuthology.orchestra.run.smithi039.stdout:187 trunc from 0x9c8f031 to 0x5d4115b 2024-09-14T06:16:25.667 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:25.664+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:25.679 INFO:teuthology.orchestra.run.smithi039.stdout:191 writesame 0x4a7594d thru 0x4a7a9c9 data_size 0xd(0x507d bytes) 2024-09-14T06:16:25.681 INFO:teuthology.orchestra.run.smithi039.stdout:192 read 0x3663947 thru 0x36712d3 (0xd98d bytes) 2024-09-14T06:16:25.689 INFO:teuthology.orchestra.run.smithi039.stdout:194 punch from 0x5cf24f4 to 0x5cfc819, (0xa325 bytes) 2024-09-14T06:16:25.689 INFO:teuthology.orchestra.run.smithi039.stdout:195 punch from 0x3c93b92 to 0x3ca317f, (0xf5ed bytes) 2024-09-14T06:16:25.716 INFO:teuthology.orchestra.run.smithi039.stdout:196 writesame 0x9141949 thru 0x914dd98 data_size 0x30(0xc450 bytes) 2024-09-14T06:16:25.730 INFO:teuthology.orchestra.run.smithi039.stdout:197 write 0xc5633c2 thru 0xc56b5ee (0x822d bytes) 2024-09-14T06:16:25.737 INFO:teuthology.orchestra.run.smithi039.stdout:198 punch from 0x3291cc6 to 0x329c691, (0xa9cb bytes) 2024-09-14T06:16:25.737 INFO:teuthology.orchestra.run.smithi039.stdout:200 read 0x5884870 thru 0x5884fc9 (0x75a bytes) 2024-09-14T06:16:25.743 INFO:teuthology.orchestra.run.smithi039.stdout:201 trunc from 0xc56b5ef to 0x964221f 2024-09-14T06:16:25.749 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:25.749 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:25.758 INFO:teuthology.orchestra.run.smithi039.stdout:202 write 0x8b75f12 thru 0x8b781db (0x22ca bytes) 2024-09-14T06:16:25.762 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:16:25.764 INFO:teuthology.orchestra.run.smithi039.stdout:203 write 0x6c97786 thru 0x6c9f0f5 (0x7970 bytes) 2024-09-14T06:16:25.767 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:25.764+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:25.773 INFO:teuthology.orchestra.run.smithi039.stdout:204 trunc from 0x964221f to 0x9d5f76f 2024-09-14T06:16:25.784 INFO:teuthology.orchestra.run.smithi039.stdout:205 read 0x7f30e3c thru 0x7f355f6 (0x47bb bytes) 2024-09-14T06:16:25.784 INFO:teuthology.orchestra.run.smithi039.stdout:207 punch from 0x8363083 to 0x836a5e8, (0x7565 bytes) 2024-09-14T06:16:25.784 INFO:teuthology.orchestra.run.smithi039.stdout:208 read 0x38c4991 thru 0x38cf882 (0xaef2 bytes) 2024-09-14T06:16:25.791 INFO:teuthology.orchestra.run.smithi039.stdout:210 trunc from 0x9d5f76f to 0x6144901 2024-09-14T06:16:25.812 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:25.812 INFO:teuthology.orchestra.run.smithi039.stdout:213 read 0x5071dc5 thru 0x507569c (0x38d8 bytes) 2024-09-14T06:16:25.813 INFO:teuthology.orchestra.run.smithi039.stdout:214 write 0x59e2281 thru 0x59ee0e5 (0xbe65 bytes) 2024-09-14T06:16:25.817 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:25.817 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:25.817 INFO:teuthology.orchestra.run.smithi039.stdout:217 punch from 0x5ab2eb6 to 0x5abfb03, (0xcc4d bytes) 2024-09-14T06:16:25.826 INFO:teuthology.orchestra.run.smithi039.stdout:218 trunc from 0x6144901 to 0x23d9480 2024-09-14T06:16:25.858 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:16:25.868 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:25.864+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:25.909 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:25.909 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:25.927 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:16:25.969 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:25.964+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:25.970 INFO:teuthology.orchestra.run.smithi039.stdout:219 write 0xae6702 thru 0xaf5017 (0xe916 bytes) 2024-09-14T06:16:26.032 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:26.052 INFO:teuthology.orchestra.run.smithi039.stdout:222 trunc from 0x23d9480 to 0xdd7314b 2024-09-14T06:16:26.057 INFO:teuthology.orchestra.run.smithi039.stdout:223 read 0x553eba thru 0x554606 (0x74d bytes) 2024-09-14T06:16:26.063 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:26.063 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:26.070 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:26.068+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:26.072 INFO:teuthology.orchestra.run.smithi039.stdout:225 trunc from 0xdd7314b to 0x570918d 2024-09-14T06:16:26.081 INFO:teuthology.orchestra.run.smithi039.stdout:226 trunc from 0x570918d to 0x39e8c9a 2024-09-14T06:16:26.087 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:26.087 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:26.088 INFO:teuthology.orchestra.run.smithi039.stdout:227 read 0x1de754 thru 0x1e3938 (0x51e5 bytes) 2024-09-14T06:16:26.089 INFO:teuthology.orchestra.run.smithi039.stdout:229 punch from 0x1e61999 to 0x1e65ad2, (0x4139 bytes) 2024-09-14T06:16:26.090 INFO:teuthology.orchestra.run.smithi039.stdout:230 read 0x31c8946 thru 0x31d0a71 (0x812c bytes) 2024-09-14T06:16:26.091 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:26.091 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:26.091 INFO:teuthology.orchestra.run.smithi039.stdout:233 trunc from 0x39e8c9a to 0x4dbb4d1 2024-09-14T06:16:26.105 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:16:26.107 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:26.107 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:26.112 INFO:teuthology.orchestra.run.smithi039.stdout:234 write 0xba0620a thru 0xba105e0 (0xa3d7 bytes) 2024-09-14T06:16:26.127 INFO:teuthology.orchestra.run.smithi039.stdout:235 write 0xd57ace1 thru 0xd587146 (0xc466 bytes) 2024-09-14T06:16:26.128 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:26.128 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:26.136 INFO:teuthology.orchestra.run.smithi039.stdout:236 read 0xbaa15de thru 0xbaafc54 (0xe677 bytes) 2024-09-14T06:16:26.136 INFO:teuthology.orchestra.run.smithi039.stdout:238 writesame 0xcd228b6 thru 0xcd265f8 data_size 0x1(0x3d43 bytes) 2024-09-14T06:16:26.144 INFO:teuthology.orchestra.run.smithi039.stdout:239 read 0x2dc21b3 thru 0x2dc43f4 (0x2242 bytes) 2024-09-14T06:16:26.145 INFO:teuthology.orchestra.run.smithi039.stdout:240 write 0xcc71044 thru 0xcc74ee1 (0x3e9e bytes) 2024-09-14T06:16:26.151 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:26.152 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:26.157 INFO:teuthology.orchestra.run.smithi039.stdout:241 trunc from 0xd587147 to 0x7f920cf 2024-09-14T06:16:26.172 INFO:teuthology.orchestra.run.smithi039.stdout:242 write 0x7b03e17 thru 0x7b0a921 (0x6b0b bytes) 2024-09-14T06:16:26.174 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:26.174 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:26.178 INFO:teuthology.orchestra.run.smithi039.stdout:243 punch from 0x6eab3d3 to 0x6ebb31d, (0xff4a bytes) 2024-09-14T06:16:26.178 INFO:teuthology.orchestra.run.smithi039.stdout:245 punch from 0x35b6c25 to 0x35c536d, (0xe748 bytes) 2024-09-14T06:16:26.178 INFO:teuthology.orchestra.run.smithi039.stdout:246 read 0x84755a thru 0x848c38 (0x16df bytes) 2024-09-14T06:16:26.178 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:26.178 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:26.178 INFO:teuthology.orchestra.run.smithi039.stdout:250 read 0x4ad86e5 thru 0x4ae5ac5 (0xd3e1 bytes) 2024-09-14T06:16:26.179 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:26.179 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:26.187 INFO:teuthology.orchestra.run.smithi039.stdout:254 write 0x9f0883f thru 0x9f17c4a (0xf40c bytes) 2024-09-14T06:16:26.193 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:26.192+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:26.198 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:26.198 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:26.200 INFO:teuthology.orchestra.run.smithi039.stdout:255 write 0xae54fd2 thru 0xae5c86c (0x789b bytes) 2024-09-14T06:16:26.216 INFO:teuthology.orchestra.run.smithi039.stdout:256 write 0xdceb777 thru 0xdcf14c1 (0x5d4b bytes) 2024-09-14T06:16:26.222 INFO:teuthology.orchestra.run.smithi039.stderr:marked in osd.2. 2024-09-14T06:16:26.223 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:26.223 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:26.225 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-09-14T06:16:26.261 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-14T06:16:26.272 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:26.268+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:26.276 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:26.276 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:16:26.276 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:16:26.276 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:16:26.276 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.276 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4214.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T06:16:26.276 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:11.166477+0000", 2024-09-14T06:16:26.276 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.111935458, 2024-09-14T06:16:26.276 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0021449770000000002, 2024-09-14T06:16:26.276 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:26.276 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:26.276 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:26.277 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:26.277 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.277 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:26.277 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.166477+0000", 2024-09-14T06:16:26.277 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:26.277 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.277 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.277 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:26.277 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.166477+0000", 2024-09-14T06:16:26.277 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:26.277 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.277 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.277 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:26.277 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.166477+0000", 2024-09-14T06:16:26.278 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4e-07 2024-09-14T06:16:26.278 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.278 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.278 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:26.278 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.166488+0000", 2024-09-14T06:16:26.278 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0823e-05 2024-09-14T06:16:26.278 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.278 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.278 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:26.278 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.166533+0000", 2024-09-14T06:16:26.278 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4699e-05 2024-09-14T06:16:26.278 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.278 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.278 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:26.279 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.166576+0000", 2024-09-14T06:16:26.279 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3319999999999999e-05 2024-09-14T06:16:26.279 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.279 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.279 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:26.279 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.166735+0000", 2024-09-14T06:16:26.279 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000158386 2024-09-14T06:16:26.279 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.279 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.279 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:26.279 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.166790+0000", 2024-09-14T06:16:26.279 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.5612999999999998e-05 2024-09-14T06:16:26.279 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.280 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.280 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:26.280 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.168535+0000", 2024-09-14T06:16:26.280 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0017443700000000001 2024-09-14T06:16:26.280 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.280 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.280 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:26.280 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:11.168622+0000", 2024-09-14T06:16:26.280 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6926000000000001e-05 2024-09-14T06:16:26.280 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:26.280 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:26.280 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:26.280 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.280 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.281 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4214.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-14T06:16:26.281 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:12.181799+0000", 2024-09-14T06:16:26.281 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 14.096613284, 2024-09-14T06:16:26.281 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001894539, 2024-09-14T06:16:26.281 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:26.281 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:26.281 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:26.281 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:26.281 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.281 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:26.281 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.181799+0000", 2024-09-14T06:16:26.281 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:26.281 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.282 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.282 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:26.282 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.181799+0000", 2024-09-14T06:16:26.282 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:26.282 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.282 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.282 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:26.282 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.181800+0000", 2024-09-14T06:16:26.282 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1999999999999998e-07 2024-09-14T06:16:26.282 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.282 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.282 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:26.282 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.181817+0000", 2024-09-14T06:16:26.283 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7255999999999999e-05 2024-09-14T06:16:26.283 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.283 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.283 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:26.283 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.181856+0000", 2024-09-14T06:16:26.283 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.943e-05 2024-09-14T06:16:26.283 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.283 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.283 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:26.283 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.181885+0000", 2024-09-14T06:16:26.283 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.845e-05 2024-09-14T06:16:26.283 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.283 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.284 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:26.284 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.182010+0000", 2024-09-14T06:16:26.284 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000125623 2024-09-14T06:16:26.284 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.284 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.284 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:26.284 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.182062+0000", 2024-09-14T06:16:26.284 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1702000000000001e-05 2024-09-14T06:16:26.284 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.284 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.284 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:26.284 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.183584+0000", 2024-09-14T06:16:26.284 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001522135 2024-09-14T06:16:26.284 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.285 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.285 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:26.285 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:12.183693+0000", 2024-09-14T06:16:26.285 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000109023 2024-09-14T06:16:26.285 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:26.285 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:26.285 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:26.285 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:26.285 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:26.285 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:26.290 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:16:26.373 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:26.372+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:26.457 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:26.458 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:26.458 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:16:26.458 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:26.472 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:16:26.473 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:26.472+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:26.575 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:26.572+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:26.593 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:26.593 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:26.593 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:16:26.594 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:16:26.594 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:26.607 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:16:26.675 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:26.672+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:26.729 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:26.730 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:16:26.730 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:16:26.730 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:16:26.730 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.730 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:13 3.6 e26/23 3:676c166b:::rbd_object_map.11528c180b7:head v 26'1, mlcod=0'0)", 2024-09-14T06:16:26.730 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:22.215716+0000", 2024-09-14T06:16:26.730 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4.5161258130000004, 2024-09-14T06:16:26.730 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00078508199999999999, 2024-09-14T06:16:26.730 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:26.730 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:26.730 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:26.730 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:26.731 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.731 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:26.731 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.215716+0000", 2024-09-14T06:16:26.731 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:26.731 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.731 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.731 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:26.731 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.215716+0000", 2024-09-14T06:16:26.731 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:26.731 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.731 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.731 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:26.731 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.215717+0000", 2024-09-14T06:16:26.731 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.68e-07 2024-09-14T06:16:26.732 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.732 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.732 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:26.732 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.215723+0000", 2024-09-14T06:16:26.732 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:16:26.732 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.732 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.732 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:26.732 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.215736+0000", 2024-09-14T06:16:26.732 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2480000000000001e-05 2024-09-14T06:16:26.732 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.732 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.732 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:26.733 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.215745+0000", 2024-09-14T06:16:26.733 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2329999999999995e-06 2024-09-14T06:16:26.733 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.733 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.733 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:26.733 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.215833+0000", 2024-09-14T06:16:26.733 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8448000000000004e-05 2024-09-14T06:16:26.733 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.733 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.733 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:26.733 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.215862+0000", 2024-09-14T06:16:26.733 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8456999999999999e-05 2024-09-14T06:16:26.733 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.734 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.734 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:26.734 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.216464+0000", 2024-09-14T06:16:26.734 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00060177400000000002 2024-09-14T06:16:26.734 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:26.734 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:26.734 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:26.734 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:22.216501+0000", 2024-09-14T06:16:26.734 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7588e-05 2024-09-14T06:16:26.734 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:26.734 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:26.734 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:26.734 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:26.734 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:26.735 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:26.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:16:26.791 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:26.788+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:26.877 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:26.872+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:26.880 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:26.880 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:26.880 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:26.880 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:26.894 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:16:26.978 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:26.976+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:27.066 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:27.066 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:27.066 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:27.066 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:27.067 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:27.079 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:27.076+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:27.081 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:16:27.179 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:27.180+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:27.259 INFO:teuthology.orchestra.run.smithi039.stdout:257 read 0x90e60b7 thru 0x90ec88e (0x67d8 bytes) 2024-09-14T06:16:27.259 INFO:teuthology.orchestra.run.smithi039.stdout:258 punch from 0xae02be1 to 0xae0e018, (0xb437 bytes) 2024-09-14T06:16:27.259 INFO:teuthology.orchestra.run.smithi039.stdout:259 write 0xf15754 thru 0xf196b1 (0x3f5e bytes) 2024-09-14T06:16:27.272 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:27.272 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:27.272 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:27.272 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:27.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.273 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:10 3.1c e26/23 3:3831950c:::rbd_directory:head v 26'1, mlcod=0'0)", 2024-09-14T06:16:27.273 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:22.204352+0000", 2024-09-14T06:16:27.273 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.0664960380000004, 2024-09-14T06:16:27.273 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0015404419999999999, 2024-09-14T06:16:27.273 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.273 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.273 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.273 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.273 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.273 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.204352+0000", 2024-09-14T06:16:27.273 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.274 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.274 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.204352+0000", 2024-09-14T06:16:27.274 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.274 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.274 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.204352+0000", 2024-09-14T06:16:27.274 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9100000000000003e-07 2024-09-14T06:16:27.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.274 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.274 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.204362+0000", 2024-09-14T06:16:27.274 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4280000000000002e-06 2024-09-14T06:16:27.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.275 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.204384+0000", 2024-09-14T06:16:27.275 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2532999999999999e-05 2024-09-14T06:16:27.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.275 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.204396+0000", 2024-09-14T06:16:27.275 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2013e-05 2024-09-14T06:16:27.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.275 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.204530+0000", 2024-09-14T06:16:27.276 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013320000000000001 2024-09-14T06:16:27.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.276 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.276 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.276 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.204585+0000", 2024-09-14T06:16:27.276 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5704000000000001e-05 2024-09-14T06:16:27.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.276 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.276 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.276 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.205801+0000", 2024-09-14T06:16:27.276 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001215317 2024-09-14T06:16:27.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.276 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.277 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.277 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.205892+0000", 2024-09-14T06:16:27.277 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1456e-05 2024-09-14T06:16:27.277 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.277 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.277 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.277 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.277 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:12 3.e 3:77456488:::rbd_header.11528c180b7:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T06:16:27.277 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:22.209238+0000", 2024-09-14T06:16:27.277 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.0616099190000003, 2024-09-14T06:16:27.277 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030347970000000001, 2024-09-14T06:16:27.277 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.277 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.278 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.278 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.278 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:27.278 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:27.278 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 12 2024-09-14T06:16:27.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.278 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.278 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.278 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.278 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.209238+0000", 2024-09-14T06:16:27.278 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.278 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.278 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.279 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.209238+0000", 2024-09-14T06:16:27.279 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.279 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.279 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.209241+0000", 2024-09-14T06:16:27.279 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7439999999999999e-06 2024-09-14T06:16:27.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.279 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.279 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.209260+0000", 2024-09-14T06:16:27.279 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9703000000000001e-05 2024-09-14T06:16:27.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.280 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.280 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.209292+0000", 2024-09-14T06:16:27.280 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1752999999999999e-05 2024-09-14T06:16:27.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.280 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.280 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.280 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.209308+0000", 2024-09-14T06:16:27.280 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5897e-05 2024-09-14T06:16:27.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.280 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.280 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.280 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.209437+0000", 2024-09-14T06:16:27.280 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012912899999999999 2024-09-14T06:16:27.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.281 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.281 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.281 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.209814+0000", 2024-09-14T06:16:27.281 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00037658 2024-09-14T06:16:27.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.281 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.281 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.281 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.210400+0000", 2024-09-14T06:16:27.281 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00058675899999999996 2024-09-14T06:16:27.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.281 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.281 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.281 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.211690+0000", 2024-09-14T06:16:27.282 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0012894829999999999 2024-09-14T06:16:27.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.282 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.282 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.282 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.212044+0000", 2024-09-14T06:16:27.282 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00035423699999999998 2024-09-14T06:16:27.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.282 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.282 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.282 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.212142+0000", 2024-09-14T06:16:27.282 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8152999999999994e-05 2024-09-14T06:16:27.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.282 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.282 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.283 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.212273+0000", 2024-09-14T06:16:27.283 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000130359 2024-09-14T06:16:27.283 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.283 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.283 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.283 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.283 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:25 3.e 3:77456488:::rbd_header.11528c180b7:head [watch watch cookie 140175317418864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T06:16:27.283 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:22.229873+0000", 2024-09-14T06:16:27.283 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.0409746699999998, 2024-09-14T06:16:27.283 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001598318, 2024-09-14T06:16:27.283 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.283 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.283 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.284 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.284 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:27.284 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:27.284 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 25 2024-09-14T06:16:27.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.284 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.284 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.284 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.284 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229873+0000", 2024-09-14T06:16:27.284 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.284 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.284 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.284 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229873+0000", 2024-09-14T06:16:27.285 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.285 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.285 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.285 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229874+0000", 2024-09-14T06:16:27.285 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1090000000000001e-06 2024-09-14T06:16:27.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.285 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.285 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.285 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229880+0000", 2024-09-14T06:16:27.285 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6400000000000002e-06 2024-09-14T06:16:27.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.285 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.286 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.286 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229895+0000", 2024-09-14T06:16:27.286 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5485000000000002e-05 2024-09-14T06:16:27.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.286 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.286 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229903+0000", 2024-09-14T06:16:27.286 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3599999999999998e-06 2024-09-14T06:16:27.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.286 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.286 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.229959+0000", 2024-09-14T06:16:27.286 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6450000000000003e-05 2024-09-14T06:16:27.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.287 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.287 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.230011+0000", 2024-09-14T06:16:27.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1711e-05 2024-09-14T06:16:27.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.287 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.287 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.230126+0000", 2024-09-14T06:16:27.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000115259 2024-09-14T06:16:27.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.287 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.287 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.230975+0000", 2024-09-14T06:16:27.288 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000849108 2024-09-14T06:16:27.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.288 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.288 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.288 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.231285+0000", 2024-09-14T06:16:27.288 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00030994900000000001 2024-09-14T06:16:27.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.288 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.288 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.288 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.231367+0000", 2024-09-14T06:16:27.288 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1592000000000001e-05 2024-09-14T06:16:27.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.288 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.288 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.289 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:22.231471+0000", 2024-09-14T06:16:27.289 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010465499999999999 2024-09-14T06:16:27.289 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.289 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.289 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.289 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.289 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:3446 3.1c e28/23 3:3d820525:::rbd_header.1152bd92e04f:head v 28'21, mlcod=28'19)", 2024-09-14T06:16:27.289 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.075122+0000", 2024-09-14T06:16:27.289 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.195725294, 2024-09-14T06:16:27.289 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00075868599999999995, 2024-09-14T06:16:27.289 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.289 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.289 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.290 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.290 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.290 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.290 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.075122+0000", 2024-09-14T06:16:27.290 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.290 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.290 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.290 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.075122+0000", 2024-09-14T06:16:27.290 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.290 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.290 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.290 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.075125+0000", 2024-09-14T06:16:27.291 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.277e-06 2024-09-14T06:16:27.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.291 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.291 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.291 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.075136+0000", 2024-09-14T06:16:27.291 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1564e-05 2024-09-14T06:16:27.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.291 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.291 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.291 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.075158+0000", 2024-09-14T06:16:27.291 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1617e-05 2024-09-14T06:16:27.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.291 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.291 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.291 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.075169+0000", 2024-09-14T06:16:27.292 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0851e-05 2024-09-14T06:16:27.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.292 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.292 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.292 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.075239+0000", 2024-09-14T06:16:27.292 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0343000000000001e-05 2024-09-14T06:16:27.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.292 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.292 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.292 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.075259+0000", 2024-09-14T06:16:27.292 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0409000000000001e-05 2024-09-14T06:16:27.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.292 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.292 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.293 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.075842+0000", 2024-09-14T06:16:27.293 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00058248600000000001 2024-09-14T06:16:27.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.293 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.293 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.293 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.075881+0000", 2024-09-14T06:16:27.293 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9138999999999999e-05 2024-09-14T06:16:27.293 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.293 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.293 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.293 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.293 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:3467 3.1c e28/23 3:3d820525:::rbd_header.1152bd92e04f:head v 28'23, mlcod=28'21)", 2024-09-14T06:16:27.294 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.095693+0000", 2024-09-14T06:16:27.294 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.175154719, 2024-09-14T06:16:27.294 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00081276899999999997, 2024-09-14T06:16:27.294 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.294 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.294 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.294 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.294 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.294 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.294 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.095693+0000", 2024-09-14T06:16:27.294 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.294 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.294 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.295 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.095693+0000", 2024-09-14T06:16:27.295 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.295 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.295 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.295 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.095694+0000", 2024-09-14T06:16:27.295 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4500000000000001e-06 2024-09-14T06:16:27.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.295 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.295 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.295 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.095702+0000", 2024-09-14T06:16:27.295 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4880000000000001e-06 2024-09-14T06:16:27.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.295 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.296 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.296 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.095719+0000", 2024-09-14T06:16:27.296 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7184e-05 2024-09-14T06:16:27.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.296 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.296 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.296 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.095729+0000", 2024-09-14T06:16:27.296 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7729999999999994e-06 2024-09-14T06:16:27.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.296 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.296 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.296 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.095824+0000", 2024-09-14T06:16:27.296 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5284000000000005e-05 2024-09-14T06:16:27.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.297 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.297 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.297 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.095851+0000", 2024-09-14T06:16:27.297 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7044999999999999e-05 2024-09-14T06:16:27.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.297 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.297 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.297 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.096454+0000", 2024-09-14T06:16:27.297 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00060239899999999999 2024-09-14T06:16:27.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.297 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.297 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.297 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.096506+0000", 2024-09-14T06:16:27.298 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2145999999999998e-05 2024-09-14T06:16:27.298 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.298 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.298 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.298 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.298 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3491 3.c 3:30be1cb6:::rbd_data.1152bd92e04f.0000000000000058:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1572864~1505] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T06:16:27.298 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.118562+0000", 2024-09-14T06:16:27.298 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.1522854819999999, 2024-09-14T06:16:27.298 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018678169999999999, 2024-09-14T06:16:27.298 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.298 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.298 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.298 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.298 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:27.299 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:27.299 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3491 2024-09-14T06:16:27.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.299 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.299 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.299 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.118562+0000", 2024-09-14T06:16:27.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.299 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.299 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.118562+0000", 2024-09-14T06:16:27.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.300 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.300 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.300 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.118564+0000", 2024-09-14T06:16:27.300 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.705e-06 2024-09-14T06:16:27.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.300 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.300 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.300 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.118571+0000", 2024-09-14T06:16:27.300 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5889999999999999e-06 2024-09-14T06:16:27.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.300 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.300 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.300 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.118585+0000", 2024-09-14T06:16:27.300 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4289e-05 2024-09-14T06:16:27.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.301 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.301 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.301 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.118591+0000", 2024-09-14T06:16:27.301 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5440000000000004e-06 2024-09-14T06:16:27.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.301 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.301 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.301 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.118643+0000", 2024-09-14T06:16:27.301 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1712999999999998e-05 2024-09-14T06:16:27.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.301 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.301 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.118735+0000", 2024-09-14T06:16:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2025000000000001e-05 2024-09-14T06:16:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 7", 2024-09-14T06:16:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.118820+0000", 2024-09-14T06:16:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5363000000000002e-05 2024-09-14T06:16:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.119396+0000", 2024-09-14T06:16:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057558200000000005 2024-09-14T06:16:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.120353+0000", 2024-09-14T06:16:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00095697999999999996 2024-09-14T06:16:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.120389+0000", 2024-09-14T06:16:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5528000000000003e-05 2024-09-14T06:16:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.120430+0000", 2024-09-14T06:16:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1498999999999997e-05 2024-09-14T06:16:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:3490 3.2 e28/24 3:42ff0881:::rbd_data.1152bd92e04f.0000000000000057:head v 28'9, mlcod=28'7)", 2024-09-14T06:16:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.119166+0000", 2024-09-14T06:16:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.1516815679999999, 2024-09-14T06:16:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00078949900000000004, 2024-09-14T06:16:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.119166+0000", 2024-09-14T06:16:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.119166+0000", 2024-09-14T06:16:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.119167+0000", 2024-09-14T06:16:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.9300000000000006e-07 2024-09-14T06:16:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.119241+0000", 2024-09-14T06:16:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3743000000000002e-05 2024-09-14T06:16:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.119252+0000", 2024-09-14T06:16:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0788000000000001e-05 2024-09-14T06:16:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.119257+0000", 2024-09-14T06:16:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2580000000000003e-06 2024-09-14T06:16:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.119304+0000", 2024-09-14T06:16:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7336999999999998e-05 2024-09-14T06:16:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.119317+0000", 2024-09-14T06:16:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2751999999999999e-05 2024-09-14T06:16:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.119911+0000", 2024-09-14T06:16:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00059444100000000002 2024-09-14T06:16:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.119956+0000", 2024-09-14T06:16:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4187000000000002e-05 2024-09-14T06:16:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:3493 3.1c e28/23 3:3d820525:::rbd_header.1152bd92e04f:head v 28'25, mlcod=28'23)", 2024-09-14T06:16:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.125888+0000", 2024-09-14T06:16:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.1449596099999999, 2024-09-14T06:16:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00083121400000000004, 2024-09-14T06:16:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.125888+0000", 2024-09-14T06:16:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.125888+0000", 2024-09-14T06:16:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.125890+0000", 2024-09-14T06:16:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9920000000000002e-06 2024-09-14T06:16:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.125900+0000", 2024-09-14T06:16:27.311 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0039e-05 2024-09-14T06:16:27.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.311 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.311 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.125920+0000", 2024-09-14T06:16:27.311 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0152999999999998e-05 2024-09-14T06:16:27.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.311 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.311 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.125930+0000", 2024-09-14T06:16:27.311 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7790000000000007e-06 2024-09-14T06:16:27.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.311 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.126005+0000", 2024-09-14T06:16:27.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5285999999999997e-05 2024-09-14T06:16:27.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.312 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.126032+0000", 2024-09-14T06:16:27.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6869e-05 2024-09-14T06:16:27.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.312 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.126667+0000", 2024-09-14T06:16:27.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00063496799999999997 2024-09-14T06:16:27.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.313 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.313 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.126719+0000", 2024-09-14T06:16:27.313 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2128e-05 2024-09-14T06:16:27.313 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.313 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.313 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.313 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3505 3.e 3:72664ea8:::rbd_data.1152bd92e04f.0000000000000066:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1879265~4895] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T06:16:27.313 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.133342+0000", 2024-09-14T06:16:27.313 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.137505875, 2024-09-14T06:16:27.313 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002641624, 2024-09-14T06:16:27.313 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.313 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.313 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.314 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.314 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:27.314 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:27.314 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3505 2024-09-14T06:16:27.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.314 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.314 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.314 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.314 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.133342+0000", 2024-09-14T06:16:27.314 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.314 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.314 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.314 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.133342+0000", 2024-09-14T06:16:27.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.315 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.315 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.133344+0000", 2024-09-14T06:16:27.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2299999999999998e-06 2024-09-14T06:16:27.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.315 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.315 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.133356+0000", 2024-09-14T06:16:27.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1985e-05 2024-09-14T06:16:27.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.315 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.316 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.133373+0000", 2024-09-14T06:16:27.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6742e-05 2024-09-14T06:16:27.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.316 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.316 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.133382+0000", 2024-09-14T06:16:27.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5840000000000001e-06 2024-09-14T06:16:27.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.316 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.316 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.133467+0000", 2024-09-14T06:16:27.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4666999999999994e-05 2024-09-14T06:16:27.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.133639+0000", 2024-09-14T06:16:27.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000172048 2024-09-14T06:16:27.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.133771+0000", 2024-09-14T06:16:27.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013148399999999999 2024-09-14T06:16:27.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.134409+0000", 2024-09-14T06:16:27.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00063798000000000001 2024-09-14T06:16:27.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.135915+0000", 2024-09-14T06:16:27.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0015064220000000001 2024-09-14T06:16:27.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.135952+0000", 2024-09-14T06:16:27.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6683000000000001e-05 2024-09-14T06:16:27.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.135983+0000", 2024-09-14T06:16:27.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1798999999999998e-05 2024-09-14T06:16:27.318 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.319 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.319 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.319 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:3512 3.1b e28/23 3:d8c7e8ff:::rbd_data.1152bd92e04f.0000000000000060:head v 28'3, mlcod=27'1)", 2024-09-14T06:16:27.319 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.143015+0000", 2024-09-14T06:16:27.319 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.1278326080000001, 2024-09-14T06:16:27.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0012654159999999999, 2024-09-14T06:16:27.319 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.319 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.319 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.319 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.319 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.143015+0000", 2024-09-14T06:16:27.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.143015+0000", 2024-09-14T06:16:27.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.143016+0000", 2024-09-14T06:16:27.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6200000000000006e-07 2024-09-14T06:16:27.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.321 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.321 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.143467+0000", 2024-09-14T06:16:27.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000450872 2024-09-14T06:16:27.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.321 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.321 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.143483+0000", 2024-09-14T06:16:27.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5557999999999999e-05 2024-09-14T06:16:27.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.321 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.321 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.143491+0000", 2024-09-14T06:16:27.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0180000000000001e-06 2024-09-14T06:16:27.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.143558+0000", 2024-09-14T06:16:27.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7840000000000001e-05 2024-09-14T06:16:27.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.143580+0000", 2024-09-14T06:16:27.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1897999999999999e-05 2024-09-14T06:16:27.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.144236+0000", 2024-09-14T06:16:27.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00065543100000000005 2024-09-14T06:16:27.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.323 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.144281+0000", 2024-09-14T06:16:27.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4836999999999998e-05 2024-09-14T06:16:27.323 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.323 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.323 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.323 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3524 3.c 3:30be1cb6:::rbd_data.1152bd92e04f.0000000000000058:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T06:16:27.323 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.161470+0000", 2024-09-14T06:16:27.323 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.109377767, 2024-09-14T06:16:27.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033787499999999998, 2024-09-14T06:16:27.324 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.324 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.324 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.324 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.324 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:27.324 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:27.324 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3524 2024-09-14T06:16:27.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.324 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.324 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.324 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161470+0000", 2024-09-14T06:16:27.324 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.325 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.325 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161470+0000", 2024-09-14T06:16:27.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.325 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.325 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161472+0000", 2024-09-14T06:16:27.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:16:27.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.325 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.325 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161479+0000", 2024-09-14T06:16:27.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.861e-06 2024-09-14T06:16:27.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161502+0000", 2024-09-14T06:16:27.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3543000000000001e-05 2024-09-14T06:16:27.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161514+0000", 2024-09-14T06:16:27.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1355000000000001e-05 2024-09-14T06:16:27.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161617+0000", 2024-09-14T06:16:27.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010331 2024-09-14T06:16:27.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.327 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.327 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161717+0000", 2024-09-14T06:16:27.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.9710000000000006e-05 2024-09-14T06:16:27.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.327 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 7", 2024-09-14T06:16:27.327 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161831+0000", 2024-09-14T06:16:27.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000114566 2024-09-14T06:16:27.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.327 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.162498+0000", 2024-09-14T06:16:27.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00066716099999999999 2024-09-14T06:16:27.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.164730+0000", 2024-09-14T06:16:27.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022317529999999999 2024-09-14T06:16:27.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.164784+0000", 2024-09-14T06:16:27.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3983999999999997e-05 2024-09-14T06:16:27.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.329 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.329 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.164849+0000", 2024-09-14T06:16:27.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4554999999999994e-05 2024-09-14T06:16:27.329 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.329 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.329 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.329 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3529 3.e 3:72664ea8:::rbd_data.1152bd92e04f.0000000000000066:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T06:16:27.329 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.161555+0000", 2024-09-14T06:16:27.329 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.1092924049999999, 2024-09-14T06:16:27.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002339907, 2024-09-14T06:16:27.329 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.329 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.330 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.330 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.330 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:27.330 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:27.330 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3529 2024-09-14T06:16:27.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.330 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161555+0000", 2024-09-14T06:16:27.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161555+0000", 2024-09-14T06:16:27.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161556+0000", 2024-09-14T06:16:27.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5199999999999996e-07 2024-09-14T06:16:27.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161560+0000", 2024-09-14T06:16:27.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8539999999999999e-06 2024-09-14T06:16:27.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161571+0000", 2024-09-14T06:16:27.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0627e-05 2024-09-14T06:16:27.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161581+0000", 2024-09-14T06:16:27.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0835999999999999e-05 2024-09-14T06:16:27.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161648+0000", 2024-09-14T06:16:27.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6587999999999993e-05 2024-09-14T06:16:27.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.333 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161739+0000", 2024-09-14T06:16:27.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0674000000000006e-05 2024-09-14T06:16:27.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.333 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.161887+0000", 2024-09-14T06:16:27.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00014874300000000001 2024-09-14T06:16:27.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.333 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.162706+0000", 2024-09-14T06:16:27.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00081902099999999996 2024-09-14T06:16:27.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.334 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.163816+0000", 2024-09-14T06:16:27.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001109608 2024-09-14T06:16:27.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.334 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.163867+0000", 2024-09-14T06:16:27.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0611e-05 2024-09-14T06:16:27.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.334 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.163895+0000", 2024-09-14T06:16:27.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8592999999999999e-05 2024-09-14T06:16:27.335 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.335 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.335 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.335 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:3525 3.1b e28/23 3:d8c7e8ff:::rbd_data.1152bd92e04f.0000000000000060:head v 28'4, mlcod=27'2)", 2024-09-14T06:16:27.335 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.163614+0000", 2024-09-14T06:16:27.335 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.107233578, 2024-09-14T06:16:27.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0012428700000000001, 2024-09-14T06:16:27.335 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.335 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.335 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.335 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.163614+0000", 2024-09-14T06:16:27.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.163614+0000", 2024-09-14T06:16:27.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.163615+0000", 2024-09-14T06:16:27.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:16:27.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.337 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.337 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.163618+0000", 2024-09-14T06:16:27.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.844e-06 2024-09-14T06:16:27.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.337 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.337 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.163628+0000", 2024-09-14T06:16:27.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4380000000000001e-06 2024-09-14T06:16:27.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.337 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.337 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.163631+0000", 2024-09-14T06:16:27.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3500000000000001e-06 2024-09-14T06:16:27.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.163677+0000", 2024-09-14T06:16:27.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6264000000000003e-05 2024-09-14T06:16:27.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.163696+0000", 2024-09-14T06:16:27.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8751999999999999e-05 2024-09-14T06:16:27.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.164769+0000", 2024-09-14T06:16:27.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0010726939999999999 2024-09-14T06:16:27.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.339 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.164857+0000", 2024-09-14T06:16:27.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8166999999999998e-05 2024-09-14T06:16:27.339 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.339 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.339 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.339 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3548 3.e 3:76656e66:::rbd_data.1152bd92e04f.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write 262144~10530] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T06:16:27.339 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.176440+0000", 2024-09-14T06:16:27.339 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.0944080860000001, 2024-09-14T06:16:27.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002904214, 2024-09-14T06:16:27.340 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.340 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.340 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.340 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.340 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:27.340 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:27.340 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3548 2024-09-14T06:16:27.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.340 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176440+0000", 2024-09-14T06:16:27.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176440+0000", 2024-09-14T06:16:27.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176441+0000", 2024-09-14T06:16:27.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6700000000000001e-06 2024-09-14T06:16:27.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176453+0000", 2024-09-14T06:16:27.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1532000000000001e-05 2024-09-14T06:16:27.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176466+0000", 2024-09-14T06:16:27.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3628999999999999e-05 2024-09-14T06:16:27.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176477+0000", 2024-09-14T06:16:27.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0971999999999999e-05 2024-09-14T06:16:27.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176529+0000", 2024-09-14T06:16:27.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1748e-05 2024-09-14T06:16:27.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176624+0000", 2024-09-14T06:16:27.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4895000000000005e-05 2024-09-14T06:16:27.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176711+0000", 2024-09-14T06:16:27.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7233999999999999e-05 2024-09-14T06:16:27.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.344 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.177515+0000", 2024-09-14T06:16:27.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00080336100000000005 2024-09-14T06:16:27.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.344 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.344 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.179260+0000", 2024-09-14T06:16:27.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0017456590000000001 2024-09-14T06:16:27.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.344 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.344 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.179295+0000", 2024-09-14T06:16:27.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5111000000000003e-05 2024-09-14T06:16:27.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.345 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.345 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.179344+0000", 2024-09-14T06:16:27.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8402999999999997e-05 2024-09-14T06:16:27.345 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.345 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.345 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.345 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:3547 3.1b e28/23 3:df53c65b:::rbd_data.1152bd92e04f.000000000000003d:head v 28'5, mlcod=28'3)", 2024-09-14T06:16:27.345 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.176733+0000", 2024-09-14T06:16:27.345 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.0941148519999999, 2024-09-14T06:16:27.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001100777, 2024-09-14T06:16:27.345 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.345 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.346 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.346 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176733+0000", 2024-09-14T06:16:27.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176733+0000", 2024-09-14T06:16:27.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176734+0000", 2024-09-14T06:16:27.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8100000000000001e-07 2024-09-14T06:16:27.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176750+0000", 2024-09-14T06:16:27.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6617e-05 2024-09-14T06:16:27.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176767+0000", 2024-09-14T06:16:27.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6560000000000001e-05 2024-09-14T06:16:27.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176774+0000", 2024-09-14T06:16:27.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2389999999999996e-06 2024-09-14T06:16:27.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176848+0000", 2024-09-14T06:16:27.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3638000000000002e-05 2024-09-14T06:16:27.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.176868+0000", 2024-09-14T06:16:27.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0302e-05 2024-09-14T06:16:27.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.177798+0000", 2024-09-14T06:16:27.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00093027599999999995 2024-09-14T06:16:27.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.177834+0000", 2024-09-14T06:16:27.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5163999999999997e-05 2024-09-14T06:16:27.349 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.349 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.349 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.350 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3566 3.f 3:f3a57165:::rbd_data.1152bd92e04f.000000000000004c:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1310720~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T06:16:27.350 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.192999+0000", 2024-09-14T06:16:27.350 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.077848457, 2024-09-14T06:16:27.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001510679, 2024-09-14T06:16:27.350 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.350 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.350 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.350 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.350 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:27.350 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:27.350 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3566 2024-09-14T06:16:27.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.350 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.351 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.351 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.192999+0000", 2024-09-14T06:16:27.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.351 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.351 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.192999+0000", 2024-09-14T06:16:27.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.351 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.351 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.193001+0000", 2024-09-14T06:16:27.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1969999999999999e-06 2024-09-14T06:16:27.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.193017+0000", 2024-09-14T06:16:27.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5291999999999999e-05 2024-09-14T06:16:27.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.193040+0000", 2024-09-14T06:16:27.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3595999999999999e-05 2024-09-14T06:16:27.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.193050+0000", 2024-09-14T06:16:27.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0005e-05 2024-09-14T06:16:27.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.193121+0000", 2024-09-14T06:16:27.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0210999999999996e-05 2024-09-14T06:16:27.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.193208+0000", 2024-09-14T06:16:27.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6942999999999997e-05 2024-09-14T06:16:27.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:27.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.193329+0000", 2024-09-14T06:16:27.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012139800000000001 2024-09-14T06:16:27.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.194045+0000", 2024-09-14T06:16:27.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00071623100000000001 2024-09-14T06:16:27.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.194444+0000", 2024-09-14T06:16:27.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00039861200000000001 2024-09-14T06:16:27.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.194480+0000", 2024-09-14T06:16:27.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6247e-05 2024-09-14T06:16:27.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.194510+0000", 2024-09-14T06:16:27.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9947e-05 2024-09-14T06:16:27.355 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.355 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.355 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.355 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3580 3.15 3:ae096092:::rbd_data.1152bd92e04f.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 544722~12334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T06:16:27.356 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.205310+0000", 2024-09-14T06:16:27.356 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.065538106, 2024-09-14T06:16:27.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022896140000000001, 2024-09-14T06:16:27.356 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.356 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.356 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.356 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.356 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:27.356 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:27.356 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3580 2024-09-14T06:16:27.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.356 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.205310+0000", 2024-09-14T06:16:27.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.205310+0000", 2024-09-14T06:16:27.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.205311+0000", 2024-09-14T06:16:27.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7460000000000001e-06 2024-09-14T06:16:27.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.205325+0000", 2024-09-14T06:16:27.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3816e-05 2024-09-14T06:16:27.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.205340+0000", 2024-09-14T06:16:27.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4367e-05 2024-09-14T06:16:27.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.205348+0000", 2024-09-14T06:16:27.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5429999999999994e-06 2024-09-14T06:16:27.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.205418+0000", 2024-09-14T06:16:27.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9789999999999994e-05 2024-09-14T06:16:27.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.205564+0000", 2024-09-14T06:16:27.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000146093 2024-09-14T06:16:27.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:27.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.205680+0000", 2024-09-14T06:16:27.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000116304 2024-09-14T06:16:27.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.206391+0000", 2024-09-14T06:16:27.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00071030099999999999 2024-09-14T06:16:27.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.207525+0000", 2024-09-14T06:16:27.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011343410000000001 2024-09-14T06:16:27.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.207561+0000", 2024-09-14T06:16:27.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:16:27.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.207599+0000", 2024-09-14T06:16:27.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7764e-05 2024-09-14T06:16:27.361 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.361 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.361 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.361 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:3581 3.2 e28/24 3:42a54017:::rbd_data.1152bd92e04f.000000000000005e:head v 28'11, mlcod=28'9)", 2024-09-14T06:16:27.362 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.206046+0000", 2024-09-14T06:16:27.362 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.0648018109999999, 2024-09-14T06:16:27.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00078221900000000001, 2024-09-14T06:16:27.362 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.362 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.362 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.362 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.206046+0000", 2024-09-14T06:16:27.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.206046+0000", 2024-09-14T06:16:27.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.206047+0000", 2024-09-14T06:16:27.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4600000000000004e-07 2024-09-14T06:16:27.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.206060+0000", 2024-09-14T06:16:27.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3760999999999999e-05 2024-09-14T06:16:27.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.206070+0000", 2024-09-14T06:16:27.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7389999999999994e-06 2024-09-14T06:16:27.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.206075+0000", 2024-09-14T06:16:27.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.826e-06 2024-09-14T06:16:27.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.206118+0000", 2024-09-14T06:16:27.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2576e-05 2024-09-14T06:16:27.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.206132+0000", 2024-09-14T06:16:27.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4544e-05 2024-09-14T06:16:27.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.206797+0000", 2024-09-14T06:16:27.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00066520700000000002 2024-09-14T06:16:27.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.206828+0000", 2024-09-14T06:16:27.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.082e-05 2024-09-14T06:16:27.366 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.366 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.366 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.366 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:3584 3.1c e28/23 3:3d820525:::rbd_header.1152bd92e04f:head v 28'29, mlcod=28'27)", 2024-09-14T06:16:27.366 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.214717+0000", 2024-09-14T06:16:27.366 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.0561310580000001, 2024-09-14T06:16:27.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00096712999999999996, 2024-09-14T06:16:27.366 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.366 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.366 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.366 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.214717+0000", 2024-09-14T06:16:27.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.214717+0000", 2024-09-14T06:16:27.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.214718+0000", 2024-09-14T06:16:27.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.621e-06 2024-09-14T06:16:27.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.214729+0000", 2024-09-14T06:16:27.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.113e-05 2024-09-14T06:16:27.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.214750+0000", 2024-09-14T06:16:27.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0398999999999999e-05 2024-09-14T06:16:27.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.214760+0000", 2024-09-14T06:16:27.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0638999999999999e-05 2024-09-14T06:16:27.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.214835+0000", 2024-09-14T06:16:27.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4474000000000006e-05 2024-09-14T06:16:27.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.214857+0000", 2024-09-14T06:16:27.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2531000000000002e-05 2024-09-14T06:16:27.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.215633+0000", 2024-09-14T06:16:27.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00077556499999999998 2024-09-14T06:16:27.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.370 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.370 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.215684+0000", 2024-09-14T06:16:27.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0770999999999999e-05 2024-09-14T06:16:27.370 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.370 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.370 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.370 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3597 3.c 3:314382a0:::rbd_data.1152bd92e04f.0000000000000070:head [set-alloc-hint object_size 2097152 write_size 2097152,write 425984~5314] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T06:16:27.370 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:26.222016+0000", 2024-09-14T06:16:27.370 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.048831651, 2024-09-14T06:16:27.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032931509999999998, 2024-09-14T06:16:27.371 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.371 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.371 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.371 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.371 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:27.371 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:27.371 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3597 2024-09-14T06:16:27.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.371 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.222016+0000", 2024-09-14T06:16:27.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.222016+0000", 2024-09-14T06:16:27.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.222018+0000", 2024-09-14T06:16:27.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8589999999999999e-06 2024-09-14T06:16:27.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.222031+0000", 2024-09-14T06:16:27.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2964e-05 2024-09-14T06:16:27.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.222045+0000", 2024-09-14T06:16:27.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3984e-05 2024-09-14T06:16:27.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.222052+0000", 2024-09-14T06:16:27.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5610000000000001e-06 2024-09-14T06:16:27.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.222112+0000", 2024-09-14T06:16:27.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9750000000000002e-05 2024-09-14T06:16:27.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.222240+0000", 2024-09-14T06:16:27.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012790800000000001 2024-09-14T06:16:27.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 7", 2024-09-14T06:16:27.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.222336+0000", 2024-09-14T06:16:27.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5443000000000001e-05 2024-09-14T06:16:27.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.223158+0000", 2024-09-14T06:16:27.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00082215200000000004 2024-09-14T06:16:27.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.225133+0000", 2024-09-14T06:16:27.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001975058 2024-09-14T06:16:27.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.225245+0000", 2024-09-14T06:16:27.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000112452 2024-09-14T06:16:27.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:26.225309+0000", 2024-09-14T06:16:27.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.402e-05 2024-09-14T06:16:27.376 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.376 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.376 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.376 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.376 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.376 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:27.378 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:16:27.379 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:27.379 INFO:teuthology.orchestra.run.smithi039.stdout:261 read 0xa3297fd thru 0xa32d60a (0x3e0e bytes) 2024-09-14T06:16:27.379 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:27.379 INFO:teuthology.orchestra.run.smithi039.stdout:263 write 0xb20c0f5 thru 0xb21407b (0x7f87 bytes) 2024-09-14T06:16:27.379 INFO:teuthology.orchestra.run.smithi039.stdout:264 write 0xe93fb0a thru 0xe93fba8 (0x9f bytes) 2024-09-14T06:16:27.379 INFO:teuthology.orchestra.run.smithi039.stdout:266 punch from 0xe603b84 to 0xe611e41, (0xe2bd bytes) 2024-09-14T06:16:27.379 INFO:teuthology.orchestra.run.smithi039.stdout:267 trunc from 0xe93fba9 to 0x31c3793 2024-09-14T06:16:27.379 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:27.372+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:27.383 INFO:teuthology.orchestra.run.smithi039.stdout:268 write 0xbf2b169 thru 0xbf2f984 (0x481c bytes) 2024-09-14T06:16:27.390 INFO:teuthology.orchestra.run.smithi039.stdout:270 punch from 0x1604a40 to 0x160f5e8, (0xaba8 bytes) 2024-09-14T06:16:27.390 INFO:teuthology.orchestra.run.smithi039.stdout:271 write 0xb730f3f thru 0xb73b0d8 (0xa19a bytes) 2024-09-14T06:16:27.399 INFO:teuthology.orchestra.run.smithi039.stdout:272 read 0x5797061 thru 0x57a6ee5 (0xfe85 bytes) 2024-09-14T06:16:27.409 INFO:teuthology.orchestra.run.smithi039.stdout:273 writesame 0xd2f346f thru 0xd2fb216 data_size 0x4(0x7da8 bytes) 2024-09-14T06:16:27.409 INFO:teuthology.orchestra.run.smithi039.stdout:275 punch from 0x35fffbc to 0x36057d4, (0x5818 bytes) 2024-09-14T06:16:27.409 INFO:teuthology.orchestra.run.smithi039.stdout:276 read 0xaed343d thru 0xaedec94 (0xb858 bytes) 2024-09-14T06:16:27.410 INFO:teuthology.orchestra.run.smithi039.stdout:277 write 0x9ca8d2d thru 0x9cb049a (0x776e bytes) 2024-09-14T06:16:27.418 INFO:teuthology.orchestra.run.smithi039.stdout:278 read 0xba71716 thru 0xba776c2 (0x5fad bytes) 2024-09-14T06:16:27.418 INFO:teuthology.orchestra.run.smithi039.stdout:279 punch from 0x99a3dd5 to 0x99b2bf8, (0xee23 bytes) 2024-09-14T06:16:27.418 INFO:teuthology.orchestra.run.smithi039.stdout:280 read 0x3120dbd thru 0x312bf53 (0xb197 bytes) 2024-09-14T06:16:27.418 INFO:teuthology.orchestra.run.smithi039.stdout:282 read 0xb0a8749 thru 0xb0b0fea (0x88a2 bytes) 2024-09-14T06:16:27.419 INFO:teuthology.orchestra.run.smithi039.stdout:283 read 0xc7e78c3 thru 0xc7f5127 (0xd865 bytes) 2024-09-14T06:16:27.419 INFO:teuthology.orchestra.run.smithi039.stdout:284 write 0xd2cb9a8 thru 0xd2d43c4 (0x8a1d bytes) 2024-09-14T06:16:27.435 INFO:teuthology.orchestra.run.smithi039.stdout:286 trunc from 0xd2fb217 to 0x8032549 2024-09-14T06:16:27.455 INFO:teuthology.orchestra.run.smithi039.stdout:288 punch from 0x576f847 to 0x5775f03, (0x66bc bytes) 2024-09-14T06:16:27.469 INFO:teuthology.orchestra.run.smithi039.stdout:289 write 0xc840291 thru 0xc8498d2 (0x9642 bytes) 2024-09-14T06:16:27.478 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:27.476+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:27.491 INFO:teuthology.orchestra.run.smithi039.stdout:290 trunc from 0xc8498d3 to 0x2eda4b7 2024-09-14T06:16:27.513 INFO:teuthology.orchestra.run.smithi039.stdout:291 read 0x12ed154 thru 0x12f2484 (0x5331 bytes) 2024-09-14T06:16:27.526 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:27.526 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:27.526 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:27.526 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:27.539 INFO:teuthology.orchestra.run.smithi039.stdout:292 write 0xd6576d0 thru 0xd65b58b (0x3ebc bytes) 2024-09-14T06:16:27.547 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:16:27.565 INFO:teuthology.orchestra.run.smithi039.stdout:296 trunc from 0xd65b58c to 0x1238399 2024-09-14T06:16:27.579 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:27.576+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:27.633 INFO:teuthology.orchestra.run.smithi039.stdout:297 write 0x20b80b5 thru 0x20c1e07 (0x9d53 bytes) 2024-09-14T06:16:27.657 INFO:teuthology.orchestra.run.smithi039.stdout:298 write 0xaa30ac7 thru 0xaa38d75 (0x82af bytes) 2024-09-14T06:16:27.671 INFO:teuthology.orchestra.run.smithi039.stdout:301 write 0xb5af73d thru 0xb5b8960 (0x9224 bytes) 2024-09-14T06:16:27.680 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:27.676+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:27.680 INFO:teuthology.orchestra.run.smithi039.stdout:302 read 0x4828f89 thru 0x48377e0 (0xe858 bytes) 2024-09-14T06:16:27.680 INFO:teuthology.orchestra.run.smithi039.stdout:303 read 0x8099b8a thru 0x809c604 (0x2a7b bytes) 2024-09-14T06:16:27.680 INFO:teuthology.orchestra.run.smithi039.stdout:304 read 0x11c694 thru 0x124e7b (0x87e8 bytes) 2024-09-14T06:16:27.681 INFO:teuthology.orchestra.run.smithi039.stdout:305 read 0x1117018 thru 0x111b25d (0x4246 bytes) 2024-09-14T06:16:27.682 INFO:teuthology.orchestra.run.smithi039.stdout:306 punch from 0x1d06ca9 to 0x1d10487, (0x97de bytes) 2024-09-14T06:16:27.682 INFO:teuthology.orchestra.run.smithi039.stdout:307 punch from 0x5b33db6 to 0x5b3b310, (0x755a bytes) 2024-09-14T06:16:27.682 INFO:teuthology.orchestra.run.smithi039.stdout:309 write 0x51dbe49 thru 0x51de4c9 (0x2681 bytes) 2024-09-14T06:16:27.694 INFO:teuthology.orchestra.run.smithi039.stdout:310 write 0xba9937a thru 0xba9d965 (0x45ec bytes) 2024-09-14T06:16:27.699 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:27.699 INFO:teuthology.orchestra.run.smithi039.stdout:312 read 0x941cd72 thru 0x9422268 (0x54f7 bytes) 2024-09-14T06:16:27.699 INFO:teuthology.orchestra.run.smithi039.stdout:316 writesame 0x76a5b64 thru 0x76b1e83 data_size 0x10(0xc320 bytes) 2024-09-14T06:16:27.716 INFO:teuthology.orchestra.run.smithi039.stdout:319 write 0xc8b2351 thru 0xc8bff90 (0xdc40 bytes) 2024-09-14T06:16:27.722 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:27.722 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:27.722 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:27.722 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:27.722 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:27.727 INFO:teuthology.orchestra.run.smithi039.stdout:320 read 0x3c770f4 thru 0x3c7905d (0x1f6a bytes) 2024-09-14T06:16:27.727 INFO:teuthology.orchestra.run.smithi039.stdout:321 writesame 0xb6021ce thru 0xb604dc8 data_size 0x1b(0x2bfb bytes) 2024-09-14T06:16:27.730 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:27.730 INFO:teuthology.orchestra.run.smithi039.stdout:323 writesame 0x963e2a thru 0x96f9f9 data_size 0x28(0xbbd0 bytes) 2024-09-14T06:16:27.737 INFO:teuthology.orchestra.run.smithi039.stdout:325 trunc from 0xc8bff91 to 0xad76763 2024-09-14T06:16:27.739 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:16:27.764 INFO:teuthology.orchestra.run.smithi039.stdout:326 trunc from 0xad76763 to 0xdc90db5 2024-09-14T06:16:27.770 INFO:teuthology.orchestra.run.smithi039.stdout:327 read 0x85e28d4 thru 0x85edc10 (0xb33d bytes) 2024-09-14T06:16:27.770 INFO:teuthology.orchestra.run.smithi039.stdout:328 write 0x37dc5ef thru 0x37e3217 (0x6c29 bytes) 2024-09-14T06:16:27.776 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:27.776 INFO:teuthology.orchestra.run.smithi039.stdout:330 write 0x6f45f9b thru 0x6f4a76a (0x47d0 bytes) 2024-09-14T06:16:27.780 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:27.776+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:27.782 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:27.782 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:27.782 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:27.782 INFO:teuthology.orchestra.run.smithi039.stdout:335 read 0x72a94e0 thru 0x72b3973 (0xa494 bytes) 2024-09-14T06:16:27.783 INFO:teuthology.orchestra.run.smithi039.stdout:336 write 0x8c84d11 thru 0x8c8b701 (0x69f1 bytes) 2024-09-14T06:16:27.788 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:27.788 INFO:teuthology.orchestra.run.smithi039.stdout:340 read 0x7edc155 thru 0x7ee6d5f (0xac0b bytes) 2024-09-14T06:16:27.788 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:27.788 INFO:teuthology.orchestra.run.smithi039.stdout:344 read 0x37dd277 thru 0x37e0be7 (0x3971 bytes) 2024-09-14T06:16:27.789 INFO:teuthology.orchestra.run.smithi039.stdout:345 read 0xc5586f7 thru 0xc55906f (0x979 bytes) 2024-09-14T06:16:27.789 INFO:teuthology.orchestra.run.smithi039.stdout:346 read 0xcea10ca thru 0xceabf1a (0xae51 bytes) 2024-09-14T06:16:27.789 INFO:teuthology.orchestra.run.smithi039.stdout:347 write 0x107a424 thru 0x10889ce (0xe5ab bytes) 2024-09-14T06:16:27.794 INFO:teuthology.orchestra.run.smithi039.stdout:348 write 0xa4bccba thru 0xa4bd1bf (0x506 bytes) 2024-09-14T06:16:27.798 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:27.798 INFO:teuthology.orchestra.run.smithi039.stdout:350 read 0x859e36b thru 0x85ac680 (0xe316 bytes) 2024-09-14T06:16:27.798 INFO:teuthology.orchestra.run.smithi039.stdout:351 write 0xcd51e47 thru 0xcd54f56 (0x3110 bytes) 2024-09-14T06:16:27.803 INFO:teuthology.orchestra.run.smithi039.stdout:352 read 0xc7d453b thru 0xc7d61a7 (0x1c6d bytes) 2024-09-14T06:16:27.804 INFO:teuthology.orchestra.run.smithi039.stdout:353 read 0xdc77384 thru 0xdc7d67d (0x62fa bytes) 2024-09-14T06:16:27.804 INFO:teuthology.orchestra.run.smithi039.stdout:354 write 0x9726503 thru 0x97361f6 (0xfcf4 bytes) 2024-09-14T06:16:27.813 INFO:teuthology.orchestra.run.smithi039.stdout:356 write 0xb88a29c thru 0xb88c6e5 (0x244a bytes) 2024-09-14T06:16:27.816 INFO:teuthology.orchestra.run.smithi039.stdout:357 punch from 0x6714a39 to 0x671d8f8, (0x8ebf bytes) 2024-09-14T06:16:27.816 INFO:teuthology.orchestra.run.smithi039.stdout:358 punch from 0xad2f9b3 to 0xad3e8f1, (0xef3e bytes) 2024-09-14T06:16:27.817 INFO:teuthology.orchestra.run.smithi039.stdout:360 writesame 0x3314a3b thru 0x331c0ef data_size 0x1(0x76b5 bytes) 2024-09-14T06:16:27.842 INFO:teuthology.orchestra.run.smithi039.stdout:361 read 0x4534f91 thru 0x4536769 (0x17d9 bytes) 2024-09-14T06:16:27.842 INFO:teuthology.orchestra.run.smithi039.stdout:363 punch from 0xd10fb02 to 0xd114eb4, (0x53b2 bytes) 2024-09-14T06:16:27.842 INFO:teuthology.orchestra.run.smithi039.stdout:364 writesame 0x98c33d5 thru 0x98c835f data_size 0x1(0x4f8b bytes) 2024-09-14T06:16:27.872 INFO:teuthology.orchestra.run.smithi039.stdout:365 write 0xed08969 thru 0xed126af (0x9d47 bytes) 2024-09-14T06:16:27.877 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:27.878 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:27.878 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:27.878 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:27.878 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.878 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.878 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:27.878 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.799598191000001, 2024-09-14T06:16:27.878 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.56009052699999995, 2024-09-14T06:16:27.878 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.878 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.878 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.878 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.878 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.879 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.879 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 1 2024-09-14T06:16:27.879 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.879 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.879 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.879 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.879 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:27.879 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.879 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.879 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.879 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.879 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:27.879 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.880 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.880 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.880 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077065+0000", 2024-09-14T06:16:27.880 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6429999999999999e-06 2024-09-14T06:16:27.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.880 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.880 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.880 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077079+0000", 2024-09-14T06:16:27.880 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4447000000000001e-05 2024-09-14T06:16:27.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.880 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.880 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.880 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077103+0000", 2024-09-14T06:16:27.881 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3856e-05 2024-09-14T06:16:27.881 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.881 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.881 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.881 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077120+0000", 2024-09-14T06:16:27.881 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7337e-05 2024-09-14T06:16:27.881 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.881 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.881 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.881 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077255+0000", 2024-09-14T06:16:27.881 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000134652 2024-09-14T06:16:27.881 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.881 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.882 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:16:27.882 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077274+0000", 2024-09-14T06:16:27.882 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9003000000000001e-05 2024-09-14T06:16:27.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.882 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.882 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.882 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634272+0000", 2024-09-14T06:16:27.882 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.55699813099999995 2024-09-14T06:16:27.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.882 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.882 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.882 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634591+0000", 2024-09-14T06:16:27.882 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031910800000000002 2024-09-14T06:16:27.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.882 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.883 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.883 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634892+0000", 2024-09-14T06:16:27.883 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00030066400000000002 2024-09-14T06:16:27.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.883 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.883 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.883 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636345+0000", 2024-09-14T06:16:27.883 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001452896 2024-09-14T06:16:27.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.883 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.883 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.883 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636974+0000", 2024-09-14T06:16:27.883 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00062927800000000004 2024-09-14T06:16:27.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.884 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.884 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.884 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637071+0000", 2024-09-14T06:16:27.884 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7178999999999999e-05 2024-09-14T06:16:27.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.884 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.884 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.884 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637153+0000", 2024-09-14T06:16:27.884 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1333000000000002e-05 2024-09-14T06:16:27.884 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.884 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.884 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.884 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.885 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.885 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.674573+0000", 2024-09-14T06:16:27.885 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.20208689, 2024-09-14T06:16:27.885 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0096001760000000002, 2024-09-14T06:16:27.885 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.885 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.885 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.885 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.885 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.885 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.885 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 25 2024-09-14T06:16:27.885 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.885 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.886 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.886 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.886 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.674573+0000", 2024-09-14T06:16:27.886 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.886 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.886 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.886 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.674573+0000", 2024-09-14T06:16:27.886 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.886 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.886 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.886 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.674574+0000", 2024-09-14T06:16:27.886 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0869999999999999e-06 2024-09-14T06:16:27.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.886 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.887 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.887 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.674582+0000", 2024-09-14T06:16:27.887 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5329999999999997e-06 2024-09-14T06:16:27.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.887 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.887 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.887 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.674595+0000", 2024-09-14T06:16:27.887 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2767e-05 2024-09-14T06:16:27.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.887 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.887 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.887 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.674624+0000", 2024-09-14T06:16:27.887 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8946e-05 2024-09-14T06:16:27.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.887 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.887 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.888 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.679074+0000", 2024-09-14T06:16:27.888 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044506659999999998 2024-09-14T06:16:27.888 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.888 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.888 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.888 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.679256+0000", 2024-09-14T06:16:27.888 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000181845 2024-09-14T06:16:27.888 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.888 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.888 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.888 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.679545+0000", 2024-09-14T06:16:27.888 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00028854800000000002 2024-09-14T06:16:27.888 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.888 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.888 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.889 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.680986+0000", 2024-09-14T06:16:27.889 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001441152 2024-09-14T06:16:27.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.889 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.889 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.889 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.684029+0000", 2024-09-14T06:16:27.889 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030426060000000002 2024-09-14T06:16:27.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.889 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.889 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.889 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.684111+0000", 2024-09-14T06:16:27.889 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2109999999999998e-05 2024-09-14T06:16:27.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.889 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.889 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.890 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.684174+0000", 2024-09-14T06:16:27.890 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2916000000000005e-05 2024-09-14T06:16:27.890 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.890 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.890 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.890 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.890 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.890 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:41 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.890 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.774917+0000", 2024-09-14T06:16:27.890 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.101743407000001, 2024-09-14T06:16:27.890 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0097415190000000006, 2024-09-14T06:16:27.890 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.890 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.890 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.890 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.891 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.891 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.891 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 41 2024-09-14T06:16:27.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.891 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.891 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.891 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.891 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.774917+0000", 2024-09-14T06:16:27.891 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.891 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.891 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.891 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.774917+0000", 2024-09-14T06:16:27.891 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.892 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.892 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.892 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.774918+0000", 2024-09-14T06:16:27.892 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6400000000000001e-07 2024-09-14T06:16:27.892 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.892 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.892 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.892 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.774988+0000", 2024-09-14T06:16:27.892 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0104999999999994e-05 2024-09-14T06:16:27.892 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.892 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.892 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.892 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.774999+0000", 2024-09-14T06:16:27.892 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0794e-05 2024-09-14T06:16:27.893 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.893 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.893 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.893 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775006+0000", 2024-09-14T06:16:27.893 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6149999999999999e-06 2024-09-14T06:16:27.893 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.893 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.893 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.893 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776063+0000", 2024-09-14T06:16:27.893 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0010569049999999999 2024-09-14T06:16:27.893 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.893 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.893 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776160+0000", 2024-09-14T06:16:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6707999999999996e-05 2024-09-14T06:16:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776311+0000", 2024-09-14T06:16:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000151447 2024-09-14T06:16:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.777904+0000", 2024-09-14T06:16:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001592287 2024-09-14T06:16:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784597+0000", 2024-09-14T06:16:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0066931999999999998 2024-09-14T06:16:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784632+0000", 2024-09-14T06:16:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5524000000000001e-05 2024-09-14T06:16:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784658+0000", 2024-09-14T06:16:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6169999999999998e-05 2024-09-14T06:16:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.775024+0000", 2024-09-14T06:16:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.101636699, 2024-09-14T06:16:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0097879990000000004, 2024-09-14T06:16:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 42 2024-09-14T06:16:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775024+0000", 2024-09-14T06:16:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775024+0000", 2024-09-14T06:16:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775024+0000", 2024-09-14T06:16:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4499999999999996e-07 2024-09-14T06:16:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775030+0000", 2024-09-14T06:16:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4190000000000001e-06 2024-09-14T06:16:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775038+0000", 2024-09-14T06:16:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5869999999999993e-06 2024-09-14T06:16:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775045+0000", 2024-09-14T06:16:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2129999999999996e-06 2024-09-14T06:16:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776851+0000", 2024-09-14T06:16:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001806156 2024-09-14T06:16:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776901+0000", 2024-09-14T06:16:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0386999999999997e-05 2024-09-14T06:16:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.777032+0000", 2024-09-14T06:16:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013111500000000001 2024-09-14T06:16:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.780022+0000", 2024-09-14T06:16:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029898500000000001 2024-09-14T06:16:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784709+0000", 2024-09-14T06:16:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046872830000000004 2024-09-14T06:16:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784768+0000", 2024-09-14T06:16:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8541000000000001e-05 2024-09-14T06:16:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784812+0000", 2024-09-14T06:16:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3603000000000002e-05 2024-09-14T06:16:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.775688+0000", 2024-09-14T06:16:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.100972168999999, 2024-09-14T06:16:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0093361539999999993, 2024-09-14T06:16:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 44 2024-09-14T06:16:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775688+0000", 2024-09-14T06:16:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775688+0000", 2024-09-14T06:16:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775689+0000", 2024-09-14T06:16:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5600000000000005e-07 2024-09-14T06:16:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775743+0000", 2024-09-14T06:16:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3687999999999998e-05 2024-09-14T06:16:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775752+0000", 2024-09-14T06:16:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0699999999999996e-06 2024-09-14T06:16:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775759+0000", 2024-09-14T06:16:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2099999999999996e-06 2024-09-14T06:16:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.777956+0000", 2024-09-14T06:16:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0021972390000000001 2024-09-14T06:16:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.778013+0000", 2024-09-14T06:16:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6718999999999997e-05 2024-09-14T06:16:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.778103+0000", 2024-09-14T06:16:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0085000000000002e-05 2024-09-14T06:16:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781093+0000", 2024-09-14T06:16:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029905420000000001 2024-09-14T06:16:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784949+0000", 2024-09-14T06:16:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003855448 2024-09-14T06:16:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784983+0000", 2024-09-14T06:16:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4350999999999997e-05 2024-09-14T06:16:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.785024+0000", 2024-09-14T06:16:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1046e-05 2024-09-14T06:16:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.775774+0000", 2024-09-14T06:16:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.100886320000001, 2024-09-14T06:16:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011041741000000001, 2024-09-14T06:16:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 45 2024-09-14T06:16:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775774+0000", 2024-09-14T06:16:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775774+0000", 2024-09-14T06:16:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775774+0000", 2024-09-14T06:16:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1900000000000003e-07 2024-09-14T06:16:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775778+0000", 2024-09-14T06:16:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6270000000000002e-06 2024-09-14T06:16:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775784+0000", 2024-09-14T06:16:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5559999999999998e-06 2024-09-14T06:16:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775788+0000", 2024-09-14T06:16:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9099999999999998e-06 2024-09-14T06:16:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.778575+0000", 2024-09-14T06:16:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027878040000000001 2024-09-14T06:16:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.778626+0000", 2024-09-14T06:16:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0791000000000002e-05 2024-09-14T06:16:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.778716+0000", 2024-09-14T06:16:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9466e-05 2024-09-14T06:16:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781098+0000", 2024-09-14T06:16:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023824900000000001 2024-09-14T06:16:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.786764+0000", 2024-09-14T06:16:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056663199999999999 2024-09-14T06:16:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.786793+0000", 2024-09-14T06:16:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9008000000000001e-05 2024-09-14T06:16:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.786816+0000", 2024-09-14T06:16:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2249999999999999e-05 2024-09-14T06:16:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.775864+0000", 2024-09-14T06:16:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.100796168999999, 2024-09-14T06:16:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011089144, 2024-09-14T06:16:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 46 2024-09-14T06:16:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775864+0000", 2024-09-14T06:16:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775864+0000", 2024-09-14T06:16:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775865+0000", 2024-09-14T06:16:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.111e-06 2024-09-14T06:16:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775872+0000", 2024-09-14T06:16:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9240000000000001e-06 2024-09-14T06:16:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775884+0000", 2024-09-14T06:16:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1409e-05 2024-09-14T06:16:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775890+0000", 2024-09-14T06:16:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9900000000000002e-06 2024-09-14T06:16:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.779024+0000", 2024-09-14T06:16:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031343120000000002 2024-09-14T06:16:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.779073+0000", 2024-09-14T06:16:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8856000000000001e-05 2024-09-14T06:16:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.779152+0000", 2024-09-14T06:16:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9308999999999996e-05 2024-09-14T06:16:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781102+0000", 2024-09-14T06:16:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001950139 2024-09-14T06:16:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.786906+0000", 2024-09-14T06:16:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0058036299999999997 2024-09-14T06:16:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.786933+0000", 2024-09-14T06:16:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7226999999999999e-05 2024-09-14T06:16:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.786953+0000", 2024-09-14T06:16:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0237e-05 2024-09-14T06:16:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.775904+0000", 2024-09-14T06:16:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.100756385, 2024-09-14T06:16:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011120666, 2024-09-14T06:16:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 47 2024-09-14T06:16:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775904+0000", 2024-09-14T06:16:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775904+0000", 2024-09-14T06:16:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775904+0000", 2024-09-14T06:16:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3499999999999996e-07 2024-09-14T06:16:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775965+0000", 2024-09-14T06:16:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0340999999999997e-05 2024-09-14T06:16:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775978+0000", 2024-09-14T06:16:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2724999999999999e-05 2024-09-14T06:16:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775986+0000", 2024-09-14T06:16:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.157e-06 2024-09-14T06:16:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.779449+0000", 2024-09-14T06:16:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034635310000000002 2024-09-14T06:16:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.779501+0000", 2024-09-14T06:16:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2135999999999997e-05 2024-09-14T06:16:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.779591+0000", 2024-09-14T06:16:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9520999999999999e-05 2024-09-14T06:16:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.782131+0000", 2024-09-14T06:16:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0025397900000000001 2024-09-14T06:16:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.786979+0000", 2024-09-14T06:16:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048479939999999996 2024-09-14T06:16:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787005+0000", 2024-09-14T06:16:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6224000000000002e-05 2024-09-14T06:16:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787025+0000", 2024-09-14T06:16:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9712e-05 2024-09-14T06:16:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.776003+0000", 2024-09-14T06:16:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.100657297000001, 2024-09-14T06:16:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011128235, 2024-09-14T06:16:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48 2024-09-14T06:16:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776003+0000", 2024-09-14T06:16:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776003+0000", 2024-09-14T06:16:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776004+0000", 2024-09-14T06:16:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6400000000000002e-07 2024-09-14T06:16:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776009+0000", 2024-09-14T06:16:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3299999999999998e-06 2024-09-14T06:16:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776019+0000", 2024-09-14T06:16:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0334e-05 2024-09-14T06:16:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776026+0000", 2024-09-14T06:16:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.888e-06 2024-09-14T06:16:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.780040+0000", 2024-09-14T06:16:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040138769999999999 2024-09-14T06:16:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.780109+0000", 2024-09-14T06:16:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8901999999999999e-05 2024-09-14T06:16:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.780225+0000", 2024-09-14T06:16:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000116154 2024-09-14T06:16:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.782137+0000", 2024-09-14T06:16:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0019123230000000001 2024-09-14T06:16:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787082+0000", 2024-09-14T06:16:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0049447049999999998 2024-09-14T06:16:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787111+0000", 2024-09-14T06:16:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9241e-05 2024-09-14T06:16:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787131+0000", 2024-09-14T06:16:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9817e-05 2024-09-14T06:16:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.776059+0000", 2024-09-14T06:16:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.100601263000001, 2024-09-14T06:16:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011196868, 2024-09-14T06:16:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 49 2024-09-14T06:16:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776059+0000", 2024-09-14T06:16:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776059+0000", 2024-09-14T06:16:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776060+0000", 2024-09-14T06:16:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9500000000000001e-07 2024-09-14T06:16:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776068+0000", 2024-09-14T06:16:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0059999999999992e-06 2024-09-14T06:16:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776079+0000", 2024-09-14T06:16:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0929e-05 2024-09-14T06:16:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776086+0000", 2024-09-14T06:16:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9360000000000002e-06 2024-09-14T06:16:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.780652+0000", 2024-09-14T06:16:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045665189999999998 2024-09-14T06:16:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.780716+0000", 2024-09-14T06:16:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3993999999999998e-05 2024-09-14T06:16:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.780811+0000", 2024-09-14T06:16:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4555000000000005e-05 2024-09-14T06:16:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.783904+0000", 2024-09-14T06:16:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030928459999999998 2024-09-14T06:16:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787205+0000", 2024-09-14T06:16:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033015560000000002 2024-09-14T06:16:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787234+0000", 2024-09-14T06:16:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8856999999999998e-05 2024-09-14T06:16:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787256+0000", 2024-09-14T06:16:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1875e-05 2024-09-14T06:16:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.776122+0000", 2024-09-14T06:16:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.100538505999999, 2024-09-14T06:16:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011260246, 2024-09-14T06:16:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50 2024-09-14T06:16:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776122+0000", 2024-09-14T06:16:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776122+0000", 2024-09-14T06:16:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776123+0000", 2024-09-14T06:16:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7900000000000007e-07 2024-09-14T06:16:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776192+0000", 2024-09-14T06:16:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9509000000000001e-05 2024-09-14T06:16:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776207+0000", 2024-09-14T06:16:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5167000000000001e-05 2024-09-14T06:16:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776215+0000", 2024-09-14T06:16:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9549999999999992e-06 2024-09-14T06:16:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781137+0000", 2024-09-14T06:16:27.942 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0049216099999999999 2024-09-14T06:16:27.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.942 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.942 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.942 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781185+0000", 2024-09-14T06:16:27.942 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7763000000000003e-05 2024-09-14T06:16:27.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.942 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.942 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.942 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781270+0000", 2024-09-14T06:16:27.942 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5402000000000007e-05 2024-09-14T06:16:27.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.942 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.942 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.943 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784487+0000", 2024-09-14T06:16:27.943 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032170689999999999 2024-09-14T06:16:27.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.943 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.943 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.943 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787302+0000", 2024-09-14T06:16:27.943 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028146429999999999 2024-09-14T06:16:27.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.943 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.943 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.943 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787339+0000", 2024-09-14T06:16:27.943 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6958000000000001e-05 2024-09-14T06:16:27.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.944 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.944 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.944 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787382+0000", 2024-09-14T06:16:27.944 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3191e-05 2024-09-14T06:16:27.944 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.944 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.944 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.944 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.944 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.944 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.776232+0000", 2024-09-14T06:16:27.944 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.100427946, 2024-09-14T06:16:27.944 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011758912999999999, 2024-09-14T06:16:27.945 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.945 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.945 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.945 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.945 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.945 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.945 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 51 2024-09-14T06:16:27.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.945 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.945 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.945 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.945 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776232+0000", 2024-09-14T06:16:27.945 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.946 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.946 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.946 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776232+0000", 2024-09-14T06:16:27.946 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.946 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.946 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.946 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776233+0000", 2024-09-14T06:16:27.946 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8000000000000005e-07 2024-09-14T06:16:27.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.946 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.946 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.946 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776239+0000", 2024-09-14T06:16:27.947 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.553e-06 2024-09-14T06:16:27.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.947 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.947 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.947 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776247+0000", 2024-09-14T06:16:27.947 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:16:27.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.947 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.947 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.947 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776254+0000", 2024-09-14T06:16:27.947 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2980000000000003e-06 2024-09-14T06:16:27.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.947 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.947 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.947 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781687+0000", 2024-09-14T06:16:27.948 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0054321350000000003 2024-09-14T06:16:27.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.948 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.948 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.948 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781739+0000", 2024-09-14T06:16:27.948 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2719999999999997e-05 2024-09-14T06:16:27.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.948 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.948 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.948 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781851+0000", 2024-09-14T06:16:27.948 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011188600000000001 2024-09-14T06:16:27.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.948 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.948 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.948 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784533+0000", 2024-09-14T06:16:27.949 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026823110000000002 2024-09-14T06:16:27.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.949 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.949 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787940+0000", 2024-09-14T06:16:27.949 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034064770000000002 2024-09-14T06:16:27.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.949 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.949 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787970+0000", 2024-09-14T06:16:27.949 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9626999999999999e-05 2024-09-14T06:16:27.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.949 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.949 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787991+0000", 2024-09-14T06:16:27.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1662e-05 2024-09-14T06:16:27.950 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.950 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.950 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.950 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.950 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.950 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.776268+0000", 2024-09-14T06:16:27.950 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.100392681999999, 2024-09-14T06:16:27.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.012268451, 2024-09-14T06:16:27.950 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.950 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.950 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.950 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.951 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.951 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.951 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 52 2024-09-14T06:16:27.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.951 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.951 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.951 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.951 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776268+0000", 2024-09-14T06:16:27.951 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.951 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.951 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.951 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776268+0000", 2024-09-14T06:16:27.951 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.952 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.952 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.952 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776268+0000", 2024-09-14T06:16:27.952 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4700000000000001e-07 2024-09-14T06:16:27.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.952 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.952 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.952 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776274+0000", 2024-09-14T06:16:27.952 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3669999999999996e-06 2024-09-14T06:16:27.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.952 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.952 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.952 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776282+0000", 2024-09-14T06:16:27.952 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2369999999999992e-06 2024-09-14T06:16:27.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.953 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.953 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.953 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776288+0000", 2024-09-14T06:16:27.953 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4450000000000001e-06 2024-09-14T06:16:27.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.953 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.953 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.953 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.782157+0000", 2024-09-14T06:16:27.953 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0058688430000000003 2024-09-14T06:16:27.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.953 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.953 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.953 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.782207+0000", 2024-09-14T06:16:27.953 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9468999999999997e-05 2024-09-14T06:16:27.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.954 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.954 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.954 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.782284+0000", 2024-09-14T06:16:27.954 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7368999999999997e-05 2024-09-14T06:16:27.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.954 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.954 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.954 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784550+0000", 2024-09-14T06:16:27.954 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022660969999999999 2024-09-14T06:16:27.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.955 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.955 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.788458+0000", 2024-09-14T06:16:27.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0039080290000000004 2024-09-14T06:16:27.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.955 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.955 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.788495+0000", 2024-09-14T06:16:27.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6468999999999999e-05 2024-09-14T06:16:27.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.955 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.955 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.788536+0000", 2024-09-14T06:16:27.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1479e-05 2024-09-14T06:16:27.955 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.955 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.956 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.956 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.956 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.956 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:05.011523+0000", 2024-09-14T06:16:27.956 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.865136947, 2024-09-14T06:16:27.956 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011115544, 2024-09-14T06:16:27.956 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.956 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.956 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.956 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.956 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.956 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.956 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 70 2024-09-14T06:16:27.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.957 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.957 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.957 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011523+0000", 2024-09-14T06:16:27.957 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.957 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.957 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011523+0000", 2024-09-14T06:16:27.957 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.957 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.958 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011525+0000", 2024-09-14T06:16:27.958 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5239999999999999e-06 2024-09-14T06:16:27.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.958 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.958 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011533+0000", 2024-09-14T06:16:27.958 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4500000000000004e-06 2024-09-14T06:16:27.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.958 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.958 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011556+0000", 2024-09-14T06:16:27.958 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2858999999999999e-05 2024-09-14T06:16:27.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.959 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.959 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011573+0000", 2024-09-14T06:16:27.959 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7283e-05 2024-09-14T06:16:27.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.959 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.959 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.959 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.016587+0000", 2024-09-14T06:16:27.959 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050139349999999997 2024-09-14T06:16:27.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.959 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.959 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.959 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.016730+0000", 2024-09-14T06:16:27.959 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00014216200000000001 2024-09-14T06:16:27.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.960 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.960 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.016956+0000", 2024-09-14T06:16:27.960 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022678499999999999 2024-09-14T06:16:27.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.960 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.960 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.018945+0000", 2024-09-14T06:16:27.960 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00198832 2024-09-14T06:16:27.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.960 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.960 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.022306+0000", 2024-09-14T06:16:27.961 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033608420000000002 2024-09-14T06:16:27.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.961 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.961 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.961 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.022506+0000", 2024-09-14T06:16:27.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00019999899999999999 2024-09-14T06:16:27.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.964 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.964 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.022639+0000", 2024-09-14T06:16:27.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000133385 2024-09-14T06:16:27.964 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.964 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.964 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.965 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.965 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:05.011599+0000", 2024-09-14T06:16:27.965 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.865061518000001, 2024-09-14T06:16:27.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011527848, 2024-09-14T06:16:27.965 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.965 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.965 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.965 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.965 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.965 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.965 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 71 2024-09-14T06:16:27.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.966 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.966 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.966 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.966 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011599+0000", 2024-09-14T06:16:27.966 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.966 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.966 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.966 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011599+0000", 2024-09-14T06:16:27.966 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.966 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.967 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.967 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011600+0000", 2024-09-14T06:16:27.967 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.08e-06 2024-09-14T06:16:27.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.967 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.967 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011609+0000", 2024-09-14T06:16:27.967 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0399999999999998e-06 2024-09-14T06:16:27.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.967 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.967 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011624+0000", 2024-09-14T06:16:27.967 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4970000000000001e-05 2024-09-14T06:16:27.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.968 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.968 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011637+0000", 2024-09-14T06:16:27.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2724e-05 2024-09-14T06:16:27.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.968 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.968 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.017661+0000", 2024-09-14T06:16:27.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0060242919999999997 2024-09-14T06:16:27.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.968 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.968 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.017768+0000", 2024-09-14T06:16:27.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010715199999999999 2024-09-14T06:16:27.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.017953+0000", 2024-09-14T06:16:27.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00018450199999999999 2024-09-14T06:16:27.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.022730+0000", 2024-09-14T06:16:27.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047772839999999997 2024-09-14T06:16:27.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.022910+0000", 2024-09-14T06:16:27.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000180213 2024-09-14T06:16:27.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.970 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.970 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.022994+0000", 2024-09-14T06:16:27.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3518999999999998e-05 2024-09-14T06:16:27.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.970 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.970 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.023127+0000", 2024-09-14T06:16:27.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000133072 2024-09-14T06:16:27.970 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.970 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.970 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.971 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.971 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:05.011653+0000", 2024-09-14T06:16:27.971 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.865007548000001, 2024-09-14T06:16:27.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011747893000000001, 2024-09-14T06:16:27.971 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.971 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.971 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.971 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.971 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.971 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.971 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 72 2024-09-14T06:16:27.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.971 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011653+0000", 2024-09-14T06:16:27.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011653+0000", 2024-09-14T06:16:27.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011654+0000", 2024-09-14T06:16:27.973 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2399999999999997e-07 2024-09-14T06:16:27.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.973 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.973 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011842+0000", 2024-09-14T06:16:27.973 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000188841 2024-09-14T06:16:27.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.973 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.973 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011867+0000", 2024-09-14T06:16:27.973 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4122999999999999e-05 2024-09-14T06:16:27.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.973 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.973 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011880+0000", 2024-09-14T06:16:27.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.376e-05 2024-09-14T06:16:27.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.019044+0000", 2024-09-14T06:16:27.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0071632980000000002 2024-09-14T06:16:27.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.019237+0000", 2024-09-14T06:16:27.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000193792 2024-09-14T06:16:27.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.019565+0000", 2024-09-14T06:16:27.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032795399999999998 2024-09-14T06:16:27.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.022877+0000", 2024-09-14T06:16:27.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003311731 2024-09-14T06:16:27.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.023151+0000", 2024-09-14T06:16:27.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027372799999999999 2024-09-14T06:16:27.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.023316+0000", 2024-09-14T06:16:27.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000165353 2024-09-14T06:16:27.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.023401+0000", 2024-09-14T06:16:27.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4488999999999997e-05 2024-09-14T06:16:27.976 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.976 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.976 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.976 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.976 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:05.011934+0000", 2024-09-14T06:16:27.977 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.864726727000001, 2024-09-14T06:16:27.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013924748000000001, 2024-09-14T06:16:27.977 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.977 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.977 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.977 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.977 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.977 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.977 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 73 2024-09-14T06:16:27.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.977 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.977 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.977 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.977 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011934+0000", 2024-09-14T06:16:27.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011934+0000", 2024-09-14T06:16:27.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011935+0000", 2024-09-14T06:16:27.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.164e-06 2024-09-14T06:16:27.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.979 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011942+0000", 2024-09-14T06:16:27.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9630000000000001e-06 2024-09-14T06:16:27.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.979 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.979 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011960+0000", 2024-09-14T06:16:27.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8272999999999999e-05 2024-09-14T06:16:27.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.979 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.979 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011980+0000", 2024-09-14T06:16:27.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.959e-05 2024-09-14T06:16:27.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.020767+0000", 2024-09-14T06:16:27.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0087871249999999998 2024-09-14T06:16:27.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.021037+0000", 2024-09-14T06:16:27.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027065800000000001 2024-09-14T06:16:27.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.021412+0000", 2024-09-14T06:16:27.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00037428400000000001 2024-09-14T06:16:27.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.023453+0000", 2024-09-14T06:16:27.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002041181 2024-09-14T06:16:27.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025788+0000", 2024-09-14T06:16:27.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023351819999999999 2024-09-14T06:16:27.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025829+0000", 2024-09-14T06:16:27.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0951000000000001e-05 2024-09-14T06:16:27.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025858+0000", 2024-09-14T06:16:27.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9377e-05 2024-09-14T06:16:27.982 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.982 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.982 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.982 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.982 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:05.011997+0000", 2024-09-14T06:16:27.983 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.864662877000001, 2024-09-14T06:16:27.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013974349, 2024-09-14T06:16:27.983 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.983 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.983 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.983 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.983 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.983 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.983 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 74 2024-09-14T06:16:27.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.983 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.983 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.983 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011997+0000", 2024-09-14T06:16:27.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.984 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011997+0000", 2024-09-14T06:16:27.984 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.984 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011998+0000", 2024-09-14T06:16:27.984 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9299999999999996e-07 2024-09-14T06:16:27.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.984 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012004+0000", 2024-09-14T06:16:27.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.037e-06 2024-09-14T06:16:27.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012016+0000", 2024-09-14T06:16:27.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2055999999999999e-05 2024-09-14T06:16:27.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012029+0000", 2024-09-14T06:16:27.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2787e-05 2024-09-14T06:16:27.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.023591+0000", 2024-09-14T06:16:27.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011562004000000001 2024-09-14T06:16:27.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.023732+0000", 2024-09-14T06:16:27.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000140679 2024-09-14T06:16:27.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.024001+0000", 2024-09-14T06:16:27.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00026883499999999998 2024-09-14T06:16:27.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025870+0000", 2024-09-14T06:16:27.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018691890000000001 2024-09-14T06:16:27.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025905+0000", 2024-09-14T06:16:27.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4805000000000003e-05 2024-09-14T06:16:27.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025941+0000", 2024-09-14T06:16:27.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6594000000000002e-05 2024-09-14T06:16:27.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025972+0000", 2024-09-14T06:16:27.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0470000000000001e-05 2024-09-14T06:16:27.988 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.988 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.988 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.988 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.988 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:05.012059+0000", 2024-09-14T06:16:27.988 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.864601161, 2024-09-14T06:16:27.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014493252, 2024-09-14T06:16:27.988 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.988 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.988 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.989 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.989 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.989 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.989 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 75 2024-09-14T06:16:27.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.989 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012059+0000", 2024-09-14T06:16:27.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012059+0000", 2024-09-14T06:16:27.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012060+0000", 2024-09-14T06:16:27.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.47e-07 2024-09-14T06:16:27.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012185+0000", 2024-09-14T06:16:27.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000125165 2024-09-14T06:16:27.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012209+0000", 2024-09-14T06:16:27.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3753000000000001e-05 2024-09-14T06:16:27.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012226+0000", 2024-09-14T06:16:27.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.664e-05 2024-09-14T06:16:27.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.024653+0000", 2024-09-14T06:16:27.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.012427493 2024-09-14T06:16:27.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.024714+0000", 2024-09-14T06:16:27.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0585999999999999e-05 2024-09-14T06:16:27.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.024803+0000", 2024-09-14T06:16:27.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9881000000000002e-05 2024-09-14T06:16:27.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.026474+0000", 2024-09-14T06:16:27.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001670405 2024-09-14T06:16:27.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.026490+0000", 2024-09-14T06:16:27.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6421000000000001e-05 2024-09-14T06:16:27.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.026526+0000", 2024-09-14T06:16:27.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5651000000000001e-05 2024-09-14T06:16:27.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.026552+0000", 2024-09-14T06:16:27.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.641e-05 2024-09-14T06:16:27.993 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.994 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.994 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.994 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:27.994 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:05.012255+0000", 2024-09-14T06:16:27.994 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.864405169000001, 2024-09-14T06:16:27.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015138888, 2024-09-14T06:16:27.994 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:27.994 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:27.994 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:27.994 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:27.994 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:27.994 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:27.994 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 76 2024-09-14T06:16:27.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.995 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:27.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:27.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012255+0000", 2024-09-14T06:16:27.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:27.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012255+0000", 2024-09-14T06:16:27.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:27.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:27.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012256+0000", 2024-09-14T06:16:27.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0899999999999999e-06 2024-09-14T06:16:27.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:27.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012263+0000", 2024-09-14T06:16:27.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2030000000000003e-06 2024-09-14T06:16:27.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:27.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012275+0000", 2024-09-14T06:16:27.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1883e-05 2024-09-14T06:16:27.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:27.997 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012284+0000", 2024-09-14T06:16:27.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8000000000000004e-06 2024-09-14T06:16:27.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.997 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:27.997 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025348+0000", 2024-09-14T06:16:27.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013064157 2024-09-14T06:16:27.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.997 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:27.997 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025415+0000", 2024-09-14T06:16:27.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7152999999999999e-05 2024-09-14T06:16:27.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:27.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025519+0000", 2024-09-14T06:16:27.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000104083 2024-09-14T06:16:27.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:27.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.026870+0000", 2024-09-14T06:16:27.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0013508089999999999 2024-09-14T06:16:27.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:27.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.027336+0000", 2024-09-14T06:16:27.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00046525199999999998 2024-09-14T06:16:27.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:27.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.027370+0000", 2024-09-14T06:16:27.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4128999999999998e-05 2024-09-14T06:16:27.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:27.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:27.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:27.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.027394+0000", 2024-09-14T06:16:27.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4329e-05 2024-09-14T06:16:27.999 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.999 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.999 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.999 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:27.999 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:27.999 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:28.000 INFO:teuthology.orchestra.run.smithi039.stdout:366 read 0x19edecc thru 0x19f1c6c (0x3da1 bytes) 2024-09-14T06:16:28.001 INFO:teuthology.orchestra.run.smithi039.stdout:368 trunc from 0xed126b0 to 0xac9859c 2024-09-14T06:16:28.002 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:16:28.002 INFO:teuthology.orchestra.run.smithi039.stdout:370 writesame 0x7e23e47 thru 0x7e314f3 data_size 0x7(0xd6ad bytes) 2024-09-14T06:16:28.002 INFO:teuthology.orchestra.run.smithi039.stdout:371 trunc from 0xac9859c to 0x166a34d 2024-09-14T06:16:28.003 INFO:teuthology.orchestra.run.smithi039.stdout:372 read 0xef609a thru 0xef73b5 (0x131c bytes) 2024-09-14T06:16:28.003 INFO:teuthology.orchestra.run.smithi039.stdout:374 write 0x5c3d2ae thru 0x5c42288 (0x4fdb bytes) 2024-09-14T06:16:28.003 INFO:teuthology.orchestra.run.smithi039.stdout:375 writesame 0x4452916 thru 0x4453ccd data_size 0x4(0x13b8 bytes) 2024-09-14T06:16:28.003 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:28.003 INFO:teuthology.orchestra.run.smithi039.stdout:378 writesame 0x25d626f thru 0x25e3343 data_size 0x1(0xd0d5 bytes) 2024-09-14T06:16:28.003 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:27.996+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:28.030 INFO:teuthology.orchestra.run.smithi039.stdout:380 trunc from 0x5c42289 to 0x4fa5d82 2024-09-14T06:16:28.056 INFO:teuthology.orchestra.run.smithi039.stdout:381 writesame 0x79ae16a thru 0x79b4473 data_size 0x7(0x630a bytes) 2024-09-14T06:16:28.073 INFO:teuthology.orchestra.run.smithi039.stdout:382 write 0xa1cc754 thru 0xa1d5c49 (0x94f6 bytes) 2024-09-14T06:16:28.096 INFO:teuthology.orchestra.run.smithi039.stdout:383 write 0xd09c56e thru 0xd0a81df (0xbc72 bytes) 2024-09-14T06:16:28.102 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:28.100+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:28.104 INFO:teuthology.orchestra.run.smithi039.stdout:384 write 0x6cc9f44 thru 0x6cd0e38 (0x6ef5 bytes) 2024-09-14T06:16:28.130 INFO:teuthology.orchestra.run.smithi039.stdout:385 trunc from 0xd0a81e0 to 0x3644985 2024-09-14T06:16:28.156 INFO:teuthology.orchestra.run.smithi039.stdout:386 punch from 0x8868a2 to 0x88c28c, (0x59ea bytes) 2024-09-14T06:16:28.168 INFO:teuthology.orchestra.run.smithi039.stdout:389 trunc from 0x3644985 to 0x9b9fdfd 2024-09-14T06:16:28.173 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:28.173 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:28.173 INFO:teuthology.orchestra.run.smithi039.stdout:392 write 0x4aa402e thru 0x4aaccb9 (0x8c8c bytes) 2024-09-14T06:16:28.181 INFO:teuthology.orchestra.run.smithi039.stdout:393 write 0x289cefe thru 0x28a2890 (0x5993 bytes) 2024-09-14T06:16:28.187 INFO:teuthology.orchestra.run.smithi039.stdout:395 read 0x650ffe9 thru 0x651f8ce (0xf8e6 bytes) 2024-09-14T06:16:28.187 INFO:teuthology.orchestra.run.smithi039.stdout:396 writesame 0x7c87b92 thru 0x7c8c5eb data_size 0x2(0x4a5a bytes) 2024-09-14T06:16:28.197 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:28.197 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:28.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.197 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4873 3.e 3:76656e66:::rbd_data.1152bd92e04f.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=2b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:28.197 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:28.197 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0044310299999999999, 2024-09-14T06:16:28.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044828730000000001, 2024-09-14T06:16:28.197 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:28.197 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:28.198 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "waiting for sub ops", 2024-09-14T06:16:28.198 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:28.198 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:28.198 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:28.198 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4873 2024-09-14T06:16:28.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:28.198 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:28.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:28.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:28.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:28.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:28.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:28.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:28.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:28.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:28.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.199 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:28.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193651+0000", 2024-09-14T06:16:28.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:16:28.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:28.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.199 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:28.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193663+0000", 2024-09-14T06:16:28.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2047e-05 2024-09-14T06:16:28.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:28.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:28.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193692+0000", 2024-09-14T06:16:28.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9346e-05 2024-09-14T06:16:28.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:28.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:28.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193708+0000", 2024-09-14T06:16:28.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6123000000000001e-05 2024-09-14T06:16:28.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:28.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:28.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193798+0000", 2024-09-14T06:16:28.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9909999999999998e-05 2024-09-14T06:16:28.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:28.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:28.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193863+0000", 2024-09-14T06:16:28.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4583000000000002e-05 2024-09-14T06:16:28.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:28.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:28.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.194717+0000", 2024-09-14T06:16:28.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00085473299999999999 2024-09-14T06:16:28.201 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:28.201 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:28.201 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:28.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:28.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.202 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:4873 3.e e30/29)", 2024-09-14T06:16:28.202 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:28.197511+0000", 2024-09-14T06:16:28.202 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.000567991, 2024-09-14T06:16:28.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00074489300000000005, 2024-09-14T06:16:28.202 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:28.202 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:28.202 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "queued for pg", 2024-09-14T06:16:28.202 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:28.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:28.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.197511+0000", 2024-09-14T06:16:28.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:28.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:28.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:28.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.197511+0000", 2024-09-14T06:16:28.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:28.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:28.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:28.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.197512+0000", 2024-09-14T06:16:28.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2389999999999999e-06 2024-09-14T06:16:28.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:28.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:28.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.197518+0000", 2024-09-14T06:16:28.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7300000000000002e-06 2024-09-14T06:16:28.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:28.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:28.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.197535+0000", 2024-09-14T06:16:28.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7147999999999999e-05 2024-09-14T06:16:28.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:28.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:28.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:28.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.197545+0000", 2024-09-14T06:16:28.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0307e-05 2024-09-14T06:16:28.204 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:28.204 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:28.204 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:28.204 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:28.204 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:16:28.204 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 2 2024-09-14T06:16:28.204 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:28.205 INFO:teuthology.orchestra.run.smithi039.stdout:397 write 0x5c7e656 thru 0x5c88e45 (0xa7f0 bytes) 2024-09-14T06:16:28.205 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:28.205 INFO:teuthology.orchestra.run.smithi039.stdout:400 write 0x780094d thru 0x7805805 (0x4eb9 bytes) 2024-09-14T06:16:28.206 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:28.204+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:28.208 INFO:teuthology.orchestra.run.smithi039.stdout:401 write 0x80b09bb thru 0x80b6c19 (0x625f bytes) 2024-09-14T06:16:28.215 INFO:teuthology.orchestra.run.smithi039.stdout:402 write 0x5d994eb thru 0x5da7c4a (0xe760 bytes) 2024-09-14T06:16:28.216 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:16:28.219 INFO:teuthology.orchestra.run.smithi039.stdout:404 read 0x3689761 thru 0x36923d7 (0x8c77 bytes) 2024-09-14T06:16:28.219 INFO:teuthology.orchestra.run.smithi039.stdout:405 read 0x681223 thru 0x684824 (0x3602 bytes) 2024-09-14T06:16:28.221 INFO:teuthology.orchestra.run.smithi039.stdout:406 write 0x70137de thru 0x701eab5 (0xb2d8 bytes) 2024-09-14T06:16:28.228 INFO:teuthology.orchestra.run.smithi039.stdout:407 read 0x46a1b42 thru 0x46a359f (0x1a5e bytes) 2024-09-14T06:16:28.228 INFO:teuthology.orchestra.run.smithi039.stdout:408 read 0x7c33c1a thru 0x7c429a8 (0xed8f bytes) 2024-09-14T06:16:28.228 INFO:teuthology.orchestra.run.smithi039.stdout:409 punch from 0x77784cd to 0x7781410, (0x8f43 bytes) 2024-09-14T06:16:28.229 INFO:teuthology.orchestra.run.smithi039.stdout:410 read 0x2030903 thru 0x203db5e (0xd25c bytes) 2024-09-14T06:16:28.235 INFO:teuthology.orchestra.run.smithi039.stdout:411 write 0x9bcdd8e thru 0x9bcec72 (0xee5 bytes) 2024-09-14T06:16:28.240 INFO:teuthology.orchestra.run.smithi039.stdout:413 writesame 0x42b0d90 thru 0x42bb911 data_size 0x1d(0xab82 bytes) 2024-09-14T06:16:28.250 INFO:teuthology.orchestra.run.smithi039.stdout:414 write 0x89cc956 thru 0x89dc89e (0xff49 bytes) 2024-09-14T06:16:28.254 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:16:28.255 INFO:teuthology.orchestra.run.smithi039.stdout:415 writesame 0x543d890 thru 0x544121b data_size 0x3a(0x398c bytes) 2024-09-14T06:16:28.259 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:16:28.259 INFO:teuthology.orchestra.run.smithi039.stdout:418 write 0x8eb67bd thru 0x8ec370e (0xcf52 bytes) 2024-09-14T06:16:28.262 INFO:teuthology.orchestra.run.smithi039.stdout:420 writesame 0x43d77c5 thru 0x43e5d88 data_size 0x55(0xe5c4 bytes) 2024-09-14T06:16:28.272 INFO:teuthology.orchestra.run.smithi039.stdout:425 read 0x78ecece thru 0x78f2b87 (0x5cba bytes) 2024-09-14T06:16:28.273 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:28.273 INFO:teuthology.orchestra.run.smithi039.stdout:427 read 0x6c74d4c thru 0x6c7e16b (0x9420 bytes) 2024-09-14T06:16:28.273 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:28.273 INFO:teuthology.orchestra.run.smithi039.stdout:429 trunc from 0x9bcec73 to 0xa1b0511 2024-09-14T06:16:28.292 INFO:teuthology.orchestra.run.smithi039.stdout:430 trunc from 0xa1b0511 to 0x2441143 2024-09-14T06:16:28.307 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:28.304+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:28.324 INFO:teuthology.orchestra.run.smithi039.stdout:432 read 0x1b22d39 thru 0x1b30a3e (0xdd06 bytes) 2024-09-14T06:16:28.342 INFO:teuthology.orchestra.run.smithi039.stdout:434 trunc from 0x2441143 to 0xe0c8464 2024-09-14T06:16:28.352 INFO:teuthology.orchestra.run.smithi039.stdout:435 read 0x851fe67 thru 0x8528e38 (0x8fd2 bytes) 2024-09-14T06:16:28.352 INFO:teuthology.orchestra.run.smithi039.stdout:436 writesame 0xde5753b thru 0xde5cc19 data_size 0x15(0x56df bytes) 2024-09-14T06:16:28.357 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:28.357 INFO:teuthology.orchestra.run.smithi039.stdout:439 write 0x9664d4e thru 0x966f8c6 (0xab79 bytes) 2024-09-14T06:16:28.365 INFO:teuthology.orchestra.run.smithi039.stdout:440 punch from 0xc1c09c9 to 0xc1c8621, (0x7c58 bytes) 2024-09-14T06:16:28.365 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:28.365 INFO:teuthology.orchestra.run.smithi039.stdout:442 punch from 0x738a431 to 0x7392e7b, (0x8a4a bytes) 2024-09-14T06:16:28.365 INFO:teuthology.orchestra.run.smithi039.stdout:443 read 0xbd755de thru 0xbd8112e (0xbb51 bytes) 2024-09-14T06:16:28.366 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:28.366 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:28.366 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:28.366 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:28.366 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:28.376 INFO:teuthology.orchestra.run.smithi039.stdout:444 trunc from 0xe0c8464 to 0x72ab4a5 2024-09-14T06:16:28.392 INFO:teuthology.orchestra.run.smithi039.stdout:446 punch from 0x33e5c36 to 0x33ea0eb, (0x44b5 bytes) 2024-09-14T06:16:28.392 INFO:teuthology.orchestra.run.smithi039.stdout:447 write 0x23a19af thru 0x23acd6b (0xb3bd bytes) 2024-09-14T06:16:28.393 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:16:28.399 INFO:teuthology.orchestra.run.smithi039.stdout:448 write 0x6fafc6e thru 0x6fb88fa (0x8c8d bytes) 2024-09-14T06:16:28.421 INFO:teuthology.orchestra.run.smithi039.stdout:449 write 0xbb6f812 thru 0xbb6fa26 (0x215 bytes) 2024-09-14T06:16:28.422 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:28.420+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:28.427 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:28.427 INFO:teuthology.orchestra.run.smithi039.stdout:453 write 0xb986a72 thru 0xb98b3fb (0x498a bytes) 2024-09-14T06:16:28.435 INFO:teuthology.orchestra.run.smithi039.stdout:454 trunc from 0xbb6fa27 to 0xafa37e5 2024-09-14T06:16:28.447 INFO:teuthology.orchestra.run.smithi039.stdout:455 write 0xf2aeb3 thru 0xf2f089 (0x41d7 bytes) 2024-09-14T06:16:28.451 INFO:teuthology.orchestra.run.smithi039.stdout:457 read 0x3c4bb87 thru 0x3c4fd82 (0x41fc bytes) 2024-09-14T06:16:28.451 INFO:teuthology.orchestra.run.smithi039.stdout:458 write 0x5fa3e57 thru 0x5fa703c (0x31e6 bytes) 2024-09-14T06:16:28.461 INFO:teuthology.orchestra.run.smithi039.stdout:459 punch from 0x261e923 to 0x262a8e6, (0xbfc3 bytes) 2024-09-14T06:16:28.468 INFO:teuthology.orchestra.run.smithi039.stdout:461 write 0xb539b8b thru 0xb53c57b (0x29f1 bytes) 2024-09-14T06:16:28.469 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:28.469 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:28.476 INFO:teuthology.orchestra.run.smithi039.stdout:462 write 0x978723a thru 0x9793097 (0xbe5e bytes) 2024-09-14T06:16:28.486 INFO:teuthology.orchestra.run.smithi039.stdout:463 write 0x2aa3dfd thru 0x2ab2b05 (0xed09 bytes) 2024-09-14T06:16:28.494 INFO:teuthology.orchestra.run.smithi039.stdout:464 write 0x60e29c1 thru 0x60e6f89 (0x45c9 bytes) 2024-09-14T06:16:28.499 INFO:teuthology.orchestra.run.smithi039.stdout:465 write 0x6400fab thru 0x6408f79 (0x7fcf bytes) 2024-09-14T06:16:28.504 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:28.504 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:28.508 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:28.504+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:28.520 INFO:teuthology.orchestra.run.smithi039.stdout:466 trunc from 0xb53c57c to 0x34271cf 2024-09-14T06:16:28.522 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:28.522 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:28.547 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:28.547 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:28.548 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:28.552 INFO:teuthology.orchestra.run.smithi039.stdout:467 write 0x6a69c46 thru 0x6a79b91 (0xff4c bytes) 2024-09-14T06:16:28.563 INFO:teuthology.orchestra.run.smithi039.stdout:469 read 0x6a367d9 thru 0x6a36d43 (0x56b bytes) 2024-09-14T06:16:28.563 INFO:teuthology.orchestra.run.smithi039.stdout:470 punch from 0xe0e367 to 0xe1b692, (0xd32b bytes) 2024-09-14T06:16:28.563 INFO:teuthology.orchestra.run.smithi039.stdout:471 read 0x5173d thru 0x57bad (0x6471 bytes) 2024-09-14T06:16:28.567 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:16:28.576 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:28.577 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:28.580 INFO:teuthology.orchestra.run.smithi039.stdout:472 writesame 0xdde37aa thru 0xdde84fa data_size 0x1(0x4d51 bytes) 2024-09-14T06:16:28.580 INFO:teuthology.orchestra.run.smithi039.stdout:475 write 0x8c067ed thru 0x8c0d4fc (0x6d10 bytes) 2024-09-14T06:16:28.588 INFO:teuthology.orchestra.run.smithi039.stdout:476 punch from 0xdaa4fc6 to 0xdab378a, (0xe7c4 bytes) 2024-09-14T06:16:28.588 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:28.588 INFO:teuthology.orchestra.run.smithi039.stdout:479 writesame 0xbf3d760 thru 0xbf48c89 data_size 0x1(0xb52a bytes) 2024-09-14T06:16:28.603 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:28.603 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:28.609 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:28.604+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:28.620 INFO:teuthology.orchestra.run.smithi039.stdout:480 punch from 0x3940c63 to 0x3946c81, (0x601e bytes) 2024-09-14T06:16:28.620 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:28.620 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:28.624 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:28.624 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:28.635 INFO:teuthology.orchestra.run.smithi039.stdout:483 trunc from 0xdde84fb to 0x367a077 2024-09-14T06:16:28.648 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:28.649 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:28.652 INFO:teuthology.orchestra.run.smithi039.stdout:484 write 0x2abec57 thru 0x2ac8094 (0x943e bytes) 2024-09-14T06:16:28.657 INFO:teuthology.orchestra.run.smithi039.stdout:485 punch from 0x2778263 to 0x2781ef4, (0x9c91 bytes) 2024-09-14T06:16:28.657 INFO:teuthology.orchestra.run.smithi039.stdout:486 write 0x211b125 thru 0x211f55b (0x4437 bytes) 2024-09-14T06:16:28.668 INFO:teuthology.orchestra.run.smithi039.stdout:487 write 0x5521acb thru 0x552b5ca (0x9b00 bytes) 2024-09-14T06:16:28.675 INFO:teuthology.orchestra.run.smithi039.stdout:488 read 0x7b1d3f thru 0x7bef1a (0xd1dc bytes) 2024-09-14T06:16:28.676 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:28.694 INFO:teuthology.orchestra.run.smithi039.stdout:491 write 0xe67dda9 thru 0xe67e341 (0x599 bytes) 2024-09-14T06:16:28.710 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:28.708+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:28.715 INFO:teuthology.orchestra.run.smithi039.stdout:492 trunc from 0xe67e342 to 0x271665c 2024-09-14T06:16:28.733 INFO:teuthology.orchestra.run.smithi039.stdout:493 punch from 0x3b0a63 to 0x3be9b9, (0xdf56 bytes) 2024-09-14T06:16:28.735 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:28.736 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:28.740 INFO:teuthology.orchestra.run.smithi039.stdout:494 write 0x31f51ff thru 0x31f7014 (0x1e16 bytes) 2024-09-14T06:16:28.753 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:16:28.757 INFO:teuthology.orchestra.run.smithi039.stdout:495 trunc from 0x31f7015 to 0xba6a5aa 2024-09-14T06:16:28.762 INFO:teuthology.orchestra.run.smithi039.stdout:496 write 0x71e648 thru 0x7249fc (0x63b5 bytes) 2024-09-14T06:16:28.766 INFO:teuthology.orchestra.run.smithi039.stdout:497 write 0x6149f0d thru 0x614f941 (0x5a35 bytes) 2024-09-14T06:16:28.771 INFO:teuthology.orchestra.run.smithi039.stdout:499 write 0x21e12e4 thru 0x21ec3ed (0xb10a bytes) 2024-09-14T06:16:28.775 INFO:teuthology.orchestra.run.smithi039.stdout:500 punch from 0x3db70ac to 0x3dc0320, (0x9274 bytes) 2024-09-14T06:16:28.775 INFO:teuthology.orchestra.run.smithi039.stdout:502 write 0xaa16e1e thru 0xaa1c627 (0x580a bytes) 2024-09-14T06:16:28.788 INFO:teuthology.orchestra.run.smithi039.stdout:503 trunc from 0xba6a5aa to 0x7d21860 2024-09-14T06:16:28.802 INFO:teuthology.orchestra.run.smithi039.stdout:504 trunc from 0x7d21860 to 0x71474e5 2024-09-14T06:16:28.810 INFO:teuthology.orchestra.run.smithi039.stdout:505 write 0x2266368 thru 0x226d60a (0x72a3 bytes) 2024-09-14T06:16:28.811 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:28.808+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:28.813 INFO:teuthology.orchestra.run.smithi039.stdout:506 write 0x4a9ef89 thru 0x4a9fd68 (0xde0 bytes) 2024-09-14T06:16:28.829 INFO:teuthology.orchestra.run.smithi039.stdout:508 write 0xc99d16f thru 0xc9a45af (0x7441 bytes) 2024-09-14T06:16:28.845 INFO:teuthology.orchestra.run.smithi039.stdout:510 writesame 0xee05708 thru 0xee0faa9 data_size 0x76(0xa3a2 bytes) 2024-09-14T06:16:28.845 INFO:teuthology.orchestra.run.smithi039.stdout:512 read 0x1005185 thru 0x1005858 (0x6d4 bytes) 2024-09-14T06:16:28.846 INFO:teuthology.orchestra.run.smithi039.stdout:513 read 0x50707ad thru 0x50710cd (0x921 bytes) 2024-09-14T06:16:28.846 INFO:teuthology.orchestra.run.smithi039.stdout:515 punch from 0x6dabcb to 0x6e4dc1, (0xa1f6 bytes) 2024-09-14T06:16:28.846 INFO:teuthology.orchestra.run.smithi039.stdout:516 write 0x4bb44e8 thru 0x4bbff41 (0xba5a bytes) 2024-09-14T06:16:28.852 INFO:teuthology.orchestra.run.smithi039.stdout:517 read 0x92367fe thru 0x9242228 (0xba2b bytes) 2024-09-14T06:16:28.852 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:28.852 INFO:teuthology.orchestra.run.smithi039.stdout:520 writesame 0x670dfbd thru 0x671d775 data_size 0x1(0xf7b9 bytes) 2024-09-14T06:16:28.881 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:28.882 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:28.894 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:16:28.899 INFO:teuthology.orchestra.run.smithi039.stdout:521 write 0xc660d2e thru 0xc66db06 (0xcdd9 bytes) 2024-09-14T06:16:28.906 INFO:teuthology.orchestra.run.smithi039.stdout:522 writesame 0xd274d06 thru 0xd27c9ef data_size 0x2(0x7cea bytes) 2024-09-14T06:16:28.911 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:28.908+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:28.919 INFO:teuthology.orchestra.run.smithi039.stdout:523 trunc from 0xee0faaa to 0xac700e7 2024-09-14T06:16:28.934 INFO:teuthology.orchestra.run.smithi039.stdout:525 read 0x1010e73 thru 0x101ddc3 (0xcf51 bytes) 2024-09-14T06:16:28.935 INFO:teuthology.orchestra.run.smithi039.stdout:526 writesame 0x232627d thru 0x2329126 data_size 0x21(0x2eaa bytes) 2024-09-14T06:16:28.938 INFO:teuthology.orchestra.run.smithi039.stdout:527 writesame 0x293290f thru 0x2937a2d data_size 0x1(0x511f bytes) 2024-09-14T06:16:28.958 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:28.958 INFO:teuthology.orchestra.run.smithi039.stdout:529 punch from 0x7e6b796 to 0x7e6f078, (0x38e2 bytes) 2024-09-14T06:16:28.958 INFO:teuthology.orchestra.run.smithi039.stdout:534 read 0x8668e9d thru 0x866a3bd (0x1521 bytes) 2024-09-14T06:16:28.959 INFO:teuthology.orchestra.run.smithi039.stdout:535 read 0x7dd22cd thru 0x7dd528e (0x2fc2 bytes) 2024-09-14T06:16:28.959 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:28.959 INFO:teuthology.orchestra.run.smithi039.stdout:538 writesame 0x20dedd1 thru 0x20dedd7 data_size 0x1(0x7 bytes) 2024-09-14T06:16:28.961 INFO:teuthology.orchestra.run.smithi039.stdout:539 write 0x987bcb thru 0x988576 (0x9ac bytes) 2024-09-14T06:16:28.965 INFO:teuthology.orchestra.run.smithi039.stdout:540 writesame 0x1bfd0d6 thru 0x1bffa03 data_size 0x6(0x292e bytes) 2024-09-14T06:16:28.968 INFO:teuthology.orchestra.run.smithi039.stdout:541 writesame 0x21b577d thru 0x21b68e7 data_size 0x7(0x116b bytes) 2024-09-14T06:16:28.971 INFO:teuthology.orchestra.run.smithi039.stdout:542 writesame 0x9fb0229 thru 0x9fbae4c data_size 0x2e(0xac24 bytes) 2024-09-14T06:16:28.979 INFO:teuthology.orchestra.run.smithi039.stdout:543 clone 2 order 19 su 262144 sc 3 2024-09-14T06:16:29.012 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:29.008+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:29.019 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:29.020 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:29.033 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:16:29.113 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:29.112+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:29.167 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:29.168 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:29.168 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-09-14T06:16:29.183 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:16:29.226 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:29.224+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:29.256 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone1 from 0xac700e7 (overlap 0x1238399) to 0x749b0a 2024-09-14T06:16:29.314 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:29.312+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:29.353 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:29.353 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:29.366 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:16:29.372 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-14T06:16:29.458 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:29.456+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:29.516 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:29.512+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:29.532 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:29.533 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:29.546 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:16:29.633 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:29.632+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:29.712 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:29.713 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:29.718 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:29.716+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:29.727 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:16:29.819 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:29.816+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:29.834 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:29.834 INFO:teuthology.orchestra.run.smithi039.stdout:547 punch from 0x7b3d3a7 to 0x7b48dbf, (0xba18 bytes) 2024-09-14T06:16:29.845 INFO:teuthology.orchestra.run.smithi039.stdout:548 trunc from 0xac700e7 to 0x7d51de0 2024-09-14T06:16:29.870 INFO:teuthology.orchestra.run.smithi039.stdout:550 trunc from 0x7d51de0 to 0xcdbe1a 2024-09-14T06:16:29.896 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:29.896 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:29.909 INFO:teuthology.orchestra.run.smithi039.stdout:552 punch from 0xbc0cf1 to 0xbd04af, (0xf7be bytes) 2024-09-14T06:16:29.911 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:16:29.920 INFO:teuthology.orchestra.run.smithi039.stdout:553 writesame 0x1b90f60 thru 0x1b95e12 data_size 0x1(0x4eb3 bytes) 2024-09-14T06:16:29.920 INFO:teuthology.orchestra.run.smithi039.stdout:554 punch from 0x1279290 to 0x1287ff8, (0xed68 bytes) 2024-09-14T06:16:29.936 INFO:teuthology.orchestra.run.smithi039.stdout:555 writesame 0x3fe194a thru 0x3fe444c data_size 0xd(0x2b03 bytes) 2024-09-14T06:16:29.936 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:29.953 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:29.956+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:29.954 INFO:teuthology.orchestra.run.smithi039.stdout:557 writesame 0x6a848ef thru 0x6a8e972 data_size 0x2(0xa084 bytes) 2024-09-14T06:16:29.954 INFO:teuthology.orchestra.run.smithi039.stdout:558 punch from 0x47037b2 to 0x470e6ff, (0xaf4d bytes) 2024-09-14T06:16:29.971 INFO:teuthology.orchestra.run.smithi039.stdout:559 writesame 0x8ab8310 thru 0x8ac61b1 data_size 0x2a(0xdea2 bytes) 2024-09-14T06:16:29.971 INFO:teuthology.orchestra.run.smithi039.stdout:560 write 0x44df98c thru 0x44e2262 (0x28d7 bytes) 2024-09-14T06:16:29.978 INFO:teuthology.orchestra.run.smithi039.stdout:561 read 0x44536f4 thru 0x446026d (0xcb7a bytes) 2024-09-14T06:16:29.978 INFO:teuthology.orchestra.run.smithi039.stdout:563 read 0x7bdfe4e thru 0x7be9f0b (0xa0be bytes) 2024-09-14T06:16:29.979 INFO:teuthology.orchestra.run.smithi039.stdout:564 read 0x19002b3 thru 0x19034ad (0x31fb bytes) 2024-09-14T06:16:29.979 INFO:teuthology.orchestra.run.smithi039.stdout:565 writesame 0x652b3d6 thru 0x6531dd5 data_size 0x6a(0x6a00 bytes) 2024-09-14T06:16:30.009 INFO:teuthology.orchestra.run.smithi039.stdout:566 write 0xcae4b5c thru 0xcae60b7 (0x155c bytes) 2024-09-14T06:16:30.016 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:30.017 INFO:teuthology.orchestra.run.smithi039.stdout:568 read 0x4e11f03 thru 0x4e1cc5e (0xad5c bytes) 2024-09-14T06:16:30.017 INFO:teuthology.orchestra.run.smithi039.stdout:569 punch from 0x8bb05b6 to 0x8bb6bec, (0x6636 bytes) 2024-09-14T06:16:30.017 INFO:teuthology.orchestra.run.smithi039.stdout:570 write 0x5ac2e9a thru 0x5acc79f (0x9906 bytes) 2024-09-14T06:16:30.021 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:30.016+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:30.023 INFO:teuthology.orchestra.run.smithi039.stdout:571 read 0x3509ba6 thru 0x35196ad (0xfb08 bytes) 2024-09-14T06:16:30.024 INFO:teuthology.orchestra.run.smithi039.stdout:573 read 0x666d9b6 thru 0x6672ff3 (0x563e bytes) 2024-09-14T06:16:30.024 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:30.024 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:30.024 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:30.024 INFO:teuthology.orchestra.run.smithi039.stdout:578 read 0xd13696 thru 0xd154d6 (0x1e41 bytes) 2024-09-14T06:16:30.024 INFO:teuthology.orchestra.run.smithi039.stdout:580 read 0xb80abf8 thru 0xb811b0f (0x6f18 bytes) 2024-09-14T06:16:30.025 INFO:teuthology.orchestra.run.smithi039.stdout:582 write 0x763e50f thru 0x764dc55 (0xf747 bytes) 2024-09-14T06:16:30.052 INFO:teuthology.orchestra.run.smithi039.stdout:583 write 0xe17d9aa thru 0xe17dd1b (0x372 bytes) 2024-09-14T06:16:30.062 INFO:teuthology.orchestra.run.smithi039.stdout:584 read 0x21ec704 thru 0x21f3544 (0x6e41 bytes) 2024-09-14T06:16:30.062 INFO:teuthology.orchestra.run.smithi039.stdout:587 writesame 0x6f9db1d thru 0x6f9ef92 data_size 0x1(0x1476 bytes) 2024-09-14T06:16:30.073 INFO:teuthology.orchestra.run.smithi039.stdout:588 read 0xb664f77 thru 0xb670593 (0xb61d bytes) 2024-09-14T06:16:30.073 INFO:teuthology.orchestra.run.smithi039.stdout:589 write 0x1c5fc28 thru 0x1c62a93 (0x2e6c bytes) 2024-09-14T06:16:30.079 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:30.079 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:30.081 INFO:teuthology.orchestra.run.smithi039.stdout:592 punch from 0x17b1408 to 0x17b7ea9, (0x6aa1 bytes) 2024-09-14T06:16:30.081 INFO:teuthology.orchestra.run.smithi039.stdout:593 clone 3 order 20 su 524288 sc 5 2024-09-14T06:16:30.094 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:16:30.137 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:30.136+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:30.211 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:30.212 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:30.223 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:30.220+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:30.225 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:16:30.324 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:30.320+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:30.387 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:30.387 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:30.402 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:16:30.424 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:30.420+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:30.526 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:30.524+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:30.571 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:30.572 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:30.585 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:16:30.634 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:30.636+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:30.667 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone2 from 0xe17dd1c (overlap 0xcdbe1a) to 0x149302 2024-09-14T06:16:30.671 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:16:30.727 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:30.724+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:30.774 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:30.774 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:30.787 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:16:30.815 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-14T06:16:30.830 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:30.824+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:30.842 INFO:teuthology.orchestra.run.smithi039.stdout:595 write 0xaeb3576 thru 0xaeb6a08 (0x3493 bytes) 2024-09-14T06:16:30.850 INFO:teuthology.orchestra.run.smithi039.stdout:596 write 0xcd8122f thru 0xcd853f1 (0x41c3 bytes) 2024-09-14T06:16:30.856 INFO:teuthology.orchestra.run.smithi039.stdout:597 trunc from 0xe17dd1c to 0xea8eef1 2024-09-14T06:16:30.862 INFO:teuthology.orchestra.run.smithi039.stdout:598 read 0xd031c31 thru 0xd034f0c (0x32dc bytes) 2024-09-14T06:16:30.866 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:30.866 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:30.878 INFO:teuthology.orchestra.run.smithi039.stdout:599 trunc from 0xea8eef1 to 0x42737a0 2024-09-14T06:16:30.886 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:30.886 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:30.897 INFO:teuthology.orchestra.run.smithi039.stdout:600 read 0x3f17eed thru 0x3f1bfcc (0x40e0 bytes) 2024-09-14T06:16:30.897 INFO:teuthology.orchestra.run.smithi039.stdout:601 read 0x39b6b7d thru 0x39c537c (0xe800 bytes) 2024-09-14T06:16:30.897 INFO:teuthology.orchestra.run.smithi039.stdout:603 punch from 0x19617ba to 0x196d35f, (0xbba5 bytes) 2024-09-14T06:16:30.913 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:30.913 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:30.918 INFO:teuthology.orchestra.run.smithi039.stdout:606 write 0xd66eb56 thru 0xd6737c3 (0x4c6e bytes) 2024-09-14T06:16:30.921 INFO:teuthology.orchestra.run.smithi039.stdout:609 write 0x790f354 thru 0x79111f9 (0x1ea6 bytes) 2024-09-14T06:16:30.925 INFO:teuthology.orchestra.run.smithi039.stdout:611 punch from 0x9e5d2e0 to 0x9e644e7, (0x7207 bytes) 2024-09-14T06:16:30.926 INFO:teuthology.orchestra.run.smithi039.stdout:612 read 0xc88ebfd thru 0xc89062c (0x1a30 bytes) 2024-09-14T06:16:30.926 INFO:teuthology.orchestra.run.smithi039.stdout:613 read 0xa22524e thru 0xa22b93d (0x66f0 bytes) 2024-09-14T06:16:30.926 INFO:teuthology.orchestra.run.smithi039.stdout:614 read 0x47856a0 thru 0x4786233 (0xb94 bytes) 2024-09-14T06:16:30.926 INFO:teuthology.orchestra.run.smithi039.stdout:616 write 0x666673f thru 0x666abe3 (0x44a5 bytes) 2024-09-14T06:16:30.928 INFO:teuthology.orchestra.run.smithi039.stdout:617 read 0x99b21f2 thru 0x99bbdf8 (0x9c07 bytes) 2024-09-14T06:16:30.929 INFO:teuthology.orchestra.run.smithi039.stdout:618 write 0x75092ed thru 0x75168ef (0xd603 bytes) 2024-09-14T06:16:30.930 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:30.928+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:30.932 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:30.932 INFO:teuthology.orchestra.run.smithi039.stdout:620 writesame 0xd3027fb thru 0xd30da7e data_size 0x32(0xb284 bytes) 2024-09-14T06:16:30.932 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:30.933 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:30.936 INFO:teuthology.orchestra.run.smithi039.stdout:621 read 0x764b985 thru 0x7655638 (0x9cb4 bytes) 2024-09-14T06:16:30.936 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:30.936 INFO:teuthology.orchestra.run.smithi039.stdout:624 read 0xada3cc1 thru 0xadaf670 (0xb9b0 bytes) 2024-09-14T06:16:30.936 INFO:teuthology.orchestra.run.smithi039.stdout:625 writesame 0xd39848e thru 0xd3a5f59 data_size 0x86(0xdacc bytes) 2024-09-14T06:16:30.938 INFO:teuthology.orchestra.run.smithi039.stdout:628 writesame 0x651842d thru 0x6526e91 data_size 0xb(0xea65 bytes) 2024-09-14T06:16:30.944 INFO:teuthology.orchestra.run.smithi039.stdout:630 punch from 0x3623268 to 0x363320c, (0xffa4 bytes) 2024-09-14T06:16:30.944 INFO:teuthology.orchestra.run.smithi039.stdout:631 read 0xa539c80 thru 0xa53dfdb (0x435c bytes) 2024-09-14T06:16:30.945 INFO:teuthology.orchestra.run.smithi039.stdout:632 read 0x528e5cc thru 0x529cda8 (0xe7dd bytes) 2024-09-14T06:16:30.945 INFO:teuthology.orchestra.run.smithi039.stdout:633 read 0xadebed thru 0xaea349 (0xb75d bytes) 2024-09-14T06:16:30.945 INFO:teuthology.orchestra.run.smithi039.stdout:636 read 0xa236360 thru 0xa23740a (0x10ab bytes) 2024-09-14T06:16:30.951 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:30.951 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:30.952 INFO:teuthology.orchestra.run.smithi039.stdout:637 trunc from 0xd6737c4 to 0x94482c3 2024-09-14T06:16:30.955 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:30.955 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:30.966 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:16:30.967 INFO:teuthology.orchestra.run.smithi039.stdout:638 trunc from 0x94482c3 to 0xa7221ac 2024-09-14T06:16:30.974 INFO:teuthology.orchestra.run.smithi039.stdout:640 read 0x2c7d32a thru 0x2c84a26 (0x76fd bytes) 2024-09-14T06:16:30.974 INFO:teuthology.orchestra.run.smithi039.stdout:641 write 0x7c9d55f thru 0x7ca7346 (0x9de8 bytes) 2024-09-14T06:16:30.977 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:30.977 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:30.983 INFO:teuthology.orchestra.run.smithi039.stdout:642 trunc from 0xa7221ac to 0x89010b5 2024-09-14T06:16:30.989 INFO:teuthology.orchestra.run.smithi039.stdout:644 write 0x2908e1a thru 0x2909398 (0x57f bytes) 2024-09-14T06:16:31.000 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:31.000 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:31.007 INFO:teuthology.orchestra.run.smithi039.stdout:645 writesame 0xeb0e3a5 thru 0xeb0faf4 data_size 0x8(0x1750 bytes) 2024-09-14T06:16:31.007 INFO:teuthology.orchestra.run.smithi039.stdout:646 writesame 0xcc27f9b thru 0xcc293d9 data_size 0x1(0x143f bytes) 2024-09-14T06:16:31.020 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:31.021 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:31.024 INFO:teuthology.orchestra.run.smithi039.stdout:647 trunc from 0xeb0faf5 to 0x625e110 2024-09-14T06:16:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:650 write 0xbec6812 thru 0xbecf0c3 (0x88b2 bytes) 2024-09-14T06:16:31.053 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:31.052+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:31.059 INFO:teuthology.orchestra.run.smithi039.stdout:651 write 0xbfe086e thru 0xbfe0f0f (0x6a2 bytes) 2024-09-14T06:16:31.063 INFO:teuthology.orchestra.run.smithi039.stdout:652 read 0x2764e00 thru 0x27681a7 (0x33a8 bytes) 2024-09-14T06:16:31.063 INFO:teuthology.orchestra.run.smithi039.stdout:653 read 0x98c2123 thru 0x98cf8f2 (0xd7d0 bytes) 2024-09-14T06:16:31.064 INFO:teuthology.orchestra.run.smithi039.stdout:656 read 0x78ea392 thru 0x78f4b32 (0xa7a1 bytes) 2024-09-14T06:16:31.064 INFO:teuthology.orchestra.run.smithi039.stdout:657 read 0xa2fd63a thru 0xa308b06 (0xb4cd bytes) 2024-09-14T06:16:31.064 INFO:teuthology.orchestra.run.smithi039.stdout:658 punch from 0x3e50849 to 0x3e5bdec, (0xb5a3 bytes) 2024-09-14T06:16:31.064 INFO:teuthology.orchestra.run.smithi039.stdout:659 read 0x9fa75ed thru 0x9fb11de (0x9bf2 bytes) 2024-09-14T06:16:31.064 INFO:teuthology.orchestra.run.smithi039.stdout:660 read 0xa4a1984 thru 0xa4abe6a (0xa4e7 bytes) 2024-09-14T06:16:31.064 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:31.064 INFO:teuthology.orchestra.run.smithi039.stdout:662 writesame 0x4e32bc1 thru 0x4e364e0 data_size 0x10(0x3920 bytes) 2024-09-14T06:16:31.067 INFO:teuthology.orchestra.run.smithi039.stdout:663 read 0x481d18 thru 0x488b39 (0x6e22 bytes) 2024-09-14T06:16:31.068 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:31.069 INFO:teuthology.orchestra.run.smithi039.stdout:665 write 0xaa7c417 thru 0xaa8453e (0x8128 bytes) 2024-09-14T06:16:31.082 INFO:teuthology.orchestra.run.smithi039.stdout:666 write 0xc9fa2ab thru 0xc9fc151 (0x1ea7 bytes) 2024-09-14T06:16:31.086 INFO:teuthology.orchestra.run.smithi039.stdout:667 writesame 0x5f79d2b thru 0x5f81c99 data_size 0x13(0x7f6f bytes) 2024-09-14T06:16:31.091 INFO:teuthology.orchestra.run.smithi039.stdout:668 punch from 0x3f3b058 to 0x3f3f0f2, (0x409a bytes) 2024-09-14T06:16:31.110 INFO:teuthology.orchestra.run.smithi039.stdout:669 trunc from 0xc9fc152 to 0x1eef093 2024-09-14T06:16:31.131 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:31.128+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:31.134 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:31.134 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:16:31.134 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:16:31.134 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:16:31.134 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.134 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:3735 3.c e30/23 3:373cb2d0:::rbd_data.1152bd92e04f.000000000000005c:head v 30'8, mlcod=28'6)", 2024-09-14T06:16:31.134 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:27.302988+0000", 2024-09-14T06:16:31.135 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 3.8301502649999999, 2024-09-14T06:16:31.135 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0037853980000000001, 2024-09-14T06:16:31.135 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.135 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.135 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.135 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.135 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.135 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.135 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.302988+0000", 2024-09-14T06:16:31.135 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.135 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.135 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.135 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.135 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.302988+0000", 2024-09-14T06:16:31.135 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.136 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.136 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.136 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.136 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.302989+0000", 2024-09-14T06:16:31.136 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:16:31.136 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.136 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.136 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.136 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.303026+0000", 2024-09-14T06:16:31.136 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:16:31.136 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.136 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.136 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.136 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.303058+0000", 2024-09-14T06:16:31.137 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1736000000000002e-05 2024-09-14T06:16:31.137 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.137 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.137 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.137 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.303079+0000", 2024-09-14T06:16:31.137 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0939999999999999e-05 2024-09-14T06:16:31.137 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.137 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.137 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.137 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.303226+0000", 2024-09-14T06:16:31.137 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014744899999999999 2024-09-14T06:16:31.137 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.137 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.138 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.138 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.303286+0000", 2024-09-14T06:16:31.138 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0247000000000001e-05 2024-09-14T06:16:31.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.138 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.138 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.138 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.306287+0000", 2024-09-14T06:16:31.138 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003000693 2024-09-14T06:16:31.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.138 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.138 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.138 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.306773+0000", 2024-09-14T06:16:31.138 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00048613699999999998 2024-09-14T06:16:31.139 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.139 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.139 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.139 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.139 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.139 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:3822 3.6 e30/23 3:6781ec00:::rbd_data.1152bd92e04f.0000000000000050:head v 30'93, mlcod=30'91)", 2024-09-14T06:16:31.139 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:27.442150+0000", 2024-09-14T06:16:31.139 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 3.6909883030000001, 2024-09-14T06:16:31.139 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0048341759999999999, 2024-09-14T06:16:31.139 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.139 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.139 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.139 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.139 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.139 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.140 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.442150+0000", 2024-09-14T06:16:31.140 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.140 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.140 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.140 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.442150+0000", 2024-09-14T06:16:31.140 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.140 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.140 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.140 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.442152+0000", 2024-09-14T06:16:31.140 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.86e-06 2024-09-14T06:16:31.141 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.141 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.141 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.141 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.442166+0000", 2024-09-14T06:16:31.141 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4639e-05 2024-09-14T06:16:31.141 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.141 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.141 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.141 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.442209+0000", 2024-09-14T06:16:31.141 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2772999999999999e-05 2024-09-14T06:16:31.141 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.141 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.141 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.141 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.442231+0000", 2024-09-14T06:16:31.142 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1373e-05 2024-09-14T06:16:31.142 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.142 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.142 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.142 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.442706+0000", 2024-09-14T06:16:31.142 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00047542 2024-09-14T06:16:31.142 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.142 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.142 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.142 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.442759+0000", 2024-09-14T06:16:31.142 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3409999999999999e-05 2024-09-14T06:16:31.142 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.142 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.143 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.143 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.446882+0000", 2024-09-14T06:16:31.143 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041229739999999997 2024-09-14T06:16:31.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.143 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.143 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.143 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.446984+0000", 2024-09-14T06:16:31.143 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000101727 2024-09-14T06:16:31.143 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.143 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.143 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.143 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.144 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:3830 3.6 e30/23 3:65acea60:::rbd_data.1152bd92e04f.0000000000000064:head v 30'94, mlcod=30'91)", 2024-09-14T06:16:31.144 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:27.444637+0000", 2024-09-14T06:16:31.144 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 3.688501306, 2024-09-14T06:16:31.144 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0049929309999999999, 2024-09-14T06:16:31.144 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.144 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.144 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.144 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.144 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.144 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.144 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.444637+0000", 2024-09-14T06:16:31.144 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.144 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.144 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.145 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.145 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.444637+0000", 2024-09-14T06:16:31.145 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.145 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.145 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.145 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.444639+0000", 2024-09-14T06:16:31.145 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.68e-06 2024-09-14T06:16:31.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.145 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.145 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.145 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.444652+0000", 2024-09-14T06:16:31.145 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.385e-05 2024-09-14T06:16:31.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.146 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.146 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.146 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.444691+0000", 2024-09-14T06:16:31.146 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8210000000000002e-05 2024-09-14T06:16:31.146 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.146 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.146 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.146 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.444711+0000", 2024-09-14T06:16:31.146 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0682999999999998e-05 2024-09-14T06:16:31.146 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.146 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.146 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.146 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.444862+0000", 2024-09-14T06:16:31.146 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015040499999999999 2024-09-14T06:16:31.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.147 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.147 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.147 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.444919+0000", 2024-09-14T06:16:31.147 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7720000000000003e-05 2024-09-14T06:16:31.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.147 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.147 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.147 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.449535+0000", 2024-09-14T06:16:31.147 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0046151680000000002 2024-09-14T06:16:31.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.147 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.147 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.147 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.449630+0000", 2024-09-14T06:16:31.148 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5215000000000002e-05 2024-09-14T06:16:31.148 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.148 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.148 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.148 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.148 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.148 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:3915 3.b 3:d2b65fcb:::rbd_data.1152bd92e04f.0000000000000001:head [stat,truncate 1605632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.148 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:27.580521+0000", 2024-09-14T06:16:31.148 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 3.5526166699999999, 2024-09-14T06:16:31.148 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0066592040000000002, 2024-09-14T06:16:31.148 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.148 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.148 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.148 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:31.149 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:31.149 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.149 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 3915 2024-09-14T06:16:31.149 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.149 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.149 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.149 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.149 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.580521+0000", 2024-09-14T06:16:31.149 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.149 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.149 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.149 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.149 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.580521+0000", 2024-09-14T06:16:31.149 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.150 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.150 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.150 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.150 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.580525+0000", 2024-09-14T06:16:31.150 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2030000000000001e-06 2024-09-14T06:16:31.150 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.150 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.150 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.150 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.580538+0000", 2024-09-14T06:16:31.151 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3127e-05 2024-09-14T06:16:31.151 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.151 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.151 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.151 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.580576+0000", 2024-09-14T06:16:31.151 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7746000000000001e-05 2024-09-14T06:16:31.151 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.151 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.151 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.151 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.580634+0000", 2024-09-14T06:16:31.151 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8853e-05 2024-09-14T06:16:31.151 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.151 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.152 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.152 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.580801+0000", 2024-09-14T06:16:31.152 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000166949 2024-09-14T06:16:31.152 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.152 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.152 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.152 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.581056+0000", 2024-09-14T06:16:31.152 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025468399999999998 2024-09-14T06:16:31.152 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.152 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.152 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:31.152 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.581421+0000", 2024-09-14T06:16:31.152 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036491399999999998 2024-09-14T06:16:31.152 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.152 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.153 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:31.153 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.583437+0000", 2024-09-14T06:16:31.153 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0020158870000000001 2024-09-14T06:16:31.153 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.153 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.153 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:31.153 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.586891+0000", 2024-09-14T06:16:31.153 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0034537499999999998 2024-09-14T06:16:31.153 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.153 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.153 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.153 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.587053+0000", 2024-09-14T06:16:31.153 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00016265600000000001 2024-09-14T06:16:31.153 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.154 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.154 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.154 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.587181+0000", 2024-09-14T06:16:31.154 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012743499999999999 2024-09-14T06:16:31.154 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.154 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.154 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.154 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.154 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.154 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:4547 3.b 3:d2b65fcb:::rbd_data.1152bd92e04f.0000000000000001:head [stat,write 819200~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.154 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:27.733429+0000", 2024-09-14T06:16:31.154 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 3.39970908, 2024-09-14T06:16:31.154 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047194999999999997, 2024-09-14T06:16:31.155 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.155 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.155 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.155 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:31.155 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:31.155 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.155 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 4547 2024-09-14T06:16:31.155 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.155 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.155 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.155 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.155 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.733429+0000", 2024-09-14T06:16:31.155 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.155 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.156 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.156 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.156 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.733429+0000", 2024-09-14T06:16:31.156 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.156 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.156 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.156 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.156 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.733439+0000", 2024-09-14T06:16:31.156 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0190000000000001e-05 2024-09-14T06:16:31.156 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.156 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.156 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.156 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.733499+0000", 2024-09-14T06:16:31.156 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9886000000000002e-05 2024-09-14T06:16:31.157 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.157 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.157 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.157 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.733533+0000", 2024-09-14T06:16:31.157 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4066999999999998e-05 2024-09-14T06:16:31.157 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.157 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.157 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.157 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.733563+0000", 2024-09-14T06:16:31.157 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9679000000000002e-05 2024-09-14T06:16:31.157 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.157 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.157 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.158 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.733703+0000", 2024-09-14T06:16:31.158 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000140109 2024-09-14T06:16:31.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.158 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.158 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.158 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.733866+0000", 2024-09-14T06:16:31.158 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00016317600000000001 2024-09-14T06:16:31.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.158 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.158 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:31.158 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.734108+0000", 2024-09-14T06:16:31.158 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024189099999999999 2024-09-14T06:16:31.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.158 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.158 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:31.159 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.736433+0000", 2024-09-14T06:16:31.159 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.002324965 2024-09-14T06:16:31.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.159 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.159 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:31.159 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.737998+0000", 2024-09-14T06:16:31.159 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015648719999999999 2024-09-14T06:16:31.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.159 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.159 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.159 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.738082+0000", 2024-09-14T06:16:31.159 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4462000000000005e-05 2024-09-14T06:16:31.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.159 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.160 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.160 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.738149+0000", 2024-09-14T06:16:31.160 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6203000000000003e-05 2024-09-14T06:16:31.160 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.160 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.160 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.160 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.160 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.160 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:4554 3.8 e30/23 3:141e338d:::rbd_data.1152bd92e04f.0000000000000067:head v 30'15, mlcod=30'13)", 2024-09-14T06:16:31.160 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:27.744999+0000", 2024-09-14T06:16:31.160 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 3.3881392159999999, 2024-09-14T06:16:31.160 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038066739999999999, 2024-09-14T06:16:31.160 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.161 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.161 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.161 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.161 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.161 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.161 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.744999+0000", 2024-09-14T06:16:31.161 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.161 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.161 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.161 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.161 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.744999+0000", 2024-09-14T06:16:31.161 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.161 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.161 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.162 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.162 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.745001+0000", 2024-09-14T06:16:31.162 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6330000000000001e-06 2024-09-14T06:16:31.162 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.162 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.162 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.162 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.745014+0000", 2024-09-14T06:16:31.162 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3327e-05 2024-09-14T06:16:31.162 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.162 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.162 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.162 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.745041+0000", 2024-09-14T06:16:31.162 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7078999999999999e-05 2024-09-14T06:16:31.162 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.163 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.163 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.163 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.745055+0000", 2024-09-14T06:16:31.163 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4167e-05 2024-09-14T06:16:31.163 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.163 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.163 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.163 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.745181+0000", 2024-09-14T06:16:31.163 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012539199999999999 2024-09-14T06:16:31.163 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.163 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.163 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.163 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.745228+0000", 2024-09-14T06:16:31.163 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7663000000000001e-05 2024-09-14T06:16:31.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.164 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.164 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.164 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.748723+0000", 2024-09-14T06:16:31.164 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0034948869999999999 2024-09-14T06:16:31.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.164 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.164 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.164 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.748806+0000", 2024-09-14T06:16:31.164 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2526000000000002e-05 2024-09-14T06:16:31.164 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.164 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.164 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.165 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.165 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:4598 3.b 3:d2b65fcb:::rbd_data.1152bd92e04f.0000000000000001:head [stat,write 1441792~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.165 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:27.792567+0000", 2024-09-14T06:16:31.165 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 3.3405713389999998, 2024-09-14T06:16:31.165 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041338010000000003, 2024-09-14T06:16:31.165 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.165 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.165 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.165 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:31.165 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:31.165 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.165 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 4598 2024-09-14T06:16:31.165 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.166 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.166 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.166 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.166 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.792567+0000", 2024-09-14T06:16:31.166 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.166 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.166 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.166 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.792567+0000", 2024-09-14T06:16:31.166 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.166 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.166 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.166 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.792569+0000", 2024-09-14T06:16:31.167 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3630000000000001e-06 2024-09-14T06:16:31.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.167 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.167 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.167 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.792616+0000", 2024-09-14T06:16:31.167 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6743000000000003e-05 2024-09-14T06:16:31.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.167 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.167 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.167 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.792651+0000", 2024-09-14T06:16:31.167 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4832999999999997e-05 2024-09-14T06:16:31.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.167 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.168 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.168 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.792672+0000", 2024-09-14T06:16:31.168 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1696999999999999e-05 2024-09-14T06:16:31.168 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.168 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.168 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.168 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.792843+0000", 2024-09-14T06:16:31.168 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000170975 2024-09-14T06:16:31.168 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.168 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.168 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.168 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.793077+0000", 2024-09-14T06:16:31.168 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00023375199999999999 2024-09-14T06:16:31.169 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.169 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.169 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:31.169 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.793462+0000", 2024-09-14T06:16:31.169 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038441000000000001 2024-09-14T06:16:31.169 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.169 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.169 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:31.169 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.795155+0000", 2024-09-14T06:16:31.169 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0016928970000000001 2024-09-14T06:16:31.169 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.169 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.169 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:31.169 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.796490+0000", 2024-09-14T06:16:31.170 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001335436 2024-09-14T06:16:31.170 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.170 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.170 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.170 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.796631+0000", 2024-09-14T06:16:31.170 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000140572 2024-09-14T06:16:31.170 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.170 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.170 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.170 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.796701+0000", 2024-09-14T06:16:31.170 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0123000000000006e-05 2024-09-14T06:16:31.170 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.170 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.170 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.171 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.171 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.171 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:4724 3.b 3:d2b65fcb:::rbd_data.1152bd92e04f.0000000000000001:head [stat,truncate 1966080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.171 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:27.956662+0000", 2024-09-14T06:16:31.171 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 3.176476321, 2024-09-14T06:16:31.171 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043017999999999997, 2024-09-14T06:16:31.171 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.171 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.171 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.171 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:31.171 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:31.171 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.171 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 4724 2024-09-14T06:16:31.171 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.172 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.172 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.172 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.172 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.956662+0000", 2024-09-14T06:16:31.172 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.172 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.172 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.172 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.172 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.956662+0000", 2024-09-14T06:16:31.172 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.172 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.172 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.172 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.173 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.956665+0000", 2024-09-14T06:16:31.173 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4630000000000001e-06 2024-09-14T06:16:31.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.173 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.173 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.173 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.956679+0000", 2024-09-14T06:16:31.173 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4070000000000001e-05 2024-09-14T06:16:31.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.173 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.173 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.173 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.956722+0000", 2024-09-14T06:16:31.173 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:16:31.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.173 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.174 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.174 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.956747+0000", 2024-09-14T06:16:31.174 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4409e-05 2024-09-14T06:16:31.174 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.174 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.174 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.174 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.956918+0000", 2024-09-14T06:16:31.174 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017164900000000001 2024-09-14T06:16:31.174 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.174 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.174 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.174 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.957123+0000", 2024-09-14T06:16:31.174 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00020520799999999999 2024-09-14T06:16:31.174 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.175 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.175 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:31.175 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.957320+0000", 2024-09-14T06:16:31.175 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019682599999999999 2024-09-14T06:16:31.175 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.175 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.175 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:31.175 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.959926+0000", 2024-09-14T06:16:31.175 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.002605343 2024-09-14T06:16:31.175 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.175 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.175 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:31.175 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.960699+0000", 2024-09-14T06:16:31.175 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00077343400000000004 2024-09-14T06:16:31.176 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.176 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.176 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.176 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.960856+0000", 2024-09-14T06:16:31.176 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000156608 2024-09-14T06:16:31.176 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.176 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.176 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.176 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.960964+0000", 2024-09-14T06:16:31.176 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000108 2024-09-14T06:16:31.176 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.176 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.176 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.176 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.176 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.177 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:4835 3.8 e30/23 3:141e338d:::rbd_data.1152bd92e04f.0000000000000067:head v 30'19, mlcod=30'17)", 2024-09-14T06:16:31.177 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:28.136402+0000", 2024-09-14T06:16:31.177 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 2.996736549, 2024-09-14T06:16:31.177 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00431724, 2024-09-14T06:16:31.177 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.177 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.177 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.177 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.177 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.177 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.177 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.136402+0000", 2024-09-14T06:16:31.177 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.177 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.177 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.178 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.178 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.136402+0000", 2024-09-14T06:16:31.178 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.178 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.178 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.178 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.178 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.136404+0000", 2024-09-14T06:16:31.178 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0099999999999998e-06 2024-09-14T06:16:31.178 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.178 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.178 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.178 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.136420+0000", 2024-09-14T06:16:31.178 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6200000000000001e-05 2024-09-14T06:16:31.178 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.179 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.179 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.179 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.136462+0000", 2024-09-14T06:16:31.179 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2079999999999997e-05 2024-09-14T06:16:31.179 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.179 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.179 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.179 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.136491+0000", 2024-09-14T06:16:31.179 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9431999999999999e-05 2024-09-14T06:16:31.179 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.179 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.179 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.179 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.136716+0000", 2024-09-14T06:16:31.179 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022424599999999999 2024-09-14T06:16:31.180 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.180 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.180 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.180 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.136776+0000", 2024-09-14T06:16:31.180 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0168999999999999e-05 2024-09-14T06:16:31.180 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.180 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.180 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.180 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.140607+0000", 2024-09-14T06:16:31.180 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038313269999999998 2024-09-14T06:16:31.180 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.180 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.180 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.180 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.140719+0000", 2024-09-14T06:16:31.181 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000111776 2024-09-14T06:16:31.181 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.181 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.181 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.181 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.181 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.181 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:4832 3.6 e30/23 3:645c5b98:::rbd_data.1152bd92e04f.0000000000000048:head v 30'102, mlcod=30'100)", 2024-09-14T06:16:31.181 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:28.136674+0000", 2024-09-14T06:16:31.181 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 2.9964638150000003, 2024-09-14T06:16:31.181 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0066804450000000001, 2024-09-14T06:16:31.181 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.181 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.181 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.182 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.182 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.182 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.182 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.136674+0000", 2024-09-14T06:16:31.182 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.182 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.182 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.182 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.182 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.136674+0000", 2024-09-14T06:16:31.182 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.182 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.182 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.182 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.182 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.136676+0000", 2024-09-14T06:16:31.183 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4440000000000001e-06 2024-09-14T06:16:31.183 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.183 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.183 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.183 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.136688+0000", 2024-09-14T06:16:31.183 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2336e-05 2024-09-14T06:16:31.183 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.183 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.183 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.183 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.136713+0000", 2024-09-14T06:16:31.183 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4567e-05 2024-09-14T06:16:31.183 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.183 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.183 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.184 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.136729+0000", 2024-09-14T06:16:31.184 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6673000000000001e-05 2024-09-14T06:16:31.184 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.184 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.184 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.184 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.136867+0000", 2024-09-14T06:16:31.184 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013786199999999999 2024-09-14T06:16:31.184 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.184 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.184 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.184 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.136923+0000", 2024-09-14T06:16:31.184 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.5946000000000003e-05 2024-09-14T06:16:31.184 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.185 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.185 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.185 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.143272+0000", 2024-09-14T06:16:31.185 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0063490380000000004 2024-09-14T06:16:31.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.185 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.185 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.185 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.143355+0000", 2024-09-14T06:16:31.185 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2578999999999997e-05 2024-09-14T06:16:31.185 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.185 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.185 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.185 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.186 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:4925 3.0 e30/23 3:04f17704:::rbd_data.1152bd92e04f.0000000000000022:head v 30'4, mlcod=28'2)", 2024-09-14T06:16:31.186 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:28.248288+0000", 2024-09-14T06:16:31.186 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 2.8848503389999998, 2024-09-14T06:16:31.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.004097715, 2024-09-14T06:16:31.186 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.186 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.186 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.186 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.248288+0000", 2024-09-14T06:16:31.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.248288+0000", 2024-09-14T06:16:31.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.248290+0000", 2024-09-14T06:16:31.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8530000000000001e-06 2024-09-14T06:16:31.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.248349+0000", 2024-09-14T06:16:31.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9673000000000003e-05 2024-09-14T06:16:31.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.188 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.188 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.188 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.248384+0000", 2024-09-14T06:16:31.188 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5086000000000003e-05 2024-09-14T06:16:31.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.188 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.188 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.188 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.248416+0000", 2024-09-14T06:16:31.188 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1217000000000003e-05 2024-09-14T06:16:31.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.188 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.188 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.188 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.248591+0000", 2024-09-14T06:16:31.188 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017499099999999999 2024-09-14T06:16:31.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.189 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.189 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.189 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.248646+0000", 2024-09-14T06:16:31.189 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.541e-05 2024-09-14T06:16:31.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.189 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.189 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.189 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.252265+0000", 2024-09-14T06:16:31.189 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036187789999999999 2024-09-14T06:16:31.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.189 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.189 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.189 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.252386+0000", 2024-09-14T06:16:31.189 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012070600000000001 2024-09-14T06:16:31.190 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.190 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.190 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.190 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:4959 3.b 3:d11b050d:::rbd_data.1152bd92e04f.0000000000000019:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.190 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:28.297414+0000", 2024-09-14T06:16:31.190 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 2.8357245120000001, 2024-09-14T06:16:31.190 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0091376930000000006, 2024-09-14T06:16:31.190 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.190 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.190 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.190 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:31.190 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:31.191 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.191 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 4959 2024-09-14T06:16:31.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.191 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.191 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.297414+0000", 2024-09-14T06:16:31.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.191 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.297414+0000", 2024-09-14T06:16:31.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.297416+0000", 2024-09-14T06:16:31.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7599999999999998e-06 2024-09-14T06:16:31.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.297430+0000", 2024-09-14T06:16:31.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3383e-05 2024-09-14T06:16:31.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.297470+0000", 2024-09-14T06:16:31.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0213000000000003e-05 2024-09-14T06:16:31.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.193 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.193 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.297491+0000", 2024-09-14T06:16:31.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1313000000000001e-05 2024-09-14T06:16:31.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.193 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.193 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.297667+0000", 2024-09-14T06:16:31.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017570199999999999 2024-09-14T06:16:31.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.193 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.194 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.297889+0000", 2024-09-14T06:16:31.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022198500000000001 2024-09-14T06:16:31.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.194 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:31.194 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.298180+0000", 2024-09-14T06:16:31.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000290678 2024-09-14T06:16:31.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.194 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:31.194 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.301715+0000", 2024-09-14T06:16:31.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0035352589999999998 2024-09-14T06:16:31.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.194 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:31.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.306273+0000", 2024-09-14T06:16:31.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0045575590000000001 2024-09-14T06:16:31.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.306424+0000", 2024-09-14T06:16:31.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015178500000000001 2024-09-14T06:16:31.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.306551+0000", 2024-09-14T06:16:31.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012705600000000001 2024-09-14T06:16:31.195 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.196 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.196 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.196 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:4968 3.6 e30/23 3:66977818:::rbd_data.1152bd92e04f.0000000000000028:head v 30'109, mlcod=30'106)", 2024-09-14T06:16:31.196 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:28.301727+0000", 2024-09-14T06:16:31.196 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 2.8314114090000002, 2024-09-14T06:16:31.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003649846, 2024-09-14T06:16:31.196 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.196 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.196 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.196 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.301727+0000", 2024-09-14T06:16:31.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.301727+0000", 2024-09-14T06:16:31.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.301729+0000", 2024-09-14T06:16:31.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.844e-06 2024-09-14T06:16:31.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.198 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.198 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.301742+0000", 2024-09-14T06:16:31.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3852999999999999e-05 2024-09-14T06:16:31.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.198 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.198 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.301773+0000", 2024-09-14T06:16:31.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0142999999999999e-05 2024-09-14T06:16:31.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.198 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.198 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.301791+0000", 2024-09-14T06:16:31.199 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8414e-05 2024-09-14T06:16:31.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.199 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.301940+0000", 2024-09-14T06:16:31.199 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014853499999999999 2024-09-14T06:16:31.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.199 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.302007+0000", 2024-09-14T06:16:31.199 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7199000000000005e-05 2024-09-14T06:16:31.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.199 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.305255+0000", 2024-09-14T06:16:31.200 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032481889999999999 2024-09-14T06:16:31.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.200 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.200 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.200 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.305377+0000", 2024-09-14T06:16:31.200 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000121669 2024-09-14T06:16:31.200 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.200 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.200 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.200 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.200 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:4971 3.6 e30/23 3:65fece57:::rbd_data.1152bd92e04f.0000000000000038:head v 30'110, mlcod=30'107)", 2024-09-14T06:16:31.200 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:28.304041+0000", 2024-09-14T06:16:31.200 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 2.8290969370000001, 2024-09-14T06:16:31.201 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041365020000000002, 2024-09-14T06:16:31.201 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.201 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.201 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.201 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.201 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.201 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.201 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.304041+0000", 2024-09-14T06:16:31.201 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.201 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.201 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.201 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.201 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.304041+0000", 2024-09-14T06:16:31.201 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.202 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.202 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.202 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.304043+0000", 2024-09-14T06:16:31.202 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8500000000000001e-06 2024-09-14T06:16:31.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.202 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.202 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.202 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.304057+0000", 2024-09-14T06:16:31.202 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3687e-05 2024-09-14T06:16:31.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.202 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.202 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.202 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.304096+0000", 2024-09-14T06:16:31.203 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8797000000000001e-05 2024-09-14T06:16:31.203 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.203 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.203 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.203 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.304114+0000", 2024-09-14T06:16:31.203 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8816000000000001e-05 2024-09-14T06:16:31.203 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.203 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.203 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.203 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.304273+0000", 2024-09-14T06:16:31.203 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015859899999999999 2024-09-14T06:16:31.203 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.203 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.204 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.304337+0000", 2024-09-14T06:16:31.204 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4133000000000004e-05 2024-09-14T06:16:31.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.204 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.308069+0000", 2024-09-14T06:16:31.204 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0037317869999999999 2024-09-14T06:16:31.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.204 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.308178+0000", 2024-09-14T06:16:31.204 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00010883300000000001 2024-09-14T06:16:31.205 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.205 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.205 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.205 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.205 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:5054 3.6 e30/23 3:65fece57:::rbd_data.1152bd92e04f.0000000000000038:head v 30'111, mlcod=30'109)", 2024-09-14T06:16:31.205 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:28.407323+0000", 2024-09-14T06:16:31.205 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 2.7258156499999999, 2024-09-14T06:16:31.205 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041132920000000002, 2024-09-14T06:16:31.205 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.205 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.205 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.205 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.206 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.206 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.407323+0000", 2024-09-14T06:16:31.206 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.206 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.206 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.206 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.407323+0000", 2024-09-14T06:16:31.206 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.206 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.206 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.206 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.407325+0000", 2024-09-14T06:16:31.206 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2730000000000001e-06 2024-09-14T06:16:31.207 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.207 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.207 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.207 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.407381+0000", 2024-09-14T06:16:31.207 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6137000000000001e-05 2024-09-14T06:16:31.207 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.207 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.207 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.207 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.407412+0000", 2024-09-14T06:16:31.207 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1250000000000001e-05 2024-09-14T06:16:31.207 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.207 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.207 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.207 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.407432+0000", 2024-09-14T06:16:31.207 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9568999999999998e-05 2024-09-14T06:16:31.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.208 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.208 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.208 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.407603+0000", 2024-09-14T06:16:31.208 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000170932 2024-09-14T06:16:31.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.208 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.208 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.208 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.407659+0000", 2024-09-14T06:16:31.208 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6023000000000002e-05 2024-09-14T06:16:31.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.208 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.208 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.208 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.411337+0000", 2024-09-14T06:16:31.209 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036784880000000002 2024-09-14T06:16:31.209 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.209 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.209 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.209 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.411436+0000", 2024-09-14T06:16:31.209 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.8620000000000001e-05 2024-09-14T06:16:31.209 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.209 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.209 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.209 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.209 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.209 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:5073 3.8 e30/23 3:17a31888:::rbd_data.1152bd92e04f.000000000000005a:head v 30'23, mlcod=30'21)", 2024-09-14T06:16:31.209 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:28.439548+0000", 2024-09-14T06:16:31.209 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 2.6935905920000001, 2024-09-14T06:16:31.210 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0035628859999999999, 2024-09-14T06:16:31.210 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.210 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.210 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.210 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.210 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.210 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.210 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.439548+0000", 2024-09-14T06:16:31.210 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.210 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.210 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.210 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.439548+0000", 2024-09-14T06:16:31.210 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.211 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.211 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.211 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.211 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.439549+0000", 2024-09-14T06:16:31.211 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.813e-06 2024-09-14T06:16:31.211 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.211 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.211 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.211 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.439564+0000", 2024-09-14T06:16:31.211 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4877e-05 2024-09-14T06:16:31.211 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.211 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.211 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.211 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.439599+0000", 2024-09-14T06:16:31.211 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4996000000000002e-05 2024-09-14T06:16:31.212 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.212 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.212 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.212 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.439618+0000", 2024-09-14T06:16:31.212 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9106999999999998e-05 2024-09-14T06:16:31.212 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.212 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.212 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.212 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.439802+0000", 2024-09-14T06:16:31.212 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00018358799999999999 2024-09-14T06:16:31.212 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.212 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.212 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.212 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.439863+0000", 2024-09-14T06:16:31.213 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1080000000000005e-05 2024-09-14T06:16:31.213 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.213 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.213 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.213 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.443026+0000", 2024-09-14T06:16:31.213 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0031632679999999999 2024-09-14T06:16:31.213 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.213 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.213 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.213 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.443110+0000", 2024-09-14T06:16:31.213 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4157000000000001e-05 2024-09-14T06:16:31.213 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.213 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.213 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.214 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.214 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.214 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:5099 3.8 e30/23 3:17a31888:::rbd_data.1152bd92e04f.000000000000005a:head v 30'25, mlcod=30'23)", 2024-09-14T06:16:31.214 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:28.472126+0000", 2024-09-14T06:16:31.214 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 2.6610122700000001, 2024-09-14T06:16:31.214 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038435129999999998, 2024-09-14T06:16:31.214 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.214 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.214 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.214 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.214 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.214 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.214 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.472126+0000", 2024-09-14T06:16:31.214 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.214 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.215 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.215 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.215 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.472126+0000", 2024-09-14T06:16:31.215 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.215 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.215 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.215 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.215 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.472128+0000", 2024-09-14T06:16:31.215 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.866e-06 2024-09-14T06:16:31.215 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.215 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.215 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.215 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.472182+0000", 2024-09-14T06:16:31.215 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4079999999999997e-05 2024-09-14T06:16:31.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.216 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.216 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.216 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.472232+0000", 2024-09-14T06:16:31.216 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0358999999999997e-05 2024-09-14T06:16:31.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.216 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.216 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.216 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.472252+0000", 2024-09-14T06:16:31.216 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9417000000000001e-05 2024-09-14T06:16:31.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.216 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.216 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.216 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.472384+0000", 2024-09-14T06:16:31.217 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000132192 2024-09-14T06:16:31.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.217 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.217 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.217 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.472417+0000", 2024-09-14T06:16:31.217 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3243e-05 2024-09-14T06:16:31.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.217 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.217 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.217 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.475893+0000", 2024-09-14T06:16:31.217 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0034759330000000001 2024-09-14T06:16:31.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.217 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.217 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.218 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.475969+0000", 2024-09-14T06:16:31.218 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6422999999999997e-05 2024-09-14T06:16:31.218 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.218 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.218 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.218 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.218 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.218 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:5117 3.6 e30/23 3:65fece57:::rbd_data.1152bd92e04f.0000000000000038:head v 30'112, mlcod=30'110)", 2024-09-14T06:16:31.218 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:28.497826+0000", 2024-09-14T06:16:31.218 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 2.6353117619999997, 2024-09-14T06:16:31.218 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040073790000000001, 2024-09-14T06:16:31.218 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.218 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.218 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.219 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.219 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.219 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.219 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.497826+0000", 2024-09-14T06:16:31.219 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.219 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.219 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.219 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.219 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.497826+0000", 2024-09-14T06:16:31.219 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.219 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.219 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.219 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.219 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.497828+0000", 2024-09-14T06:16:31.220 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.73e-06 2024-09-14T06:16:31.220 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.220 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.220 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.220 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.497883+0000", 2024-09-14T06:16:31.220 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.5309000000000002e-05 2024-09-14T06:16:31.220 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.220 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.220 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.221 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.497912+0000", 2024-09-14T06:16:31.221 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.853e-05 2024-09-14T06:16:31.221 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.221 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.221 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.221 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.497929+0000", 2024-09-14T06:16:31.221 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6543999999999999e-05 2024-09-14T06:16:31.221 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.221 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.221 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.221 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.498077+0000", 2024-09-14T06:16:31.221 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000148565 2024-09-14T06:16:31.221 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.221 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.222 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.222 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.498138+0000", 2024-09-14T06:16:31.222 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1023000000000002e-05 2024-09-14T06:16:31.222 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.222 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.222 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.222 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.501719+0000", 2024-09-14T06:16:31.222 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0035809919999999999 2024-09-14T06:16:31.222 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.222 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.222 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.222 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.501834+0000", 2024-09-14T06:16:31.222 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00011468600000000001 2024-09-14T06:16:31.222 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.223 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.223 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.223 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.223 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.223 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:5132 3.6 e30/23 3:65fece57:::rbd_data.1152bd92e04f.0000000000000038:head v 30'113, mlcod=30'111)", 2024-09-14T06:16:31.223 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:28.527954+0000", 2024-09-14T06:16:31.223 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 2.6051839139999999, 2024-09-14T06:16:31.223 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036883520000000002, 2024-09-14T06:16:31.223 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.223 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.223 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.223 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.223 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.223 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.224 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.527954+0000", 2024-09-14T06:16:31.224 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.224 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.224 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.224 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.224 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.527954+0000", 2024-09-14T06:16:31.224 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.224 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.224 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.224 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.224 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.527956+0000", 2024-09-14T06:16:31.224 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.567e-06 2024-09-14T06:16:31.224 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.225 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.225 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.225 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.527969+0000", 2024-09-14T06:16:31.225 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.342e-05 2024-09-14T06:16:31.225 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.225 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.225 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.225 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.527994+0000", 2024-09-14T06:16:31.225 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4579e-05 2024-09-14T06:16:31.225 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.225 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.225 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.225 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.528010+0000", 2024-09-14T06:16:31.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6492999999999999e-05 2024-09-14T06:16:31.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.226 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.226 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.528139+0000", 2024-09-14T06:16:31.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000128443 2024-09-14T06:16:31.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.226 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.226 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.528192+0000", 2024-09-14T06:16:31.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3409999999999999e-05 2024-09-14T06:16:31.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.226 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.226 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.531492+0000", 2024-09-14T06:16:31.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003299705 2024-09-14T06:16:31.227 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.227 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.227 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.227 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.531643+0000", 2024-09-14T06:16:31.227 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015073500000000001 2024-09-14T06:16:31.227 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.227 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.227 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.227 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.227 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.227 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:5129 3.8 e30/23 3:115cee87:::rbd_data.1152bd92e04f.0000000000000035:head v 30'29, mlcod=30'26)", 2024-09-14T06:16:31.227 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:28.528225+0000", 2024-09-14T06:16:31.227 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 2.6049127429999999, 2024-09-14T06:16:31.227 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036040059999999999, 2024-09-14T06:16:31.228 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:31.228 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:31.228 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:31.228 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:31.228 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.228 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:31.228 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.528225+0000", 2024-09-14T06:16:31.228 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.228 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.228 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.228 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:31.228 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.528225+0000", 2024-09-14T06:16:31.228 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:31.228 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.229 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.229 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:31.229 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.528227+0000", 2024-09-14T06:16:31.229 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4869999999999999e-06 2024-09-14T06:16:31.229 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.229 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.229 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:31.229 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.528239+0000", 2024-09-14T06:16:31.229 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2293000000000001e-05 2024-09-14T06:16:31.229 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.229 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.229 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:31.229 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.528276+0000", 2024-09-14T06:16:31.229 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6455999999999998e-05 2024-09-14T06:16:31.230 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.230 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.230 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.230 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.528292+0000", 2024-09-14T06:16:31.230 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.679e-05 2024-09-14T06:16:31.230 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.230 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.230 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:31.230 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.528413+0000", 2024-09-14T06:16:31.230 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000120653 2024-09-14T06:16:31.231 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.231 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.231 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:31.231 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.528461+0000", 2024-09-14T06:16:31.231 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7926000000000002e-05 2024-09-14T06:16:31.231 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.231 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.231 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:31.231 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.531702+0000", 2024-09-14T06:16:31.231 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032410120000000001 2024-09-14T06:16:31.231 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:31.231 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:31.231 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:31.231 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:28.531829+0000", 2024-09-14T06:16:31.232 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000127389 2024-09-14T06:16:31.232 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.232 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.232 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.232 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:31.232 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:31.232 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:31.232 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:16:31.234 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:31.236+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:31.235 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:16:31.235 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:31.235 INFO:teuthology.orchestra.run.smithi039.stdout:671 punch from 0x1ab275b to 0x1abdc23, (0xb4c8 bytes) 2024-09-14T06:16:31.236 INFO:teuthology.orchestra.run.smithi039.stdout:672 read 0x1067bb7 thru 0x10755ec (0xda36 bytes) 2024-09-14T06:16:31.236 INFO:teuthology.orchestra.run.smithi039.stdout:673 read 0x420a7c thru 0x42adc2 (0xa347 bytes) 2024-09-14T06:16:31.236 INFO:teuthology.orchestra.run.smithi039.stdout:674 write 0xc71a3ed thru 0xc71d76c (0x3380 bytes) 2024-09-14T06:16:31.236 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:31.236 INFO:teuthology.orchestra.run.smithi039.stdout:676 write 0x7ab2665 thru 0x7ac04d8 (0xde74 bytes) 2024-09-14T06:16:31.236 INFO:teuthology.orchestra.run.smithi039.stdout:680 read 0xb0af93 thru 0xb107c4 (0x5832 bytes) 2024-09-14T06:16:31.236 INFO:teuthology.orchestra.run.smithi039.stdout:681 trunc from 0xc71d76d to 0x5025152 2024-09-14T06:16:31.236 INFO:teuthology.orchestra.run.smithi039.stdout:682 read 0x3bfe8e9 thru 0x3c04f94 (0x66ac bytes) 2024-09-14T06:16:31.237 INFO:teuthology.orchestra.run.smithi039.stdout:683 write 0x69cb7e0 thru 0x69cc27d (0xa9e bytes) 2024-09-14T06:16:31.253 INFO:teuthology.orchestra.run.smithi039.stdout:686 trunc from 0x69cc27e to 0xa242fe6 2024-09-14T06:16:31.262 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:16:31.263 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:16:31.263 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T06:16:31.263 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.000000 2024-09-14T06:16:31.263 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0 2024-09-14T06:16:31.264 INFO:teuthology.orchestra.run.smithi039.stdout:687 read 0x4ed0bca thru 0x4ed3a9f (0x2ed6 bytes) 2024-09-14T06:16:31.264 INFO:teuthology.orchestra.run.smithi039.stdout:688 read 0x425b0ce thru 0x42633cb (0x82fe bytes) 2024-09-14T06:16:31.264 INFO:teuthology.orchestra.run.smithi039.stdout:689 write 0x7370e0b thru 0x737a9a3 (0x9b99 bytes) 2024-09-14T06:16:31.270 INFO:teuthology.orchestra.run.smithi039.stdout:690 punch from 0x13a138f to 0x13ab55b, (0xa1cc bytes) 2024-09-14T06:16:31.281 INFO:teuthology.orchestra.run.smithi039.stdout:691 write 0xae463b5 thru 0xae48cf4 (0x2940 bytes) 2024-09-14T06:16:31.287 INFO:teuthology.orchestra.run.smithi039.stdout:693 read 0x4b7987c thru 0x4b85664 (0xbde9 bytes) 2024-09-14T06:16:31.288 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:31.288 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:31.288 INFO:teuthology.orchestra.run.smithi039.stdout:697 punch from 0x1629b2e to 0x16338e8, (0x9dba bytes) 2024-09-14T06:16:31.302 INFO:teuthology.orchestra.run.smithi039.stdout:698 trunc from 0xae48cf5 to 0x3850d68 2024-09-14T06:16:31.320 INFO:teuthology.orchestra.run.smithi039.stdout:699 punch from 0x226710b to 0x226c685, (0x557a bytes) 2024-09-14T06:16:31.320 INFO:teuthology.orchestra.run.smithi039.stdout:700 writesame 0x5319a3 thru 0x53e4e4 data_size 0x1(0xcb42 bytes) 2024-09-14T06:16:31.335 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:31.332+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:31.357 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:31.370 INFO:teuthology.orchestra.run.smithi039.stdout:703 writesame 0x5a3711a thru 0x5a3d552 data_size 0x1(0x6439 bytes) 2024-09-14T06:16:31.370 INFO:teuthology.orchestra.run.smithi039.stdout:704 punch from 0x2363df8 to 0x2368bf6, (0x4dfe bytes) 2024-09-14T06:16:31.370 INFO:teuthology.orchestra.run.smithi039.stdout:705 read 0x54aaed5 thru 0x54b3058 (0x8184 bytes) 2024-09-14T06:16:31.370 INFO:teuthology.orchestra.run.smithi039.stdout:706 punch from 0x1264e8d to 0x126e66e, (0x97e1 bytes) 2024-09-14T06:16:31.371 INFO:teuthology.orchestra.run.smithi039.stdout:707 clone 4 order 19 su 8192 sc 15 2024-09-14T06:16:31.378 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:31.378 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:31.378 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:31.378 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:31.396 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:16:31.436 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:31.436+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:31.537 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:31.536+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:31.566 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:31.567 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:31.567 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:31.567 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:31.567 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:31.584 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:16:31.632 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone3 from 0x5a3d553 (overlap 0x1eef093) to 0x128f6be 2024-09-14T06:16:31.638 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:31.636+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:31.711 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:31.711 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:31.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:31.711 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:31.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.712 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3614 3.f 3:f0701b33:::rbd_data.11528c180b7.0000000000000000:3 [read 3047424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.712 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.262033+0000", 2024-09-14T06:16:31.712 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4486836690000002, 2024-09-14T06:16:31.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016128916, 2024-09-14T06:16:31.712 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.712 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.712 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.712 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.712 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.712 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.712 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3614 2024-09-14T06:16:31.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.712 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262033+0000", 2024-09-14T06:16:31.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262033+0000", 2024-09-14T06:16:31.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262034+0000", 2024-09-14T06:16:31.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1230000000000001e-06 2024-09-14T06:16:31.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262040+0000", 2024-09-14T06:16:31.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8100000000000003e-06 2024-09-14T06:16:31.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262079+0000", 2024-09-14T06:16:31.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9520000000000001e-05 2024-09-14T06:16:31.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262097+0000", 2024-09-14T06:16:31.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8357000000000001e-05 2024-09-14T06:16:31.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.274879+0000", 2024-09-14T06:16:31.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.012781391 2024-09-14T06:16:31.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275121+0000", 2024-09-14T06:16:31.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00024238499999999999 2024-09-14T06:16:31.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.278162+0000", 2024-09-14T06:16:31.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030403299999999999 2024-09-14T06:16:31.716 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.716 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.716 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.717 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3618 3.f 3:f0701b33:::rbd_data.11528c180b7.0000000000000000:3 [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.717 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.262344+0000", 2024-09-14T06:16:31.717 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4483719280000003, 2024-09-14T06:16:31.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015084447000000001, 2024-09-14T06:16:31.717 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.717 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.717 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.717 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.717 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.717 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.718 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3618 2024-09-14T06:16:31.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.718 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262344+0000", 2024-09-14T06:16:31.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262344+0000", 2024-09-14T06:16:31.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262346+0000", 2024-09-14T06:16:31.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.657e-06 2024-09-14T06:16:31.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262352+0000", 2024-09-14T06:16:31.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.677e-06 2024-09-14T06:16:31.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262374+0000", 2024-09-14T06:16:31.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2089000000000001e-05 2024-09-14T06:16:31.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262391+0000", 2024-09-14T06:16:31.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.698e-05 2024-09-14T06:16:31.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277035+0000", 2024-09-14T06:16:31.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014643959999999999 2024-09-14T06:16:31.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277230+0000", 2024-09-14T06:16:31.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00019529899999999999 2024-09-14T06:16:31.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277429+0000", 2024-09-14T06:16:31.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00019878499999999999 2024-09-14T06:16:31.721 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.721 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.721 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.721 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3619 3.f 3:f0701b33:::rbd_data.11528c180b7.0000000000000000:3 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.721 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.262413+0000", 2024-09-14T06:16:31.721 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4483028789999999, 2024-09-14T06:16:31.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015481155, 2024-09-14T06:16:31.721 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.722 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.722 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.722 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.722 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.722 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.722 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3619 2024-09-14T06:16:31.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.722 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262413+0000", 2024-09-14T06:16:31.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262413+0000", 2024-09-14T06:16:31.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262415+0000", 2024-09-14T06:16:31.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.73e-06 2024-09-14T06:16:31.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262420+0000", 2024-09-14T06:16:31.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.254e-06 2024-09-14T06:16:31.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262448+0000", 2024-09-14T06:16:31.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7509999999999999e-05 2024-09-14T06:16:31.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262463+0000", 2024-09-14T06:16:31.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.471e-05 2024-09-14T06:16:31.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277501+0000", 2024-09-14T06:16:31.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015037913999999999 2024-09-14T06:16:31.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277700+0000", 2024-09-14T06:16:31.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000199211 2024-09-14T06:16:31.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277895+0000", 2024-09-14T06:16:31.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00019482599999999999 2024-09-14T06:16:31.726 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.726 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.726 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.726 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4373 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.726 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589054+0000", 2024-09-14T06:16:31.726 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.1216625599999999, 2024-09-14T06:16:31.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015113086, 2024-09-14T06:16:31.727 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.727 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.727 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.727 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.727 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.727 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.727 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4373 2024-09-14T06:16:31.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.727 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589054+0000", 2024-09-14T06:16:31.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589054+0000", 2024-09-14T06:16:31.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589054+0000", 2024-09-14T06:16:31.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6400000000000003e-07 2024-09-14T06:16:31.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589063+0000", 2024-09-14T06:16:31.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5699999999999993e-06 2024-09-14T06:16:31.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589069+0000", 2024-09-14T06:16:31.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6359999999999997e-06 2024-09-14T06:16:31.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589079+0000", 2024-09-14T06:16:31.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.9469999999999998e-06 2024-09-14T06:16:31.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604036+0000", 2024-09-14T06:16:31.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014956410999999999 2024-09-14T06:16:31.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604088+0000", 2024-09-14T06:16:31.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2163000000000002e-05 2024-09-14T06:16:31.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604167+0000", 2024-09-14T06:16:31.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8894999999999996e-05 2024-09-14T06:16:31.731 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.731 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.731 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.731 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4374 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 1146880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.731 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589089+0000", 2024-09-14T06:16:31.731 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.1216274229999996, 2024-09-14T06:16:31.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015225613000000001, 2024-09-14T06:16:31.731 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.732 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.732 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.732 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.732 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.732 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.732 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4374 2024-09-14T06:16:31.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.732 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589089+0000", 2024-09-14T06:16:31.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589089+0000", 2024-09-14T06:16:31.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589089+0000", 2024-09-14T06:16:31.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3099999999999998e-07 2024-09-14T06:16:31.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589092+0000", 2024-09-14T06:16:31.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5660000000000001e-06 2024-09-14T06:16:31.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589099+0000", 2024-09-14T06:16:31.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6649999999999997e-06 2024-09-14T06:16:31.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589104+0000", 2024-09-14T06:16:31.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8919999999999999e-06 2024-09-14T06:16:31.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604198+0000", 2024-09-14T06:16:31.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015094471999999999 2024-09-14T06:16:31.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604246+0000", 2024-09-14T06:16:31.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7936999999999999e-05 2024-09-14T06:16:31.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604315+0000", 2024-09-14T06:16:31.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8549999999999999e-05 2024-09-14T06:16:31.735 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.735 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.735 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.736 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4375 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.736 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589111+0000", 2024-09-14T06:16:31.736 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.1216051309999999, 2024-09-14T06:16:31.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015436236000000001, 2024-09-14T06:16:31.736 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.736 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.736 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.736 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.736 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.736 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.736 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4375 2024-09-14T06:16:31.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.736 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589111+0000", 2024-09-14T06:16:31.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589111+0000", 2024-09-14T06:16:31.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589112+0000", 2024-09-14T06:16:31.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7e-07 2024-09-14T06:16:31.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589114+0000", 2024-09-14T06:16:31.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0099999999999998e-06 2024-09-14T06:16:31.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589120+0000", 2024-09-14T06:16:31.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.438e-06 2024-09-14T06:16:31.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589125+0000", 2024-09-14T06:16:31.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6739999999999996e-06 2024-09-14T06:16:31.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604402+0000", 2024-09-14T06:16:31.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015276993000000001 2024-09-14T06:16:31.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604483+0000", 2024-09-14T06:16:31.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1384999999999994e-05 2024-09-14T06:16:31.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604547+0000", 2024-09-14T06:16:31.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4265999999999998e-05 2024-09-14T06:16:31.740 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.740 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.740 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.740 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4376 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.740 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589132+0000", 2024-09-14T06:16:31.740 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.1215841219999998, 2024-09-14T06:16:31.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015632553, 2024-09-14T06:16:31.740 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.740 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.740 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.740 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.741 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.741 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.741 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4376 2024-09-14T06:16:31.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.741 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589132+0000", 2024-09-14T06:16:31.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589132+0000", 2024-09-14T06:16:31.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589133+0000", 2024-09-14T06:16:31.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1600000000000001e-07 2024-09-14T06:16:31.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589135+0000", 2024-09-14T06:16:31.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.035e-06 2024-09-14T06:16:31.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589141+0000", 2024-09-14T06:16:31.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.516e-06 2024-09-14T06:16:31.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589146+0000", 2024-09-14T06:16:31.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9899999999999997e-06 2024-09-14T06:16:31.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604594+0000", 2024-09-14T06:16:31.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015447341 2024-09-14T06:16:31.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604677+0000", 2024-09-14T06:16:31.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2999999999999998e-05 2024-09-14T06:16:31.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604765+0000", 2024-09-14T06:16:31.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8154999999999999e-05 2024-09-14T06:16:31.744 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.744 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.744 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.745 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4377 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.745 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589153+0000", 2024-09-14T06:16:31.745 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.1215629419999997, 2024-09-14T06:16:31.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015855904000000001, 2024-09-14T06:16:31.745 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.745 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.745 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.745 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.745 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.745 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.745 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4377 2024-09-14T06:16:31.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.746 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589153+0000", 2024-09-14T06:16:31.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589153+0000", 2024-09-14T06:16:31.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589154+0000", 2024-09-14T06:16:31.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:16:31.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589156+0000", 2024-09-14T06:16:31.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8929999999999999e-06 2024-09-14T06:16:31.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589162+0000", 2024-09-14T06:16:31.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3879999999999997e-06 2024-09-14T06:16:31.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589167+0000", 2024-09-14T06:16:31.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7230000000000004e-06 2024-09-14T06:16:31.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604810+0000", 2024-09-14T06:16:31.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015642795000000001 2024-09-14T06:16:31.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604931+0000", 2024-09-14T06:16:31.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000121038 2024-09-14T06:16:31.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605009+0000", 2024-09-14T06:16:31.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8651000000000003e-05 2024-09-14T06:16:31.749 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.749 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.749 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.749 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4378 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.749 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589177+0000", 2024-09-14T06:16:31.749 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.1215392289999997, 2024-09-14T06:16:31.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015995853000000001, 2024-09-14T06:16:31.749 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.749 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.749 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.750 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.750 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.750 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.750 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4378 2024-09-14T06:16:31.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.750 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589177+0000", 2024-09-14T06:16:31.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589177+0000", 2024-09-14T06:16:31.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589178+0000", 2024-09-14T06:16:31.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3799999999999998e-07 2024-09-14T06:16:31.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589180+0000", 2024-09-14T06:16:31.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1890000000000001e-06 2024-09-14T06:16:31.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589186+0000", 2024-09-14T06:16:31.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5270000000000004e-06 2024-09-14T06:16:31.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589193+0000", 2024-09-14T06:16:31.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.285e-06 2024-09-14T06:16:31.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605046+0000", 2024-09-14T06:16:31.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015853678 2024-09-14T06:16:31.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605105+0000", 2024-09-14T06:16:31.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8829000000000002e-05 2024-09-14T06:16:31.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.753 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605173+0000", 2024-09-14T06:16:31.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7906999999999999e-05 2024-09-14T06:16:31.753 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.753 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.753 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.753 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4379 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 2129920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.753 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589200+0000", 2024-09-14T06:16:31.753 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.1215160979999999, 2024-09-14T06:16:31.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016178273, 2024-09-14T06:16:31.753 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.754 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.754 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.754 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.754 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.754 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.754 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4379 2024-09-14T06:16:31.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.754 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589200+0000", 2024-09-14T06:16:31.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589200+0000", 2024-09-14T06:16:31.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589201+0000", 2024-09-14T06:16:31.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8800000000000003e-07 2024-09-14T06:16:31.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589203+0000", 2024-09-14T06:16:31.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0329999999999998e-06 2024-09-14T06:16:31.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589209+0000", 2024-09-14T06:16:31.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3709999999999996e-06 2024-09-14T06:16:31.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589214+0000", 2024-09-14T06:16:31.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:16:31.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605216+0000", 2024-09-14T06:16:31.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016002267000000001 2024-09-14T06:16:31.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605290+0000", 2024-09-14T06:16:31.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3662e-05 2024-09-14T06:16:31.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605378+0000", 2024-09-14T06:16:31.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8529000000000006e-05 2024-09-14T06:16:31.757 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.757 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.757 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.758 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4380 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 2326528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.758 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589221+0000", 2024-09-14T06:16:31.758 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.1214949020000002, 2024-09-14T06:16:31.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016607628999999999, 2024-09-14T06:16:31.758 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.758 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.758 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.758 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.758 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.758 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.758 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4380 2024-09-14T06:16:31.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.758 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589221+0000", 2024-09-14T06:16:31.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589221+0000", 2024-09-14T06:16:31.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589222+0000", 2024-09-14T06:16:31.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6100000000000001e-07 2024-09-14T06:16:31.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589224+0000", 2024-09-14T06:16:31.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1619999999999998e-06 2024-09-14T06:16:31.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589230+0000", 2024-09-14T06:16:31.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4080000000000003e-06 2024-09-14T06:16:31.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589235+0000", 2024-09-14T06:16:31.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:16:31.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605688+0000", 2024-09-14T06:16:31.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016452784000000002 2024-09-14T06:16:31.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605761+0000", 2024-09-14T06:16:31.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3702999999999996e-05 2024-09-14T06:16:31.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605829+0000", 2024-09-14T06:16:31.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7576999999999994e-05 2024-09-14T06:16:31.762 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.762 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.762 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.762 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4381 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 2523136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.762 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589243+0000", 2024-09-14T06:16:31.762 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.1214736859999999, 2024-09-14T06:16:31.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016792926999999999, 2024-09-14T06:16:31.762 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.762 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.762 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.763 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.763 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.763 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.763 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4381 2024-09-14T06:16:31.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.763 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589243+0000", 2024-09-14T06:16:31.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589243+0000", 2024-09-14T06:16:31.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589243+0000", 2024-09-14T06:16:31.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0900000000000002e-07 2024-09-14T06:16:31.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589245+0000", 2024-09-14T06:16:31.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2450000000000001e-06 2024-09-14T06:16:31.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589252+0000", 2024-09-14T06:16:31.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6699999999999997e-06 2024-09-14T06:16:31.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589259+0000", 2024-09-14T06:16:31.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0400000000000004e-06 2024-09-14T06:16:31.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605876+0000", 2024-09-14T06:16:31.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016617226999999998 2024-09-14T06:16:31.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605956+0000", 2024-09-14T06:16:31.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9430000000000004e-05 2024-09-14T06:16:31.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606036+0000", 2024-09-14T06:16:31.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9806000000000001e-05 2024-09-14T06:16:31.766 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.766 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.766 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.766 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4382 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 2719744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.766 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589267+0000", 2024-09-14T06:16:31.766 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.121449439, 2024-09-14T06:16:31.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016998154000000001, 2024-09-14T06:16:31.767 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.767 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.767 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.767 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.767 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.767 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.767 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4382 2024-09-14T06:16:31.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.767 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589267+0000", 2024-09-14T06:16:31.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589267+0000", 2024-09-14T06:16:31.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589267+0000", 2024-09-14T06:16:31.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7199999999999999e-07 2024-09-14T06:16:31.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589269+0000", 2024-09-14T06:16:31.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.956e-06 2024-09-14T06:16:31.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589277+0000", 2024-09-14T06:16:31.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9570000000000002e-06 2024-09-14T06:16:31.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589296+0000", 2024-09-14T06:16:31.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9185e-05 2024-09-14T06:16:31.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606080+0000", 2024-09-14T06:16:31.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016783513 2024-09-14T06:16:31.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606153+0000", 2024-09-14T06:16:31.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2622999999999999e-05 2024-09-14T06:16:31.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606265+0000", 2024-09-14T06:16:31.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000112448 2024-09-14T06:16:31.770 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.770 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.770 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.771 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4383 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 2916352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.771 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589306+0000", 2024-09-14T06:16:31.771 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.1214105390000002, 2024-09-14T06:16:31.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017123392000000001, 2024-09-14T06:16:31.771 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.771 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.771 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.771 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.771 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.771 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.771 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4383 2024-09-14T06:16:31.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.772 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589306+0000", 2024-09-14T06:16:31.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589306+0000", 2024-09-14T06:16:31.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589306+0000", 2024-09-14T06:16:31.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:16:31.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589313+0000", 2024-09-14T06:16:31.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1130000000000003e-06 2024-09-14T06:16:31.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589321+0000", 2024-09-14T06:16:31.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3370000000000002e-06 2024-09-14T06:16:31.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589326+0000", 2024-09-14T06:16:31.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.48e-06 2024-09-14T06:16:31.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606319+0000", 2024-09-14T06:16:31.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016993063999999999 2024-09-14T06:16:31.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606380+0000", 2024-09-14T06:16:31.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0921000000000002e-05 2024-09-14T06:16:31.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606429+0000", 2024-09-14T06:16:31.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8965000000000003e-05 2024-09-14T06:16:31.775 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.775 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.775 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.775 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4384 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.775 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589335+0000", 2024-09-14T06:16:31.775 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.1213816510000001, 2024-09-14T06:16:31.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017257641000000001, 2024-09-14T06:16:31.775 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.775 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.776 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.776 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.776 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.776 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.776 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4384 2024-09-14T06:16:31.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.776 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589335+0000", 2024-09-14T06:16:31.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589335+0000", 2024-09-14T06:16:31.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589335+0000", 2024-09-14T06:16:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:16:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589344+0000", 2024-09-14T06:16:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6319999999999997e-06 2024-09-14T06:16:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589351+0000", 2024-09-14T06:16:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1590000000000004e-06 2024-09-14T06:16:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589356+0000", 2024-09-14T06:16:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3070000000000002e-06 2024-09-14T06:16:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606463+0000", 2024-09-14T06:16:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017106857 2024-09-14T06:16:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606534+0000", 2024-09-14T06:16:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0728000000000005e-05 2024-09-14T06:16:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606592+0000", 2024-09-14T06:16:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8508000000000003e-05 2024-09-14T06:16:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4385 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589365+0000", 2024-09-14T06:16:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.1213512249999997, 2024-09-14T06:16:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017375774, 2024-09-14T06:16:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4385 2024-09-14T06:16:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589365+0000", 2024-09-14T06:16:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589365+0000", 2024-09-14T06:16:31.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589366+0000", 2024-09-14T06:16:31.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:16:31.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589368+0000", 2024-09-14T06:16:31.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.199e-06 2024-09-14T06:16:31.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589376+0000", 2024-09-14T06:16:31.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3720000000000005e-06 2024-09-14T06:16:31.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589381+0000", 2024-09-14T06:16:31.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2850000000000002e-06 2024-09-14T06:16:31.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606641+0000", 2024-09-14T06:16:31.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017259872999999998 2024-09-14T06:16:31.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606692+0000", 2024-09-14T06:16:31.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0803000000000001e-05 2024-09-14T06:16:31.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606741+0000", 2024-09-14T06:16:31.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8733999999999998e-05 2024-09-14T06:16:31.783 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.783 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.783 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.783 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4386 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.783 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589389+0000", 2024-09-14T06:16:31.783 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.1213270829999997, 2024-09-14T06:16:31.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017485302000000001, 2024-09-14T06:16:31.784 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.784 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.784 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.784 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.784 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.784 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.784 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4386 2024-09-14T06:16:31.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.784 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589389+0000", 2024-09-14T06:16:31.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589389+0000", 2024-09-14T06:16:31.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589390+0000", 2024-09-14T06:16:31.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6199999999999998e-07 2024-09-14T06:16:31.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589392+0000", 2024-09-14T06:16:31.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.968e-06 2024-09-14T06:16:31.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589398+0000", 2024-09-14T06:16:31.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6000000000000003e-06 2024-09-14T06:16:31.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589403+0000", 2024-09-14T06:16:31.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0390000000000004e-06 2024-09-14T06:16:31.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606773+0000", 2024-09-14T06:16:31.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017369643000000001 2024-09-14T06:16:31.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.787 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606824+0000", 2024-09-14T06:16:31.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0871999999999997e-05 2024-09-14T06:16:31.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.787 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606875+0000", 2024-09-14T06:16:31.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0717999999999998e-05 2024-09-14T06:16:31.787 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.787 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.787 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.788 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4387 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.788 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589411+0000", 2024-09-14T06:16:31.788 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.1213055189999999, 2024-09-14T06:16:31.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017590590999999999, 2024-09-14T06:16:31.788 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.788 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.788 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.788 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.788 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.788 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.788 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4387 2024-09-14T06:16:31.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.788 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589411+0000", 2024-09-14T06:16:31.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589411+0000", 2024-09-14T06:16:31.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589411+0000", 2024-09-14T06:16:31.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4299999999999998e-07 2024-09-14T06:16:31.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589413+0000", 2024-09-14T06:16:31.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1339999999999998e-06 2024-09-14T06:16:31.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589420+0000", 2024-09-14T06:16:31.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6059999999999999e-06 2024-09-14T06:16:31.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589425+0000", 2024-09-14T06:16:31.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0409999999999997e-06 2024-09-14T06:16:31.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.791 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.791 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606904+0000", 2024-09-14T06:16:31.791 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017479416000000001 2024-09-14T06:16:31.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.791 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.791 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606955+0000", 2024-09-14T06:16:31.791 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0853000000000003e-05 2024-09-14T06:16:31.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.791 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.791 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607001+0000", 2024-09-14T06:16:31.791 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6097999999999998e-05 2024-09-14T06:16:31.791 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.791 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.792 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.792 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4388 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.792 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589432+0000", 2024-09-14T06:16:31.792 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.1212838989999998, 2024-09-14T06:16:31.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017698337000000001, 2024-09-14T06:16:31.792 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.792 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.792 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.792 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.792 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.792 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.792 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4388 2024-09-14T06:16:31.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.793 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589432+0000", 2024-09-14T06:16:31.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589432+0000", 2024-09-14T06:16:31.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589433+0000", 2024-09-14T06:16:31.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2300000000000002e-07 2024-09-14T06:16:31.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589435+0000", 2024-09-14T06:16:31.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.902e-06 2024-09-14T06:16:31.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589441+0000", 2024-09-14T06:16:31.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2060000000000004e-06 2024-09-14T06:16:31.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589446+0000", 2024-09-14T06:16:31.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6500000000000004e-06 2024-09-14T06:16:31.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607037+0000", 2024-09-14T06:16:31.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017591496000000002 2024-09-14T06:16:31.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607084+0000", 2024-09-14T06:16:31.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6873999999999999e-05 2024-09-14T06:16:31.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607131+0000", 2024-09-14T06:16:31.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6786000000000002e-05 2024-09-14T06:16:31.796 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.796 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.796 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.796 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4389 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:31.796 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589453+0000", 2024-09-14T06:16:31.796 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.1212633969999999, 2024-09-14T06:16:31.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017801510999999999, 2024-09-14T06:16:31.796 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:31.796 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:31.796 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:31.796 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:31.797 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:31.797 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:31.797 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4389 2024-09-14T06:16:31.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.797 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:31.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:31.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589453+0000", 2024-09-14T06:16:31.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:31.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589453+0000", 2024-09-14T06:16:31.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:31.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:31.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589453+0000", 2024-09-14T06:16:31.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4499999999999997e-07 2024-09-14T06:16:31.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:31.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589455+0000", 2024-09-14T06:16:31.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0370000000000001e-06 2024-09-14T06:16:31.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:31.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589462+0000", 2024-09-14T06:16:31.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1360000000000002e-06 2024-09-14T06:16:31.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:31.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589466+0000", 2024-09-14T06:16:31.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.843e-06 2024-09-14T06:16:31.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:31.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607149+0000", 2024-09-14T06:16:31.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017682356 2024-09-14T06:16:31.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:31.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607209+0000", 2024-09-14T06:16:31.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9902999999999999e-05 2024-09-14T06:16:31.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:31.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:31.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:31.800 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607254+0000", 2024-09-14T06:16:31.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5791000000000003e-05 2024-09-14T06:16:31.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.800 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:31.800 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:31.800 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:31.802 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:16:31.802 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-14T06:16:31.803 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:31.796+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:31.861 INFO:teuthology.orchestra.run.smithi039.stdout:709 write 0xc2056e7 thru 0xc20d4c7 (0x7de1 bytes) 2024-09-14T06:16:31.889 INFO:teuthology.orchestra.run.smithi039.stdout:710 write 0xd19b066 thru 0xd19e0e1 (0x307c bytes) 2024-09-14T06:16:31.897 INFO:teuthology.orchestra.run.smithi039.stdout:712 read 0x86ef956 thru 0x86f4b02 (0x51ad bytes) 2024-09-14T06:16:31.897 INFO:teuthology.orchestra.run.smithi039.stdout:713 read 0x95f91d4 thru 0x95fec61 (0x5a8e bytes) 2024-09-14T06:16:31.897 INFO:teuthology.orchestra.run.smithi039.stdout:714 punch from 0xaa3692f to 0xaa42829, (0xbefa bytes) 2024-09-14T06:16:31.897 INFO:teuthology.orchestra.run.smithi039.stdout:715 clone 5 order 23 su 65536 sc 2 2024-09-14T06:16:31.902 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:31.900+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:31.980 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:31.980 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:31.981 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:31.981 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:31.994 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:16:32.039 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:32.036+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:32.105 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:32.104+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:32.166 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:32.166 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:32.166 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:32.166 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:32.166 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:32.180 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:16:32.226 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:32.224+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:32.306 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:32.300+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:32.308 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:32.308 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:32.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:32.309 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:32.309 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.309 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.309 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:32.309 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.228829749999999, 2024-09-14T06:16:32.309 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.56009052699999995, 2024-09-14T06:16:32.309 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.309 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.309 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.309 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.309 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.309 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.310 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 1 2024-09-14T06:16:32.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.310 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.310 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.310 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:32.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.310 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.310 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:32.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.311 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.311 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077065+0000", 2024-09-14T06:16:32.311 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6429999999999999e-06 2024-09-14T06:16:32.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.311 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.311 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077079+0000", 2024-09-14T06:16:32.311 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4447000000000001e-05 2024-09-14T06:16:32.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.311 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.311 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077103+0000", 2024-09-14T06:16:32.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3856e-05 2024-09-14T06:16:32.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.312 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077120+0000", 2024-09-14T06:16:32.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7337e-05 2024-09-14T06:16:32.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.312 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077255+0000", 2024-09-14T06:16:32.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000134652 2024-09-14T06:16:32.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.312 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:16:32.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077274+0000", 2024-09-14T06:16:32.313 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9003000000000001e-05 2024-09-14T06:16:32.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.313 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.313 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634272+0000", 2024-09-14T06:16:32.313 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.55699813099999995 2024-09-14T06:16:32.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.313 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.313 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634591+0000", 2024-09-14T06:16:32.313 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031910800000000002 2024-09-14T06:16:32.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.313 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.314 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634892+0000", 2024-09-14T06:16:32.314 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00030066400000000002 2024-09-14T06:16:32.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.314 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.314 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.314 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636345+0000", 2024-09-14T06:16:32.314 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001452896 2024-09-14T06:16:32.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.314 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.314 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.314 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636974+0000", 2024-09-14T06:16:32.314 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00062927800000000004 2024-09-14T06:16:32.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.314 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.314 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.315 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637071+0000", 2024-09-14T06:16:32.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7178999999999999e-05 2024-09-14T06:16:32.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.315 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.315 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637153+0000", 2024-09-14T06:16:32.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1333000000000002e-05 2024-09-14T06:16:32.315 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.315 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.315 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.315 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.315 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.674573+0000", 2024-09-14T06:16:32.315 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.631318448999998, 2024-09-14T06:16:32.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0096001760000000002, 2024-09-14T06:16:32.316 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.316 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.316 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.316 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.316 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.316 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.316 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 25 2024-09-14T06:16:32.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.316 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.316 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.316 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.674573+0000", 2024-09-14T06:16:32.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.674573+0000", 2024-09-14T06:16:32.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.674574+0000", 2024-09-14T06:16:32.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0869999999999999e-06 2024-09-14T06:16:32.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.674582+0000", 2024-09-14T06:16:32.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5329999999999997e-06 2024-09-14T06:16:32.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.674595+0000", 2024-09-14T06:16:32.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2767e-05 2024-09-14T06:16:32.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.674624+0000", 2024-09-14T06:16:32.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8946e-05 2024-09-14T06:16:32.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.679074+0000", 2024-09-14T06:16:32.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044506659999999998 2024-09-14T06:16:32.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.319 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.319 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.679256+0000", 2024-09-14T06:16:32.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000181845 2024-09-14T06:16:32.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.319 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.319 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.679545+0000", 2024-09-14T06:16:32.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00028854800000000002 2024-09-14T06:16:32.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.319 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.319 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.680986+0000", 2024-09-14T06:16:32.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001441152 2024-09-14T06:16:32.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.684029+0000", 2024-09-14T06:16:32.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030426060000000002 2024-09-14T06:16:32.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.684111+0000", 2024-09-14T06:16:32.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2109999999999998e-05 2024-09-14T06:16:32.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.684174+0000", 2024-09-14T06:16:32.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2916000000000005e-05 2024-09-14T06:16:32.321 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.321 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.321 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.321 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:41 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.321 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.774917+0000", 2024-09-14T06:16:32.321 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.530974965999999, 2024-09-14T06:16:32.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0097415190000000006, 2024-09-14T06:16:32.321 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.321 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.321 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.321 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.321 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.322 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.322 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 41 2024-09-14T06:16:32.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.322 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.774917+0000", 2024-09-14T06:16:32.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.774917+0000", 2024-09-14T06:16:32.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.323 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.774918+0000", 2024-09-14T06:16:32.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6400000000000001e-07 2024-09-14T06:16:32.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.323 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.774988+0000", 2024-09-14T06:16:32.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0104999999999994e-05 2024-09-14T06:16:32.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.323 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.774999+0000", 2024-09-14T06:16:32.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0794e-05 2024-09-14T06:16:32.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.324 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775006+0000", 2024-09-14T06:16:32.324 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6149999999999999e-06 2024-09-14T06:16:32.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.324 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.324 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776063+0000", 2024-09-14T06:16:32.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0010569049999999999 2024-09-14T06:16:32.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.325 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.325 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776160+0000", 2024-09-14T06:16:32.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6707999999999996e-05 2024-09-14T06:16:32.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.325 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.325 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776311+0000", 2024-09-14T06:16:32.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000151447 2024-09-14T06:16:32.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.777904+0000", 2024-09-14T06:16:32.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001592287 2024-09-14T06:16:32.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784597+0000", 2024-09-14T06:16:32.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0066931999999999998 2024-09-14T06:16:32.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784632+0000", 2024-09-14T06:16:32.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5524000000000001e-05 2024-09-14T06:16:32.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.327 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.327 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784658+0000", 2024-09-14T06:16:32.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6169999999999998e-05 2024-09-14T06:16:32.327 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.327 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.327 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.327 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.327 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.775024+0000", 2024-09-14T06:16:32.327 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.530868258000002, 2024-09-14T06:16:32.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0097879990000000004, 2024-09-14T06:16:32.327 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.327 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.327 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.328 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.328 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.328 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.328 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 42 2024-09-14T06:16:32.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.328 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775024+0000", 2024-09-14T06:16:32.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775024+0000", 2024-09-14T06:16:32.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.329 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.329 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775024+0000", 2024-09-14T06:16:32.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4499999999999996e-07 2024-09-14T06:16:32.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.329 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.329 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775030+0000", 2024-09-14T06:16:32.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4190000000000001e-06 2024-09-14T06:16:32.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.329 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.329 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775038+0000", 2024-09-14T06:16:32.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5869999999999993e-06 2024-09-14T06:16:32.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775045+0000", 2024-09-14T06:16:32.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2129999999999996e-06 2024-09-14T06:16:32.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776851+0000", 2024-09-14T06:16:32.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001806156 2024-09-14T06:16:32.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776901+0000", 2024-09-14T06:16:32.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0386999999999997e-05 2024-09-14T06:16:32.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.777032+0000", 2024-09-14T06:16:32.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013111500000000001 2024-09-14T06:16:32.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.780022+0000", 2024-09-14T06:16:32.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029898500000000001 2024-09-14T06:16:32.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784709+0000", 2024-09-14T06:16:32.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046872830000000004 2024-09-14T06:16:32.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784768+0000", 2024-09-14T06:16:32.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8541000000000001e-05 2024-09-14T06:16:32.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784812+0000", 2024-09-14T06:16:32.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3603000000000002e-05 2024-09-14T06:16:32.332 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.332 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.332 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.332 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.333 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.775688+0000", 2024-09-14T06:16:32.333 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.530203728, 2024-09-14T06:16:32.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0093361539999999993, 2024-09-14T06:16:32.333 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.333 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.333 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.333 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.333 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.333 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.333 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 44 2024-09-14T06:16:32.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.333 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.333 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775688+0000", 2024-09-14T06:16:32.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.334 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775688+0000", 2024-09-14T06:16:32.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.334 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775689+0000", 2024-09-14T06:16:32.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5600000000000005e-07 2024-09-14T06:16:32.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775743+0000", 2024-09-14T06:16:32.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3687999999999998e-05 2024-09-14T06:16:32.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775752+0000", 2024-09-14T06:16:32.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0699999999999996e-06 2024-09-14T06:16:32.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775759+0000", 2024-09-14T06:16:32.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2099999999999996e-06 2024-09-14T06:16:32.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.777956+0000", 2024-09-14T06:16:32.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0021972390000000001 2024-09-14T06:16:32.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.778013+0000", 2024-09-14T06:16:32.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6718999999999997e-05 2024-09-14T06:16:32.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.778103+0000", 2024-09-14T06:16:32.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0085000000000002e-05 2024-09-14T06:16:32.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.337 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.337 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781093+0000", 2024-09-14T06:16:32.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029905420000000001 2024-09-14T06:16:32.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.337 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.337 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784949+0000", 2024-09-14T06:16:32.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003855448 2024-09-14T06:16:32.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.337 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784983+0000", 2024-09-14T06:16:32.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4350999999999997e-05 2024-09-14T06:16:32.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.785024+0000", 2024-09-14T06:16:32.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1046e-05 2024-09-14T06:16:32.338 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.338 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.338 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.338 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.338 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.775774+0000", 2024-09-14T06:16:32.339 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.530117878999999, 2024-09-14T06:16:32.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011041741000000001, 2024-09-14T06:16:32.339 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.339 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.339 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.339 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.339 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.339 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.339 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 45 2024-09-14T06:16:32.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.339 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775774+0000", 2024-09-14T06:16:32.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775774+0000", 2024-09-14T06:16:32.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775774+0000", 2024-09-14T06:16:32.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1900000000000003e-07 2024-09-14T06:16:32.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775778+0000", 2024-09-14T06:16:32.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6270000000000002e-06 2024-09-14T06:16:32.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775784+0000", 2024-09-14T06:16:32.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5559999999999998e-06 2024-09-14T06:16:32.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775788+0000", 2024-09-14T06:16:32.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9099999999999998e-06 2024-09-14T06:16:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.778575+0000", 2024-09-14T06:16:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027878040000000001 2024-09-14T06:16:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.778626+0000", 2024-09-14T06:16:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0791000000000002e-05 2024-09-14T06:16:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.778716+0000", 2024-09-14T06:16:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9466e-05 2024-09-14T06:16:32.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781098+0000", 2024-09-14T06:16:32.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023824900000000001 2024-09-14T06:16:32.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.786764+0000", 2024-09-14T06:16:32.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056663199999999999 2024-09-14T06:16:32.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.786793+0000", 2024-09-14T06:16:32.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9008000000000001e-05 2024-09-14T06:16:32.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.344 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.344 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.786816+0000", 2024-09-14T06:16:32.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2249999999999999e-05 2024-09-14T06:16:32.344 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.344 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.344 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.344 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.344 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.775864+0000", 2024-09-14T06:16:32.344 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.530027728, 2024-09-14T06:16:32.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011089144, 2024-09-14T06:16:32.345 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.345 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.345 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.345 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.345 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.345 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.345 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 46 2024-09-14T06:16:32.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.345 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.345 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.345 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775864+0000", 2024-09-14T06:16:32.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775864+0000", 2024-09-14T06:16:32.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775865+0000", 2024-09-14T06:16:32.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.111e-06 2024-09-14T06:16:32.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775872+0000", 2024-09-14T06:16:32.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9240000000000001e-06 2024-09-14T06:16:32.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775884+0000", 2024-09-14T06:16:32.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1409e-05 2024-09-14T06:16:32.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775890+0000", 2024-09-14T06:16:32.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9900000000000002e-06 2024-09-14T06:16:32.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.779024+0000", 2024-09-14T06:16:32.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031343120000000002 2024-09-14T06:16:32.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.779073+0000", 2024-09-14T06:16:32.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8856000000000001e-05 2024-09-14T06:16:32.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.779152+0000", 2024-09-14T06:16:32.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9308999999999996e-05 2024-09-14T06:16:32.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781102+0000", 2024-09-14T06:16:32.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001950139 2024-09-14T06:16:32.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.786906+0000", 2024-09-14T06:16:32.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0058036299999999997 2024-09-14T06:16:32.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.786933+0000", 2024-09-14T06:16:32.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7226999999999999e-05 2024-09-14T06:16:32.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.786953+0000", 2024-09-14T06:16:32.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0237e-05 2024-09-14T06:16:32.349 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.350 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.350 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.350 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.350 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.775904+0000", 2024-09-14T06:16:32.350 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.529987943999998, 2024-09-14T06:16:32.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011120666, 2024-09-14T06:16:32.350 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.350 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.350 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.350 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.350 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.350 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.351 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 47 2024-09-14T06:16:32.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.351 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.351 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.351 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775904+0000", 2024-09-14T06:16:32.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.351 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.351 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775904+0000", 2024-09-14T06:16:32.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775904+0000", 2024-09-14T06:16:32.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3499999999999996e-07 2024-09-14T06:16:32.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775965+0000", 2024-09-14T06:16:32.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0340999999999997e-05 2024-09-14T06:16:32.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775978+0000", 2024-09-14T06:16:32.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2724999999999999e-05 2024-09-14T06:16:32.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.775986+0000", 2024-09-14T06:16:32.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.157e-06 2024-09-14T06:16:32.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.779449+0000", 2024-09-14T06:16:32.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034635310000000002 2024-09-14T06:16:32.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.779501+0000", 2024-09-14T06:16:32.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2135999999999997e-05 2024-09-14T06:16:32.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.779591+0000", 2024-09-14T06:16:32.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9520999999999999e-05 2024-09-14T06:16:32.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.782131+0000", 2024-09-14T06:16:32.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0025397900000000001 2024-09-14T06:16:32.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.786979+0000", 2024-09-14T06:16:32.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048479939999999996 2024-09-14T06:16:32.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787005+0000", 2024-09-14T06:16:32.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6224000000000002e-05 2024-09-14T06:16:32.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787025+0000", 2024-09-14T06:16:32.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9712e-05 2024-09-14T06:16:32.355 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.355 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.355 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.355 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.355 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.776003+0000", 2024-09-14T06:16:32.355 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.529888855999999, 2024-09-14T06:16:32.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011128235, 2024-09-14T06:16:32.356 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.356 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.356 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.356 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.356 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.356 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.356 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48 2024-09-14T06:16:32.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.356 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.356 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776003+0000", 2024-09-14T06:16:32.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776003+0000", 2024-09-14T06:16:32.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776004+0000", 2024-09-14T06:16:32.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6400000000000002e-07 2024-09-14T06:16:32.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776009+0000", 2024-09-14T06:16:32.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3299999999999998e-06 2024-09-14T06:16:32.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776019+0000", 2024-09-14T06:16:32.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0334e-05 2024-09-14T06:16:32.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776026+0000", 2024-09-14T06:16:32.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.888e-06 2024-09-14T06:16:32.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.780040+0000", 2024-09-14T06:16:32.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040138769999999999 2024-09-14T06:16:32.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.780109+0000", 2024-09-14T06:16:32.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8901999999999999e-05 2024-09-14T06:16:32.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.780225+0000", 2024-09-14T06:16:32.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000116154 2024-09-14T06:16:32.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.782137+0000", 2024-09-14T06:16:32.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0019123230000000001 2024-09-14T06:16:32.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787082+0000", 2024-09-14T06:16:32.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0049447049999999998 2024-09-14T06:16:32.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787111+0000", 2024-09-14T06:16:32.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9241e-05 2024-09-14T06:16:32.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787131+0000", 2024-09-14T06:16:32.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9817e-05 2024-09-14T06:16:32.360 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.360 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.360 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.361 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.361 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.776059+0000", 2024-09-14T06:16:32.361 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.529832821999999, 2024-09-14T06:16:32.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011196868, 2024-09-14T06:16:32.361 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.361 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.361 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.361 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.361 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.361 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.361 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 49 2024-09-14T06:16:32.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.361 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776059+0000", 2024-09-14T06:16:32.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776059+0000", 2024-09-14T06:16:32.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776060+0000", 2024-09-14T06:16:32.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9500000000000001e-07 2024-09-14T06:16:32.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776068+0000", 2024-09-14T06:16:32.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0059999999999992e-06 2024-09-14T06:16:32.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776079+0000", 2024-09-14T06:16:32.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0929e-05 2024-09-14T06:16:32.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776086+0000", 2024-09-14T06:16:32.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9360000000000002e-06 2024-09-14T06:16:32.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.780652+0000", 2024-09-14T06:16:32.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045665189999999998 2024-09-14T06:16:32.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.780716+0000", 2024-09-14T06:16:32.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3993999999999998e-05 2024-09-14T06:16:32.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.780811+0000", 2024-09-14T06:16:32.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4555000000000005e-05 2024-09-14T06:16:32.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.783904+0000", 2024-09-14T06:16:32.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030928459999999998 2024-09-14T06:16:32.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787205+0000", 2024-09-14T06:16:32.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033015560000000002 2024-09-14T06:16:32.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787234+0000", 2024-09-14T06:16:32.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8856999999999998e-05 2024-09-14T06:16:32.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787256+0000", 2024-09-14T06:16:32.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1875e-05 2024-09-14T06:16:32.366 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.366 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.366 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.366 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.366 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.776122+0000", 2024-09-14T06:16:32.366 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.529770065000001, 2024-09-14T06:16:32.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011260246, 2024-09-14T06:16:32.366 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.366 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.367 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.367 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.367 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.367 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.367 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50 2024-09-14T06:16:32.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.367 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776122+0000", 2024-09-14T06:16:32.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776122+0000", 2024-09-14T06:16:32.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776123+0000", 2024-09-14T06:16:32.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7900000000000007e-07 2024-09-14T06:16:32.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776192+0000", 2024-09-14T06:16:32.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9509000000000001e-05 2024-09-14T06:16:32.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776207+0000", 2024-09-14T06:16:32.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5167000000000001e-05 2024-09-14T06:16:32.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776215+0000", 2024-09-14T06:16:32.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9549999999999992e-06 2024-09-14T06:16:32.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781137+0000", 2024-09-14T06:16:32.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0049216099999999999 2024-09-14T06:16:32.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.370 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.370 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781185+0000", 2024-09-14T06:16:32.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7763000000000003e-05 2024-09-14T06:16:32.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.370 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.370 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781270+0000", 2024-09-14T06:16:32.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5402000000000007e-05 2024-09-14T06:16:32.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.370 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.370 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784487+0000", 2024-09-14T06:16:32.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032170689999999999 2024-09-14T06:16:32.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.371 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787302+0000", 2024-09-14T06:16:32.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028146429999999999 2024-09-14T06:16:32.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.371 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787339+0000", 2024-09-14T06:16:32.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6958000000000001e-05 2024-09-14T06:16:32.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787382+0000", 2024-09-14T06:16:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3191e-05 2024-09-14T06:16:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.776232+0000", 2024-09-14T06:16:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.529659505000001, 2024-09-14T06:16:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011758912999999999, 2024-09-14T06:16:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.372 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.373 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.373 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.373 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 51 2024-09-14T06:16:32.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.373 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776232+0000", 2024-09-14T06:16:32.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776232+0000", 2024-09-14T06:16:32.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776233+0000", 2024-09-14T06:16:32.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8000000000000005e-07 2024-09-14T06:16:32.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776239+0000", 2024-09-14T06:16:32.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.553e-06 2024-09-14T06:16:32.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776247+0000", 2024-09-14T06:16:32.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:16:32.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776254+0000", 2024-09-14T06:16:32.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2980000000000003e-06 2024-09-14T06:16:32.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781687+0000", 2024-09-14T06:16:32.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0054321350000000003 2024-09-14T06:16:32.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781739+0000", 2024-09-14T06:16:32.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2719999999999997e-05 2024-09-14T06:16:32.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.781851+0000", 2024-09-14T06:16:32.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011188600000000001 2024-09-14T06:16:32.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784533+0000", 2024-09-14T06:16:32.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026823110000000002 2024-09-14T06:16:32.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787940+0000", 2024-09-14T06:16:32.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034064770000000002 2024-09-14T06:16:32.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787970+0000", 2024-09-14T06:16:32.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9626999999999999e-05 2024-09-14T06:16:32.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.787991+0000", 2024-09-14T06:16:32.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1662e-05 2024-09-14T06:16:32.377 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.377 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.776268+0000", 2024-09-14T06:16:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.529624241, 2024-09-14T06:16:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.012268451, 2024-09-14T06:16:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.378 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 52 2024-09-14T06:16:32.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.379 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776268+0000", 2024-09-14T06:16:32.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776268+0000", 2024-09-14T06:16:32.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776268+0000", 2024-09-14T06:16:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4700000000000001e-07 2024-09-14T06:16:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776274+0000", 2024-09-14T06:16:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3669999999999996e-06 2024-09-14T06:16:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776282+0000", 2024-09-14T06:16:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2369999999999992e-06 2024-09-14T06:16:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.776288+0000", 2024-09-14T06:16:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4450000000000001e-06 2024-09-14T06:16:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.782157+0000", 2024-09-14T06:16:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0058688430000000003 2024-09-14T06:16:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.782207+0000", 2024-09-14T06:16:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9468999999999997e-05 2024-09-14T06:16:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.382 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.782284+0000", 2024-09-14T06:16:32.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7368999999999997e-05 2024-09-14T06:16:32.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.382 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.784550+0000", 2024-09-14T06:16:32.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022660969999999999 2024-09-14T06:16:32.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.382 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.788458+0000", 2024-09-14T06:16:32.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0039080290000000004 2024-09-14T06:16:32.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.788495+0000", 2024-09-14T06:16:32.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6468999999999999e-05 2024-09-14T06:16:32.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.788536+0000", 2024-09-14T06:16:32.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1479e-05 2024-09-14T06:16:32.383 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.383 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.383 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.384 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.384 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:05.011523+0000", 2024-09-14T06:16:32.384 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.294368506000001, 2024-09-14T06:16:32.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011115544, 2024-09-14T06:16:32.384 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.384 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.384 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.384 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.384 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.384 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.384 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 70 2024-09-14T06:16:32.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.384 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011523+0000", 2024-09-14T06:16:32.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011523+0000", 2024-09-14T06:16:32.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011525+0000", 2024-09-14T06:16:32.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5239999999999999e-06 2024-09-14T06:16:32.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.386 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.386 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011533+0000", 2024-09-14T06:16:32.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4500000000000004e-06 2024-09-14T06:16:32.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.386 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.386 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011556+0000", 2024-09-14T06:16:32.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2858999999999999e-05 2024-09-14T06:16:32.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.386 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.386 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011573+0000", 2024-09-14T06:16:32.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7283e-05 2024-09-14T06:16:32.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.387 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.016587+0000", 2024-09-14T06:16:32.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050139349999999997 2024-09-14T06:16:32.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.387 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.016730+0000", 2024-09-14T06:16:32.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00014216200000000001 2024-09-14T06:16:32.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.387 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.016956+0000", 2024-09-14T06:16:32.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022678499999999999 2024-09-14T06:16:32.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.018945+0000", 2024-09-14T06:16:32.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00198832 2024-09-14T06:16:32.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.022306+0000", 2024-09-14T06:16:32.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033608420000000002 2024-09-14T06:16:32.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.022506+0000", 2024-09-14T06:16:32.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00019999899999999999 2024-09-14T06:16:32.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.022639+0000", 2024-09-14T06:16:32.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000133385 2024-09-14T06:16:32.389 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.389 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.389 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.389 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.390 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:05.011599+0000", 2024-09-14T06:16:32.390 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.294293076999999, 2024-09-14T06:16:32.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011527848, 2024-09-14T06:16:32.390 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.390 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.390 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.390 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.390 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.390 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.390 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 71 2024-09-14T06:16:32.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.390 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011599+0000", 2024-09-14T06:16:32.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.391 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.391 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011599+0000", 2024-09-14T06:16:32.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.391 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.391 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011600+0000", 2024-09-14T06:16:32.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.08e-06 2024-09-14T06:16:32.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.391 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011609+0000", 2024-09-14T06:16:32.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0399999999999998e-06 2024-09-14T06:16:32.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011624+0000", 2024-09-14T06:16:32.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4970000000000001e-05 2024-09-14T06:16:32.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011637+0000", 2024-09-14T06:16:32.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2724e-05 2024-09-14T06:16:32.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.017661+0000", 2024-09-14T06:16:32.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0060242919999999997 2024-09-14T06:16:32.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.017768+0000", 2024-09-14T06:16:32.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010715199999999999 2024-09-14T06:16:32.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.017953+0000", 2024-09-14T06:16:32.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00018450199999999999 2024-09-14T06:16:32.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.022730+0000", 2024-09-14T06:16:32.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047772839999999997 2024-09-14T06:16:32.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.022910+0000", 2024-09-14T06:16:32.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000180213 2024-09-14T06:16:32.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.022994+0000", 2024-09-14T06:16:32.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3518999999999998e-05 2024-09-14T06:16:32.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.023127+0000", 2024-09-14T06:16:32.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000133072 2024-09-14T06:16:32.395 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.395 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.395 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.395 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.395 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:05.011653+0000", 2024-09-14T06:16:32.395 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.294239106999999, 2024-09-14T06:16:32.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011747893000000001, 2024-09-14T06:16:32.396 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.396 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.396 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.396 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.396 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.396 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.396 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 72 2024-09-14T06:16:32.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.396 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.396 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.396 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011653+0000", 2024-09-14T06:16:32.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011653+0000", 2024-09-14T06:16:32.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011654+0000", 2024-09-14T06:16:32.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2399999999999997e-07 2024-09-14T06:16:32.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011842+0000", 2024-09-14T06:16:32.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000188841 2024-09-14T06:16:32.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011867+0000", 2024-09-14T06:16:32.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4122999999999999e-05 2024-09-14T06:16:32.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011880+0000", 2024-09-14T06:16:32.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.376e-05 2024-09-14T06:16:32.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.019044+0000", 2024-09-14T06:16:32.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0071632980000000002 2024-09-14T06:16:32.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.019237+0000", 2024-09-14T06:16:32.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000193792 2024-09-14T06:16:32.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.019565+0000", 2024-09-14T06:16:32.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032795399999999998 2024-09-14T06:16:32.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.022877+0000", 2024-09-14T06:16:32.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003311731 2024-09-14T06:16:32.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.400 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.400 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.023151+0000", 2024-09-14T06:16:32.400 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027372799999999999 2024-09-14T06:16:32.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.400 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.400 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.023316+0000", 2024-09-14T06:16:32.400 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000165353 2024-09-14T06:16:32.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.400 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.400 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.023401+0000", 2024-09-14T06:16:32.400 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4488999999999997e-05 2024-09-14T06:16:32.401 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.401 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.401 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.401 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.401 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:05.011934+0000", 2024-09-14T06:16:32.401 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.293958285999999, 2024-09-14T06:16:32.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013924748000000001, 2024-09-14T06:16:32.401 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.401 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.401 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.401 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.402 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.402 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.402 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 73 2024-09-14T06:16:32.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.402 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011934+0000", 2024-09-14T06:16:32.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011934+0000", 2024-09-14T06:16:32.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011935+0000", 2024-09-14T06:16:32.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.164e-06 2024-09-14T06:16:32.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011942+0000", 2024-09-14T06:16:32.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9630000000000001e-06 2024-09-14T06:16:32.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011960+0000", 2024-09-14T06:16:32.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8272999999999999e-05 2024-09-14T06:16:32.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011980+0000", 2024-09-14T06:16:32.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.959e-05 2024-09-14T06:16:32.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.020767+0000", 2024-09-14T06:16:32.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0087871249999999998 2024-09-14T06:16:32.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.021037+0000", 2024-09-14T06:16:32.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027065800000000001 2024-09-14T06:16:32.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.021412+0000", 2024-09-14T06:16:32.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00037428400000000001 2024-09-14T06:16:32.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.023453+0000", 2024-09-14T06:16:32.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002041181 2024-09-14T06:16:32.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025788+0000", 2024-09-14T06:16:32.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023351819999999999 2024-09-14T06:16:32.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025829+0000", 2024-09-14T06:16:32.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0951000000000001e-05 2024-09-14T06:16:32.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025858+0000", 2024-09-14T06:16:32.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9377e-05 2024-09-14T06:16:32.407 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.407 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.407 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.407 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.407 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:05.011997+0000", 2024-09-14T06:16:32.407 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.293894435999999, 2024-09-14T06:16:32.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013974349, 2024-09-14T06:16:32.407 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.407 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.407 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.407 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.408 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.408 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.408 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 74 2024-09-14T06:16:32.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.408 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011997+0000", 2024-09-14T06:16:32.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011997+0000", 2024-09-14T06:16:32.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.011998+0000", 2024-09-14T06:16:32.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9299999999999996e-07 2024-09-14T06:16:32.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012004+0000", 2024-09-14T06:16:32.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.037e-06 2024-09-14T06:16:32.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012016+0000", 2024-09-14T06:16:32.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2055999999999999e-05 2024-09-14T06:16:32.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012029+0000", 2024-09-14T06:16:32.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2787e-05 2024-09-14T06:16:32.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.023591+0000", 2024-09-14T06:16:32.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011562004000000001 2024-09-14T06:16:32.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.023732+0000", 2024-09-14T06:16:32.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000140679 2024-09-14T06:16:32.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.024001+0000", 2024-09-14T06:16:32.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00026883499999999998 2024-09-14T06:16:32.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025870+0000", 2024-09-14T06:16:32.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018691890000000001 2024-09-14T06:16:32.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025905+0000", 2024-09-14T06:16:32.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4805000000000003e-05 2024-09-14T06:16:32.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025941+0000", 2024-09-14T06:16:32.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6594000000000002e-05 2024-09-14T06:16:32.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025972+0000", 2024-09-14T06:16:32.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0470000000000001e-05 2024-09-14T06:16:32.412 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.413 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.413 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.413 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.413 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:05.012059+0000", 2024-09-14T06:16:32.413 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.293832720000001, 2024-09-14T06:16:32.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014493252, 2024-09-14T06:16:32.413 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.413 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.413 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.414 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.414 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 75 2024-09-14T06:16:32.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.414 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012059+0000", 2024-09-14T06:16:32.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012059+0000", 2024-09-14T06:16:32.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012060+0000", 2024-09-14T06:16:32.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.47e-07 2024-09-14T06:16:32.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012185+0000", 2024-09-14T06:16:32.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000125165 2024-09-14T06:16:32.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012209+0000", 2024-09-14T06:16:32.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3753000000000001e-05 2024-09-14T06:16:32.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012226+0000", 2024-09-14T06:16:32.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.664e-05 2024-09-14T06:16:32.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.024653+0000", 2024-09-14T06:16:32.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.012427493 2024-09-14T06:16:32.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.024714+0000", 2024-09-14T06:16:32.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0585999999999999e-05 2024-09-14T06:16:32.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.024803+0000", 2024-09-14T06:16:32.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9881000000000002e-05 2024-09-14T06:16:32.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.026474+0000", 2024-09-14T06:16:32.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001670405 2024-09-14T06:16:32.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.026490+0000", 2024-09-14T06:16:32.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6421000000000001e-05 2024-09-14T06:16:32.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.026526+0000", 2024-09-14T06:16:32.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5651000000000001e-05 2024-09-14T06:16:32.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.026552+0000", 2024-09-14T06:16:32.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.641e-05 2024-09-14T06:16:32.418 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.418 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.419 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.419 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:32.419 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:05.012255+0000", 2024-09-14T06:16:32.419 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.293636727999999, 2024-09-14T06:16:32.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015138888, 2024-09-14T06:16:32.419 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.419 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.419 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.419 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.419 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:32.419 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:32.419 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 76 2024-09-14T06:16:32.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.420 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012255+0000", 2024-09-14T06:16:32.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012255+0000", 2024-09-14T06:16:32.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012256+0000", 2024-09-14T06:16:32.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0899999999999999e-06 2024-09-14T06:16:32.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012263+0000", 2024-09-14T06:16:32.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2030000000000003e-06 2024-09-14T06:16:32.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012275+0000", 2024-09-14T06:16:32.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1883e-05 2024-09-14T06:16:32.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.012284+0000", 2024-09-14T06:16:32.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8000000000000004e-06 2024-09-14T06:16:32.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025348+0000", 2024-09-14T06:16:32.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013064157 2024-09-14T06:16:32.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025415+0000", 2024-09-14T06:16:32.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7152999999999999e-05 2024-09-14T06:16:32.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:32.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.025519+0000", 2024-09-14T06:16:32.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000104083 2024-09-14T06:16:32.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.026870+0000", 2024-09-14T06:16:32.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0013508089999999999 2024-09-14T06:16:32.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.027336+0000", 2024-09-14T06:16:32.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00046525199999999998 2024-09-14T06:16:32.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.027370+0000", 2024-09-14T06:16:32.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4128999999999998e-05 2024-09-14T06:16:32.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.425 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:05.027394+0000", 2024-09-14T06:16:32.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4329e-05 2024-09-14T06:16:32.425 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.425 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.425 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.425 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.425 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.425 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:32.427 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:16:32.427 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:32.424+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:32.527 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:32.524+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:32.532 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-09-14T06:16:32.540 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.2 primary-affinity to 0 (802) 2024-09-14T06:16:32.624 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:32.624 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:32.624 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:32.624 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:32.627 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:32.624+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:32.639 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:16:32.729 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:32.732+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:32.806 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:32.807 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:32.821 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:16:32.870 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:32.868+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:32.931 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:32.928+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:32.948 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.5 30/29 [PushOp(3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head, version: 28'5, data_included: [0~212992,360448~311296], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head@28'5, size: 1032192, copy_subset: [(0, 1032192)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1032192, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:16:32.948 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.6711710039999996, 2024-09-14T06:16:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041630249999999999, 2024-09-14T06:16:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.949 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275888+0000", 2024-09-14T06:16:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.367e-06 2024-09-14T06:16:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277098+0000", 2024-09-14T06:16:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001210661 2024-09-14T06:16:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.950 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277122+0000", 2024-09-14T06:16:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3413000000000001e-05 2024-09-14T06:16:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277141+0000", 2024-09-14T06:16:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9133000000000001e-05 2024-09-14T06:16:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277364+0000", 2024-09-14T06:16:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000223365 2024-09-14T06:16:32.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277405+0000", 2024-09-14T06:16:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1053000000000001e-05 2024-09-14T06:16:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.280049+0000", 2024-09-14T06:16:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026440330000000001 2024-09-14T06:16:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.952 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.16 30/29 [PushOp(3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head, version: 28'3, data_included: [0~212992,344064~327680], data_size: 540672, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head@28'3, size: 1032192, copy_subset: [(0, 1032192)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1032192, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:16:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.287894+0000", 2024-09-14T06:16:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.6591628109999998, 2024-09-14T06:16:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032539600000000002, 2024-09-14T06:16:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.287894+0000", 2024-09-14T06:16:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.953 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.287894+0000", 2024-09-14T06:16:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.287899+0000", 2024-09-14T06:16:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7280000000000003e-06 2024-09-14T06:16:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.288805+0000", 2024-09-14T06:16:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00090622700000000001 2024-09-14T06:16:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.288855+0000", 2024-09-14T06:16:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9835e-05 2024-09-14T06:16:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.288902+0000", 2024-09-14T06:16:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6854999999999998e-05 2024-09-14T06:16:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.289120+0000", 2024-09-14T06:16:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000217495 2024-09-14T06:16:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.289142+0000", 2024-09-14T06:16:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2870999999999998e-05 2024-09-14T06:16:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.291148+0000", 2024-09-14T06:16:32.956 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020059489999999999 2024-09-14T06:16:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1e 30/29 [PushOp(3:7890aae8:::rbd_data.1152bd92e04f.0000000000000015:head, version: 28'1, data_included: [0~212992,344064~311296], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:7890aae8:::rbd_data.1152bd92e04f.0000000000000015:head@28'1, size: 1032192, copy_subset: [(0, 1032192)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1032192, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:16:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.304572+0000", 2024-09-14T06:16:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.6424848939999999, 2024-09-14T06:16:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0025677849999999999, 2024-09-14T06:16:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.304572+0000", 2024-09-14T06:16:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.304572+0000", 2024-09-14T06:16:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.304573+0000", 2024-09-14T06:16:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9699999999999995e-07 2024-09-14T06:16:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.305701+0000", 2024-09-14T06:16:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001127454 2024-09-14T06:16:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.305738+0000", 2024-09-14T06:16:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7292000000000001e-05 2024-09-14T06:16:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.305774+0000", 2024-09-14T06:16:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5803999999999998e-05 2024-09-14T06:16:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.305967+0000", 2024-09-14T06:16:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00019298199999999999 2024-09-14T06:16:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.305986+0000", 2024-09-14T06:16:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9110000000000002e-05 2024-09-14T06:16:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.960 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.307140+0000", 2024-09-14T06:16:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011544459999999999 2024-09-14T06:16:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3750 3.e 3:76656e66:::rbd_data.1152bd92e04f.000000000000003e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.347645+0000", 2024-09-14T06:16:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.5994122290000004, 2024-09-14T06:16:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023710099999999998, 2024-09-14T06:16:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.961 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3750 2024-09-14T06:16:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.347645+0000", 2024-09-14T06:16:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.962 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.347645+0000", 2024-09-14T06:16:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.347648+0000", 2024-09-14T06:16:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4389999999999999e-06 2024-09-14T06:16:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.347656+0000", 2024-09-14T06:16:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5229999999999995e-06 2024-09-14T06:16:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.963 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.347682+0000", 2024-09-14T06:16:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5534999999999999e-05 2024-09-14T06:16:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.347696+0000", 2024-09-14T06:16:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4541e-05 2024-09-14T06:16:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.347782+0000", 2024-09-14T06:16:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5704999999999993e-05 2024-09-14T06:16:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.347917+0000", 2024-09-14T06:16:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000135673 2024-09-14T06:16:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.348062+0000", 2024-09-14T06:16:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000144857 2024-09-14T06:16:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.349397+0000", 2024-09-14T06:16:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0013342320000000001 2024-09-14T06:16:32.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.349926+0000", 2024-09-14T06:16:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00052984799999999997 2024-09-14T06:16:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.349979+0000", 2024-09-14T06:16:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2231000000000002e-05 2024-09-14T06:16:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.350016+0000", 2024-09-14T06:16:32.966 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7425999999999997e-05 2024-09-14T06:16:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3859 3.16 3:6e3887ad:::rbd_data.1152bd92e04f.0000000000000061:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.495582+0000", 2024-09-14T06:16:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.4514749250000003, 2024-09-14T06:16:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024261769999999998, 2024-09-14T06:16:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:32.967 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3859 2024-09-14T06:16:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.495582+0000", 2024-09-14T06:16:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.495582+0000", 2024-09-14T06:16:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.495585+0000", 2024-09-14T06:16:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.317e-06 2024-09-14T06:16:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.495594+0000", 2024-09-14T06:16:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4399999999999994e-06 2024-09-14T06:16:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.495616+0000", 2024-09-14T06:16:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1571999999999999e-05 2024-09-14T06:16:32.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.495628+0000", 2024-09-14T06:16:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2544999999999999e-05 2024-09-14T06:16:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.495700+0000", 2024-09-14T06:16:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1977999999999994e-05 2024-09-14T06:16:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.495802+0000", 2024-09-14T06:16:32.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010177499999999999 2024-09-14T06:16:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:16:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.495918+0000", 2024-09-14T06:16:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000115522 2024-09-14T06:16:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.496632+0000", 2024-09-14T06:16:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00071453300000000005 2024-09-14T06:16:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.971 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.497900+0000", 2024-09-14T06:16:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001268059 2024-09-14T06:16:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.497959+0000", 2024-09-14T06:16:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.923e-05 2024-09-14T06:16:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.498009+0000", 2024-09-14T06:16:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9206000000000003e-05 2024-09-14T06:16:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3865 3.5 3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head [stat,truncate 1998848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.502395+0000", 2024-09-14T06:16:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.4446622969999998, 2024-09-14T06:16:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030449240000000001, 2024-09-14T06:16:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3865 2024-09-14T06:16:32.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502395+0000", 2024-09-14T06:16:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502395+0000", 2024-09-14T06:16:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502397+0000", 2024-09-14T06:16:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:16:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502403+0000", 2024-09-14T06:16:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3409999999999999e-06 2024-09-14T06:16:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502426+0000", 2024-09-14T06:16:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3181e-05 2024-09-14T06:16:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502437+0000", 2024-09-14T06:16:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0749e-05 2024-09-14T06:16:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502510+0000", 2024-09-14T06:16:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3606e-05 2024-09-14T06:16:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502595+0000", 2024-09-14T06:16:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5018999999999994e-05 2024-09-14T06:16:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502745+0000", 2024-09-14T06:16:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00014936600000000001 2024-09-14T06:16:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.504081+0000", 2024-09-14T06:16:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0013359310000000001 2024-09-14T06:16:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505355+0000", 2024-09-14T06:16:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001273887 2024-09-14T06:16:32.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505402+0000", 2024-09-14T06:16:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7515000000000002e-05 2024-09-14T06:16:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505440+0000", 2024-09-14T06:16:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7805999999999997e-05 2024-09-14T06:16:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3868 3.16 3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head [stat,truncate 1998848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.4445756339999996, 2024-09-14T06:16:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034445919999999998, 2024-09-14T06:16:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3868 2024-09-14T06:16:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1900000000000002e-07 2024-09-14T06:16:32.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502501+0000", 2024-09-14T06:16:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8340999999999999e-05 2024-09-14T06:16:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502512+0000", 2024-09-14T06:16:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1347e-05 2024-09-14T06:16:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502518+0000", 2024-09-14T06:16:32.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.947e-06 2024-09-14T06:16:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502570+0000", 2024-09-14T06:16:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1858999999999999e-05 2024-09-14T06:16:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502633+0000", 2024-09-14T06:16:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2974999999999999e-05 2024-09-14T06:16:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:16:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502795+0000", 2024-09-14T06:16:32.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016262899999999999 2024-09-14T06:16:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505179+0000", 2024-09-14T06:16:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002383844 2024-09-14T06:16:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505676+0000", 2024-09-14T06:16:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00049634899999999999 2024-09-14T06:16:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.983 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505816+0000", 2024-09-14T06:16:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00014026300000000001 2024-09-14T06:16:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505926+0000", 2024-09-14T06:16:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000110319 2024-09-14T06:16:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.984 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3902 3.5 3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.569016+0000", 2024-09-14T06:16:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.3780417629999997, 2024-09-14T06:16:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003356197, 2024-09-14T06:16:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3902 2024-09-14T06:16:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569016+0000", 2024-09-14T06:16:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569016+0000", 2024-09-14T06:16:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569019+0000", 2024-09-14T06:16:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2279999999999998e-06 2024-09-14T06:16:32.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569029+0000", 2024-09-14T06:16:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8079999999999999e-06 2024-09-14T06:16:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569059+0000", 2024-09-14T06:16:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0198000000000001e-05 2024-09-14T06:16:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569074+0000", 2024-09-14T06:16:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4968e-05 2024-09-14T06:16:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569165+0000", 2024-09-14T06:16:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0867999999999998e-05 2024-09-14T06:16:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569282+0000", 2024-09-14T06:16:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000117602 2024-09-14T06:16:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569405+0000", 2024-09-14T06:16:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122471 2024-09-14T06:16:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.570585+0000", 2024-09-14T06:16:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001180459 2024-09-14T06:16:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.572315+0000", 2024-09-14T06:16:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0017295329999999999 2024-09-14T06:16:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.572345+0000", 2024-09-14T06:16:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0784999999999997e-05 2024-09-14T06:16:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.572372+0000", 2024-09-14T06:16:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6276999999999999e-05 2024-09-14T06:16:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.990 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3916 3.16 3:6ab8b1d8:::rbd_data.1152bd92e04f.0000000000000002:head [stat,truncate 1590169] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.3678562029999997, 2024-09-14T06:16:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022958078999999999, 2024-09-14T06:16:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3916 2024-09-14T06:16:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579203+0000", 2024-09-14T06:16:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.748e-06 2024-09-14T06:16:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579208+0000", 2024-09-14T06:16:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:16:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579223+0000", 2024-09-14T06:16:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.508e-05 2024-09-14T06:16:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579232+0000", 2024-09-14T06:16:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9719999999999998e-06 2024-09-14T06:16:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579307+0000", 2024-09-14T06:16:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5028000000000001e-05 2024-09-14T06:16:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579412+0000", 2024-09-14T06:16:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010525400000000001 2024-09-14T06:16:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:16:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579535+0000", 2024-09-14T06:16:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122726 2024-09-14T06:16:32.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.581415+0000", 2024-09-14T06:16:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018800360000000001 2024-09-14T06:16:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602017+0000", 2024-09-14T06:16:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.020602425000000001 2024-09-14T06:16:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602096+0000", 2024-09-14T06:16:32.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8968999999999996e-05 2024-09-14T06:16:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602159+0000", 2024-09-14T06:16:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2824999999999996e-05 2024-09-14T06:16:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4680 3.5 3:a428c282:::rbd_data.1152bd92e04f.0000000000000047:head [set-alloc-hint object_size 2097152 write_size 2097152,write 524288~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.913211+0000", 2024-09-14T06:16:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.033846145, 2024-09-14T06:16:32.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026281880000000001, 2024-09-14T06:16:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4680 2024-09-14T06:16:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.913211+0000", 2024-09-14T06:16:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.913211+0000", 2024-09-14T06:16:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.913214+0000", 2024-09-14T06:16:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8830000000000002e-06 2024-09-14T06:16:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.913246+0000", 2024-09-14T06:16:32.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2274000000000003e-05 2024-09-14T06:16:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.913269+0000", 2024-09-14T06:16:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2461000000000001e-05 2024-09-14T06:16:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.913279+0000", 2024-09-14T06:16:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0448000000000001e-05 2024-09-14T06:16:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:32.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.913394+0000", 2024-09-14T06:16:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000115262 2024-09-14T06:16:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.913558+0000", 2024-09-14T06:16:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016340300000000001 2024-09-14T06:16:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.913769+0000", 2024-09-14T06:16:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021155600000000001 2024-09-14T06:16:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.914902+0000", 2024-09-14T06:16:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011329070000000001 2024-09-14T06:16:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.915680+0000", 2024-09-14T06:16:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00077796699999999996 2024-09-14T06:16:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.915747+0000", 2024-09-14T06:16:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6552999999999998e-05 2024-09-14T06:16:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.915839+0000", 2024-09-14T06:16:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2473999999999996e-05 2024-09-14T06:16:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4704 3.5 3:a428c282:::rbd_data.1152bd92e04f.0000000000000047:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:33.002 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.943845+0000", 2024-09-14T06:16:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.0032122980000002, 2024-09-14T06:16:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003139476, 2024-09-14T06:16:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4704 2024-09-14T06:16:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.943845+0000", 2024-09-14T06:16:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.943845+0000", 2024-09-14T06:16:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.943846+0000", 2024-09-14T06:16:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2049999999999999e-06 2024-09-14T06:16:33.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.943851+0000", 2024-09-14T06:16:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2909999999999998e-06 2024-09-14T06:16:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944219+0000", 2024-09-14T06:16:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036789100000000002 2024-09-14T06:16:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944232+0000", 2024-09-14T06:16:33.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2564e-05 2024-09-14T06:16:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944331+0000", 2024-09-14T06:16:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8628999999999994e-05 2024-09-14T06:16:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944481+0000", 2024-09-14T06:16:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015043700000000001 2024-09-14T06:16:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.006 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944672+0000", 2024-09-14T06:16:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000191237 2024-09-14T06:16:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.945897+0000", 2024-09-14T06:16:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0012252020000000001 2024-09-14T06:16:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.946788+0000", 2024-09-14T06:16:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000890134 2024-09-14T06:16:33.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.946894+0000", 2024-09-14T06:16:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000106062 2024-09-14T06:16:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.946984+0000", 2024-09-14T06:16:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0823999999999996e-05 2024-09-14T06:16:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4873 3.e 3:76656e66:::rbd_data.1152bd92e04f.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=2b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.7534096640000003, 2024-09-14T06:16:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048796120000000002, 2024-09-14T06:16:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4873 2024-09-14T06:16:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.009 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193651+0000", 2024-09-14T06:16:33.010 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:16:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193663+0000", 2024-09-14T06:16:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2047e-05 2024-09-14T06:16:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193692+0000", 2024-09-14T06:16:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9346e-05 2024-09-14T06:16:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.011 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193708+0000", 2024-09-14T06:16:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6123000000000001e-05 2024-09-14T06:16:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193798+0000", 2024-09-14T06:16:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9909999999999998e-05 2024-09-14T06:16:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193863+0000", 2024-09-14T06:16:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4583000000000002e-05 2024-09-14T06:16:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.194717+0000", 2024-09-14T06:16:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00085473299999999999 2024-09-14T06:16:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198400+0000", 2024-09-14T06:16:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036830890000000001 2024-09-14T06:16:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198461+0000", 2024-09-14T06:16:33.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0844000000000002e-05 2024-09-14T06:16:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198496+0000", 2024-09-14T06:16:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4820000000000002e-05 2024-09-14T06:16:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198527+0000", 2024-09-14T06:16:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1229999999999997e-05 2024-09-14T06:16:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4935 3.e 3:76656e66:::rbd_data.1152bd92e04f.000000000000003e:head [write 1982464~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:28.261170+0000", 2024-09-14T06:16:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.6858874100000003, 2024-09-14T06:16:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024817979999999999, 2024-09-14T06:16:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:33.015 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4935 2024-09-14T06:16:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.261170+0000", 2024-09-14T06:16:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.261170+0000", 2024-09-14T06:16:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.016 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.261171+0000", 2024-09-14T06:16:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4869999999999999e-06 2024-09-14T06:16:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.261189+0000", 2024-09-14T06:16:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7348999999999999e-05 2024-09-14T06:16:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.261213+0000", 2024-09-14T06:16:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4507000000000001e-05 2024-09-14T06:16:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.261226+0000", 2024-09-14T06:16:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2976000000000001e-05 2024-09-14T06:16:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.261312+0000", 2024-09-14T06:16:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5362e-05 2024-09-14T06:16:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.261445+0000", 2024-09-14T06:16:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013369200000000001 2024-09-14T06:16:33.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.261628+0000", 2024-09-14T06:16:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00018230699999999999 2024-09-14T06:16:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.262898+0000", 2024-09-14T06:16:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001270059 2024-09-14T06:16:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.263539+0000", 2024-09-14T06:16:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064181600000000002 2024-09-14T06:16:33.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.263597+0000", 2024-09-14T06:16:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7661999999999998e-05 2024-09-14T06:16:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.263652+0000", 2024-09-14T06:16:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4580999999999998e-05 2024-09-14T06:16:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4981 3.5 3:a428c282:::rbd_data.1152bd92e04f.0000000000000047:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:28.305860+0000", 2024-09-14T06:16:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.6411977130000004, 2024-09-14T06:16:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024592479999999998, 2024-09-14T06:16:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4981 2024-09-14T06:16:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.021 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.305860+0000", 2024-09-14T06:16:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.305860+0000", 2024-09-14T06:16:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.305861+0000", 2024-09-14T06:16:33.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3790000000000001e-06 2024-09-14T06:16:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.305867+0000", 2024-09-14T06:16:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.665e-06 2024-09-14T06:16:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.305881+0000", 2024-09-14T06:16:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4286e-05 2024-09-14T06:16:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:33.023 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.305890+0000", 2024-09-14T06:16:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5000000000000005e-06 2024-09-14T06:16:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.305966+0000", 2024-09-14T06:16:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5625999999999997e-05 2024-09-14T06:16:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.306047+0000", 2024-09-14T06:16:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0611000000000005e-05 2024-09-14T06:16:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.024 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.306154+0000", 2024-09-14T06:16:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000106889 2024-09-14T06:16:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.306965+0000", 2024-09-14T06:16:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00081116399999999996 2024-09-14T06:16:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.308235+0000", 2024-09-14T06:16:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001270724 2024-09-14T06:16:33.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.308287+0000", 2024-09-14T06:16:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2018999999999998e-05 2024-09-14T06:16:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.308319+0000", 2024-09-14T06:16:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1384999999999999e-05 2024-09-14T06:16:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.026 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:5022 3.e 3:72ea77a7:::rbd_data.1152bd92e04f.0000000000000049:head [set-alloc-hint object_size 2097152 write_size 2097152,write 560462~12978] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:28.363072+0000", 2024-09-14T06:16:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.5839849719999997, 2024-09-14T06:16:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024097260000000001, 2024-09-14T06:16:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 5022 2024-09-14T06:16:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.027 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.363072+0000", 2024-09-14T06:16:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.363072+0000", 2024-09-14T06:16:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.363075+0000", 2024-09-14T06:16:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4880000000000001e-06 2024-09-14T06:16:33.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.363093+0000", 2024-09-14T06:16:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7813e-05 2024-09-14T06:16:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.363116+0000", 2024-09-14T06:16:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3660999999999999e-05 2024-09-14T06:16:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.363128+0000", 2024-09-14T06:16:33.029 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2189e-05 2024-09-14T06:16:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.363215+0000", 2024-09-14T06:16:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6852000000000001e-05 2024-09-14T06:16:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.363303+0000", 2024-09-14T06:16:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7825999999999996e-05 2024-09-14T06:16:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:33.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.363459+0000", 2024-09-14T06:16:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015592500000000001 2024-09-14T06:16:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.364798+0000", 2024-09-14T06:16:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001339182 2024-09-14T06:16:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.365366+0000", 2024-09-14T06:16:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00056775300000000005 2024-09-14T06:16:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.031 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.365433+0000", 2024-09-14T06:16:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6705999999999995e-05 2024-09-14T06:16:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.365482+0000", 2024-09-14T06:16:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9330999999999999e-05 2024-09-14T06:16:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:5048 3.5 3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1015808~3436] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:33.032 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:28.397977+0000", 2024-09-14T06:16:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.5490805439999997, 2024-09-14T06:16:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0025479880000000002, 2024-09-14T06:16:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 5048 2024-09-14T06:16:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:33.033 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.397977+0000", 2024-09-14T06:16:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.397977+0000", 2024-09-14T06:16:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.397979+0000", 2024-09-14T06:16:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.396e-06 2024-09-14T06:16:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.034 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.397988+0000", 2024-09-14T06:16:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3060000000000003e-06 2024-09-14T06:16:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.398007+0000", 2024-09-14T06:16:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8734000000000001e-05 2024-09-14T06:16:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.398021+0000", 2024-09-14T06:16:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3923e-05 2024-09-14T06:16:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.035 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.398105+0000", 2024-09-14T06:16:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4074999999999997e-05 2024-09-14T06:16:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.398201+0000", 2024-09-14T06:16:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5944000000000002e-05 2024-09-14T06:16:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.398361+0000", 2024-09-14T06:16:33.036 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015944 2024-09-14T06:16:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.399104+0000", 2024-09-14T06:16:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00074315799999999995 2024-09-14T06:16:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.400399+0000", 2024-09-14T06:16:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001295257 2024-09-14T06:16:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:33.037 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.400469+0000", 2024-09-14T06:16:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9921999999999999e-05 2024-09-14T06:16:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.400525+0000", 2024-09-14T06:16:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5832999999999999e-05 2024-09-14T06:16:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:5105 3.e 3:72ea77a7:::rbd_data.1152bd92e04f.0000000000000049:head [set-alloc-hint object_size 2097152 write_size 2097152,write 668218~3526] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:28.482289+0000", 2024-09-14T06:16:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4647680889999997, 2024-09-14T06:16:33.038 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024508279999999999, 2024-09-14T06:16:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 5105 2024-09-14T06:16:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.482289+0000", 2024-09-14T06:16:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.482289+0000", 2024-09-14T06:16:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.482292+0000", 2024-09-14T06:16:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4760000000000001e-06 2024-09-14T06:16:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.482303+0000", 2024-09-14T06:16:33.040 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1049999999999999e-05 2024-09-14T06:16:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.482326+0000", 2024-09-14T06:16:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3357e-05 2024-09-14T06:16:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.482339+0000", 2024-09-14T06:16:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3274e-05 2024-09-14T06:16:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:33.041 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.482413+0000", 2024-09-14T06:16:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3707000000000004e-05 2024-09-14T06:16:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.482511+0000", 2024-09-14T06:16:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7744999999999993e-05 2024-09-14T06:16:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.482654+0000", 2024-09-14T06:16:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00014338299999999999 2024-09-14T06:16:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.042 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.483495+0000", 2024-09-14T06:16:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00084044100000000004 2024-09-14T06:16:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.484625+0000", 2024-09-14T06:16:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011300349999999999 2024-09-14T06:16:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.484687+0000", 2024-09-14T06:16:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2121999999999999e-05 2024-09-14T06:16:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.484740+0000", 2024-09-14T06:16:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3238000000000001e-05 2024-09-14T06:16:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:6925 3.5 3:a15e9af1:::rbd_data.1152f87cbac6.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:16:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.8322781259999998, 2024-09-14T06:16:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0049096490000000003, 2024-09-14T06:16:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:33.044 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 6925 2024-09-14T06:16:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.045 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114780+0000", 2024-09-14T06:16:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.09e-07 2024-09-14T06:16:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114784+0000", 2024-09-14T06:16:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4669999999999998e-06 2024-09-14T06:16:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114799+0000", 2024-09-14T06:16:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4491000000000001e-05 2024-09-14T06:16:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114810+0000", 2024-09-14T06:16:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1243000000000001e-05 2024-09-14T06:16:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115515+0000", 2024-09-14T06:16:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00070521899999999998 2024-09-14T06:16:33.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115610+0000", 2024-09-14T06:16:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4260999999999998e-05 2024-09-14T06:16:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115742+0000", 2024-09-14T06:16:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000132414 2024-09-14T06:16:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:33.048 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.116650+0000", 2024-09-14T06:16:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00090814900000000002 2024-09-14T06:16:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119568+0000", 2024-09-14T06:16:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029178440000000002 2024-09-14T06:16:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119636+0000", 2024-09-14T06:16:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8365e-05 2024-09-14T06:16:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.049 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119689+0000", 2024-09-14T06:16:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2386999999999998e-05 2024-09-14T06:16:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:7151 3.e 3:7793b17d:::rbd_data.1152f87cbac6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:16:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.6174603360000002, 2024-09-14T06:16:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.029556783, 2024-09-14T06:16:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:33.050 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 7151 2024-09-14T06:16:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.051 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329599+0000", 2024-09-14T06:16:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5270000000000001e-06 2024-09-14T06:16:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329610+0000", 2024-09-14T06:16:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0431000000000001e-05 2024-09-14T06:16:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.052 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329634+0000", 2024-09-14T06:16:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3617000000000001e-05 2024-09-14T06:16:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329644+0000", 2024-09-14T06:16:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.095e-05 2024-09-14T06:16:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329712+0000", 2024-09-14T06:16:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7067e-05 2024-09-14T06:16:33.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329827+0000", 2024-09-14T06:16:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011498 2024-09-14T06:16:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.334271+0000", 2024-09-14T06:16:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044445839999999997 2024-09-14T06:16:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.358967+0000", 2024-09-14T06:16:33.054 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024696356999999999 2024-09-14T06:16:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359027+0000", 2024-09-14T06:16:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0006000000000001e-05 2024-09-14T06:16:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359099+0000", 2024-09-14T06:16:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1052999999999999e-05 2024-09-14T06:16:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359154+0000", 2024-09-14T06:16:33.055 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5210999999999997e-05 2024-09-14T06:16:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:7151 3.e e32/29 ondisk, result = 0)", 2024-09-14T06:16:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.597515902, 2024-09-14T06:16:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0096251340000000005, 2024-09-14T06:16:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:33.056 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349543+0000", 2024-09-14T06:16:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:16:33.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349552+0000", 2024-09-14T06:16:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1039999999999996e-06 2024-09-14T06:16:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349579+0000", 2024-09-14T06:16:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6449999999999999e-05 2024-09-14T06:16:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349593+0000", 2024-09-14T06:16:33.058 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4099e-05 2024-09-14T06:16:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359003+0000", 2024-09-14T06:16:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0094096119999999995 2024-09-14T06:16:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359014+0000", 2024-09-14T06:16:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0871e-05 2024-09-14T06:16:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359167+0000", 2024-09-14T06:16:33.059 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015300700000000001 2024-09-14T06:16:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:33.060 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:33.060 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:33.060 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:16:33.061 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:33.060+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:33.062 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:16:33.162 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:33.164+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:33.230 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:33.230 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:33.230 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:33.231 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:33.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:16:33.260 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:33.261 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:33.263 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:33.260+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:33.278 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:33.294 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:33.294 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:33.311 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:33.311 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:33.332 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:33.332 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:33.354 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:33.354 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:33.363 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:33.363 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:33.364 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:33.364+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:33.377 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:33.377 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:33.378 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:16:33.398 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:33.398 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:33.465 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:33.460+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:33.524 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:33.524 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:33.542 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:16:33.583 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:33.580+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:33.606 INFO:teuthology.orchestra.run.smithi039.stdout:leaving image image_client.0-clone4 intact 2024-09-14T06:16:33.666 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:33.668+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:33.672 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:33.672 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:33.686 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:16:33.768 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:33.764+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:33.854 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:33.854 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:33.868 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:16:33.869 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:33.864+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:33.926 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-14T06:16:33.969 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:33.968+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:34.009 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:34.009 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:34.024 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:16:34.058 INFO:teuthology.orchestra.run.smithi039.stdout:718 trunc from 0xd19e0e2 to 0x5836416 2024-09-14T06:16:34.093 INFO:teuthology.orchestra.run.smithi039.stdout:721 write 0x676758 thru 0x6845a1 (0xde4a bytes) 2024-09-14T06:16:34.110 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:34.112+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:34.171 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:34.168+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:34.212 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:34.212 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:34.233 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:16:34.247 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:34.247 INFO:teuthology.orchestra.run.smithi039.stdout:724 read 0x29752c0 thru 0x2978199 (0x2eda bytes) 2024-09-14T06:16:34.272 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:34.268+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:34.279 INFO:teuthology.orchestra.run.smithi039.stdout:725 write 0xc844fc9 thru 0xc847a2f (0x2a67 bytes) 2024-09-14T06:16:34.285 INFO:teuthology.orchestra.run.smithi039.stdout:726 writesame 0xc3a0233 thru 0xc3a6b38 data_size 0x3(0x6906 bytes) 2024-09-14T06:16:34.297 INFO:teuthology.orchestra.run.smithi039.stdout:727 trunc from 0xc847a30 to 0xd785835 2024-09-14T06:16:34.308 INFO:teuthology.orchestra.run.smithi039.stdout:729 punch from 0x410793b to 0x411318a, (0xb84f bytes) 2024-09-14T06:16:34.322 INFO:teuthology.orchestra.run.smithi039.stdout:731 write 0xe83e742 thru 0xe843d3b (0x55fa bytes) 2024-09-14T06:16:34.327 INFO:teuthology.orchestra.run.smithi039.stdout:732 writesame 0xb23423f thru 0xb23c80a data_size 0x2(0x85cc bytes) 2024-09-14T06:16:34.340 INFO:teuthology.orchestra.run.smithi039.stdout:733 write 0xc0a782c thru 0xc0b0ae9 (0x92be bytes) 2024-09-14T06:16:34.347 INFO:teuthology.orchestra.run.smithi039.stdout:734 punch from 0xa223d8a to 0xa22cd59, (0x8fcf bytes) 2024-09-14T06:16:34.347 INFO:teuthology.orchestra.run.smithi039.stdout:735 write 0xa0a2f0d thru 0xa0abbac (0x8ca0 bytes) 2024-09-14T06:16:34.354 INFO:teuthology.orchestra.run.smithi039.stdout:736 punch from 0xe00dda9 to 0xe018d5a, (0xafb1 bytes) 2024-09-14T06:16:34.354 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:34.357 INFO:teuthology.orchestra.run.smithi039.stdout:738 trunc from 0xe843d3c to 0xd51922c 2024-09-14T06:16:34.373 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:34.368+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:34.381 INFO:teuthology.orchestra.run.smithi039.stdout:739 read 0x5321321 thru 0x5323b94 (0x2874 bytes) 2024-09-14T06:16:34.382 INFO:teuthology.orchestra.run.smithi039.stdout:743 punch from 0x95631b0 to 0x95695aa, (0x63fa bytes) 2024-09-14T06:16:34.399 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:34.399 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:34.413 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:16:34.416 INFO:teuthology.orchestra.run.smithi039.stdout:744 trunc from 0xd51922c to 0xee43ad 2024-09-14T06:16:34.451 INFO:teuthology.orchestra.run.smithi039.stdout:746 trunc from 0xee43ad to 0x3e3d6dc 2024-09-14T06:16:34.464 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:34.474 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:34.468+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:34.489 INFO:teuthology.orchestra.run.smithi039.stdout:749 write 0x86a07ae thru 0x86af0ea (0xe93d bytes) 2024-09-14T06:16:34.495 INFO:teuthology.orchestra.run.smithi039.stdout:750 writesame 0x77178ff thru 0x771d608 data_size 0x1(0x5d0a bytes) 2024-09-14T06:16:34.524 INFO:teuthology.orchestra.run.smithi039.stdout:751 read 0x3293738 thru 0x3298ec0 (0x5789 bytes) 2024-09-14T06:16:34.524 INFO:teuthology.orchestra.run.smithi039.stdout:752 punch from 0x55cc1d1 to 0x55da6cd, (0xe4fc bytes) 2024-09-14T06:16:34.524 INFO:teuthology.orchestra.run.smithi039.stdout:753 write 0x8f0c49 thru 0x8fc615 (0xb9cd bytes) 2024-09-14T06:16:34.528 INFO:teuthology.orchestra.run.smithi039.stdout:754 writesame 0x1079d00 thru 0x107e062 data_size 0x1(0x4363 bytes) 2024-09-14T06:16:34.541 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:34.541 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:34.551 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:34.551 INFO:teuthology.orchestra.run.smithi039.stdout:756 writesame 0x109f9ae thru 0x109fec5 data_size 0x4(0x518 bytes) 2024-09-14T06:16:34.554 INFO:teuthology.orchestra.run.smithi039.stdout:757 punch from 0x6cdd0d0 to 0x6ce1520, (0x4450 bytes) 2024-09-14T06:16:34.555 INFO:teuthology.orchestra.run.smithi039.stdout:758 read 0x30dc473 thru 0x30ded23 (0x28b1 bytes) 2024-09-14T06:16:34.555 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:34.555 INFO:teuthology.orchestra.run.smithi039.stdout:760 read 0x570af44 thru 0x571815c (0xd219 bytes) 2024-09-14T06:16:34.555 INFO:teuthology.orchestra.run.smithi039.stdout:761 flatten 2024-09-14T06:16:34.556 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:16:34.564 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:16:34.568 INFO:teuthology.orchestra.run.smithi039.stdout:762 read 0x5a58507 thru 0x5a643d0 (0xbeca bytes) 2024-09-14T06:16:34.569 INFO:teuthology.orchestra.run.smithi039.stdout:763 read 0x4ec3b17 thru 0x4ecbbf1 (0x80db bytes) 2024-09-14T06:16:34.569 INFO:teuthology.orchestra.run.smithi039.stdout:764 write 0x79f9b57 thru 0x79feec7 (0x5371 bytes) 2024-09-14T06:16:34.573 INFO:teuthology.orchestra.run.smithi039.stdout:766 punch from 0x7fb1feb to 0x7fb8905, (0x691a bytes) 2024-09-14T06:16:34.573 INFO:teuthology.orchestra.run.smithi039.stdout:767 punch from 0x4b003e6 to 0x4b039dd, (0x35f7 bytes) 2024-09-14T06:16:34.574 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:34.574 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:34.568+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:34.585 INFO:teuthology.orchestra.run.smithi039.stdout:769 trunc from 0x86af0eb to 0xca80472 2024-09-14T06:16:34.595 INFO:teuthology.orchestra.run.smithi039.stdout:770 write 0xa524fc7 thru 0xa52b19e (0x61d8 bytes) 2024-09-14T06:16:34.599 INFO:teuthology.orchestra.run.smithi039.stdout:771 write 0xdc6a5 thru 0xe954d (0xcea9 bytes) 2024-09-14T06:16:34.602 INFO:teuthology.orchestra.run.smithi039.stdout:772 write 0x3235a3 thru 0x32773e (0x419c bytes) 2024-09-14T06:16:34.604 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:34.604 INFO:teuthology.orchestra.run.smithi039.stdout:774 read 0x72d99a2 thru 0x72dbd45 (0x23a4 bytes) 2024-09-14T06:16:34.605 INFO:teuthology.orchestra.run.smithi039.stdout:776 punch from 0x1b49a51 to 0x1b4d7ec, (0x3d9b bytes) 2024-09-14T06:16:34.605 INFO:teuthology.orchestra.run.smithi039.stdout:777 write 0x25fe7ed thru 0x2601c77 (0x348b bytes) 2024-09-14T06:16:34.629 INFO:teuthology.orchestra.run.smithi039.stdout:778 trunc from 0xca80472 to 0x4fd68fe 2024-09-14T06:16:34.644 INFO:teuthology.orchestra.run.smithi039.stdout:781 read 0xd636f2 thru 0xd65eab (0x27ba bytes) 2024-09-14T06:16:34.645 INFO:teuthology.orchestra.run.smithi039.stdout:782 writesame 0x2c137f1 thru 0x2c19e73 data_size 0x17(0x6683 bytes) 2024-09-14T06:16:34.663 INFO:teuthology.orchestra.run.smithi039.stdout:785 writesame 0x75f1c45 thru 0x75fefbd data_size 0x1(0xd379 bytes) 2024-09-14T06:16:34.664 INFO:teuthology.orchestra.run.smithi039.stdout:786 trunc from 0x75fefbe to 0x8302d75 2024-09-14T06:16:34.673 INFO:teuthology.orchestra.run.smithi039.stdout:787 read 0x3f1ebf thru 0x3fa6cb (0x880d bytes) 2024-09-14T06:16:34.674 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:34.668+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:34.674 INFO:teuthology.orchestra.run.smithi039.stdout:790 write 0x1c03ebe thru 0x1c07a5c (0x3b9f bytes) 2024-09-14T06:16:34.679 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:34.679 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:34.694 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:16:34.705 INFO:teuthology.orchestra.run.smithi039.stdout:791 write 0xe7c39c7 thru 0xe7c970f (0x5d49 bytes) 2024-09-14T06:16:34.709 INFO:teuthology.orchestra.run.smithi039.stdout:792 read 0xd031b5c thru 0xd03a975 (0x8e1a bytes) 2024-09-14T06:16:34.709 INFO:teuthology.orchestra.run.smithi039.stdout:794 writesame 0xaf5737f thru 0xaf5e35d data_size 0x1(0x6fdf bytes) 2024-09-14T06:16:34.727 INFO:teuthology.orchestra.run.smithi039.stdout:796 write 0xe62c7c6 thru 0xe6308a8 (0x40e3 bytes) 2024-09-14T06:16:34.731 INFO:teuthology.orchestra.run.smithi039.stdout:797 punch from 0xc616f6a to 0xc6240dd, (0xd173 bytes) 2024-09-14T06:16:34.731 INFO:teuthology.orchestra.run.smithi039.stdout:798 write 0xaef9b35 thru 0xaefe059 (0x4525 bytes) 2024-09-14T06:16:34.733 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:34.733 INFO:teuthology.orchestra.run.smithi039.stdout:800 clone 6 order 24 su 65536 sc 7 2024-09-14T06:16:34.775 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:34.772+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:34.812 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:34.812 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:34.826 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:16:34.876 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:34.872+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:34.949 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:34.949 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:34.963 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:16:34.978 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:34.972+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:35.079 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:35.080+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:35.086 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:35.086 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:35.101 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:16:35.189 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:35.192+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:35.272 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:35.272 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:35.281 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:35.280+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:35.286 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:16:35.382 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:35.384+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:35.417 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:16:35.447 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:35.447 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:35.461 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:16:35.515 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:35.512+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:35.547 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-09-14T06:16:35.585 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:35.586 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:35.586 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:35.580+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:35.603 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:35.608 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:35.608 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:35.608 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:35.609 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:35.621 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:35.621 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:35.623 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:16:35.639 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:35.660 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:35.660 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:35.681 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:35.681 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:35.685 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:35.680+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:35.702 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:35.703 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:35.724 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:35.724 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:35.762 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:35.762 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:35.762 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:35.762 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:35.762 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:35.779 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:16:35.823 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:35.820+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:35.853 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-14T06:16:35.887 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:35.888+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:35.969 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:35.969 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:35.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3614 3.f 3:f0701b33:::rbd_data.11528c180b7.0000000000000000:3 [read 3047424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.262033+0000", 2024-09-14T06:16:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.7024924610000003, 2024-09-14T06:16:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016128916, 2024-09-14T06:16:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3614 2024-09-14T06:16:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262033+0000", 2024-09-14T06:16:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262033+0000", 2024-09-14T06:16:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262034+0000", 2024-09-14T06:16:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1230000000000001e-06 2024-09-14T06:16:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262040+0000", 2024-09-14T06:16:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8100000000000003e-06 2024-09-14T06:16:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262079+0000", 2024-09-14T06:16:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9520000000000001e-05 2024-09-14T06:16:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262097+0000", 2024-09-14T06:16:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8357000000000001e-05 2024-09-14T06:16:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.274879+0000", 2024-09-14T06:16:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.012781391 2024-09-14T06:16:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275121+0000", 2024-09-14T06:16:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00024238499999999999 2024-09-14T06:16:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.278162+0000", 2024-09-14T06:16:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030403299999999999 2024-09-14T06:16:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3618 3.f 3:f0701b33:::rbd_data.11528c180b7.0000000000000000:3 [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.262344+0000", 2024-09-14T06:16:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.7021807199999994, 2024-09-14T06:16:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015084447000000001, 2024-09-14T06:16:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3618 2024-09-14T06:16:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262344+0000", 2024-09-14T06:16:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262344+0000", 2024-09-14T06:16:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262346+0000", 2024-09-14T06:16:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.657e-06 2024-09-14T06:16:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262352+0000", 2024-09-14T06:16:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.677e-06 2024-09-14T06:16:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262374+0000", 2024-09-14T06:16:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2089000000000001e-05 2024-09-14T06:16:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262391+0000", 2024-09-14T06:16:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.698e-05 2024-09-14T06:16:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277035+0000", 2024-09-14T06:16:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014643959999999999 2024-09-14T06:16:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277230+0000", 2024-09-14T06:16:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00019529899999999999 2024-09-14T06:16:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277429+0000", 2024-09-14T06:16:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00019878499999999999 2024-09-14T06:16:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3619 3.f 3:f0701b33:::rbd_data.11528c180b7.0000000000000000:3 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.262413+0000", 2024-09-14T06:16:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.7021116710000008, 2024-09-14T06:16:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015481155, 2024-09-14T06:16:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3619 2024-09-14T06:16:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262413+0000", 2024-09-14T06:16:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262413+0000", 2024-09-14T06:16:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262415+0000", 2024-09-14T06:16:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.73e-06 2024-09-14T06:16:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262420+0000", 2024-09-14T06:16:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.254e-06 2024-09-14T06:16:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262448+0000", 2024-09-14T06:16:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7509999999999999e-05 2024-09-14T06:16:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.262463+0000", 2024-09-14T06:16:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.471e-05 2024-09-14T06:16:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277501+0000", 2024-09-14T06:16:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015037913999999999 2024-09-14T06:16:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277700+0000", 2024-09-14T06:16:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000199211 2024-09-14T06:16:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277895+0000", 2024-09-14T06:16:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00019482599999999999 2024-09-14T06:16:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4373 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589054+0000", 2024-09-14T06:16:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.3754713519999999, 2024-09-14T06:16:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015113086, 2024-09-14T06:16:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4373 2024-09-14T06:16:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589054+0000", 2024-09-14T06:16:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589054+0000", 2024-09-14T06:16:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589054+0000", 2024-09-14T06:16:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6400000000000003e-07 2024-09-14T06:16:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589063+0000", 2024-09-14T06:16:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5699999999999993e-06 2024-09-14T06:16:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589069+0000", 2024-09-14T06:16:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6359999999999997e-06 2024-09-14T06:16:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589079+0000", 2024-09-14T06:16:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.9469999999999998e-06 2024-09-14T06:16:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604036+0000", 2024-09-14T06:16:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014956410999999999 2024-09-14T06:16:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604088+0000", 2024-09-14T06:16:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2163000000000002e-05 2024-09-14T06:16:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604167+0000", 2024-09-14T06:16:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8894999999999996e-05 2024-09-14T06:16:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4374 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 1146880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589089+0000", 2024-09-14T06:16:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.3754362150000006, 2024-09-14T06:16:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015225613000000001, 2024-09-14T06:16:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4374 2024-09-14T06:16:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589089+0000", 2024-09-14T06:16:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589089+0000", 2024-09-14T06:16:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589089+0000", 2024-09-14T06:16:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3099999999999998e-07 2024-09-14T06:16:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589092+0000", 2024-09-14T06:16:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5660000000000001e-06 2024-09-14T06:16:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589099+0000", 2024-09-14T06:16:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6649999999999997e-06 2024-09-14T06:16:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589104+0000", 2024-09-14T06:16:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8919999999999999e-06 2024-09-14T06:16:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604198+0000", 2024-09-14T06:16:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015094471999999999 2024-09-14T06:16:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604246+0000", 2024-09-14T06:16:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7936999999999999e-05 2024-09-14T06:16:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604315+0000", 2024-09-14T06:16:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8549999999999999e-05 2024-09-14T06:16:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4375 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589111+0000", 2024-09-14T06:16:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.375413923, 2024-09-14T06:16:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015436236000000001, 2024-09-14T06:16:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4375 2024-09-14T06:16:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589111+0000", 2024-09-14T06:16:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589111+0000", 2024-09-14T06:16:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589112+0000", 2024-09-14T06:16:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7e-07 2024-09-14T06:16:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589114+0000", 2024-09-14T06:16:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0099999999999998e-06 2024-09-14T06:16:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589120+0000", 2024-09-14T06:16:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.438e-06 2024-09-14T06:16:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589125+0000", 2024-09-14T06:16:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6739999999999996e-06 2024-09-14T06:16:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604402+0000", 2024-09-14T06:16:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015276993000000001 2024-09-14T06:16:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604483+0000", 2024-09-14T06:16:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1384999999999994e-05 2024-09-14T06:16:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604547+0000", 2024-09-14T06:16:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4265999999999998e-05 2024-09-14T06:16:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4376 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589132+0000", 2024-09-14T06:16:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.3753929140000007, 2024-09-14T06:16:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015632553, 2024-09-14T06:16:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4376 2024-09-14T06:16:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589132+0000", 2024-09-14T06:16:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589132+0000", 2024-09-14T06:16:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589133+0000", 2024-09-14T06:16:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1600000000000001e-07 2024-09-14T06:16:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589135+0000", 2024-09-14T06:16:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.035e-06 2024-09-14T06:16:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589141+0000", 2024-09-14T06:16:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.516e-06 2024-09-14T06:16:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589146+0000", 2024-09-14T06:16:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9899999999999997e-06 2024-09-14T06:16:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604594+0000", 2024-09-14T06:16:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015447341 2024-09-14T06:16:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604677+0000", 2024-09-14T06:16:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2999999999999998e-05 2024-09-14T06:16:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604765+0000", 2024-09-14T06:16:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8154999999999999e-05 2024-09-14T06:16:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4377 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589153+0000", 2024-09-14T06:16:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.3753717339999998, 2024-09-14T06:16:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015855904000000001, 2024-09-14T06:16:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4377 2024-09-14T06:16:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589153+0000", 2024-09-14T06:16:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589153+0000", 2024-09-14T06:16:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589154+0000", 2024-09-14T06:16:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:16:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589156+0000", 2024-09-14T06:16:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8929999999999999e-06 2024-09-14T06:16:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589162+0000", 2024-09-14T06:16:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3879999999999997e-06 2024-09-14T06:16:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589167+0000", 2024-09-14T06:16:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7230000000000004e-06 2024-09-14T06:16:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604810+0000", 2024-09-14T06:16:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015642795000000001 2024-09-14T06:16:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.604931+0000", 2024-09-14T06:16:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000121038 2024-09-14T06:16:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605009+0000", 2024-09-14T06:16:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8651000000000003e-05 2024-09-14T06:16:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4378 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589177+0000", 2024-09-14T06:16:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.3753480210000006, 2024-09-14T06:16:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015995853000000001, 2024-09-14T06:16:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4378 2024-09-14T06:16:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589177+0000", 2024-09-14T06:16:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589177+0000", 2024-09-14T06:16:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589178+0000", 2024-09-14T06:16:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3799999999999998e-07 2024-09-14T06:16:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589180+0000", 2024-09-14T06:16:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1890000000000001e-06 2024-09-14T06:16:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589186+0000", 2024-09-14T06:16:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5270000000000004e-06 2024-09-14T06:16:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589193+0000", 2024-09-14T06:16:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.285e-06 2024-09-14T06:16:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605046+0000", 2024-09-14T06:16:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.015853678 2024-09-14T06:16:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605105+0000", 2024-09-14T06:16:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8829000000000002e-05 2024-09-14T06:16:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605173+0000", 2024-09-14T06:16:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7906999999999999e-05 2024-09-14T06:16:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4379 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 2129920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589200+0000", 2024-09-14T06:16:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.3753248899999999, 2024-09-14T06:16:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016178273, 2024-09-14T06:16:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4379 2024-09-14T06:16:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589200+0000", 2024-09-14T06:16:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589200+0000", 2024-09-14T06:16:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589201+0000", 2024-09-14T06:16:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8800000000000003e-07 2024-09-14T06:16:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589203+0000", 2024-09-14T06:16:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0329999999999998e-06 2024-09-14T06:16:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589209+0000", 2024-09-14T06:16:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3709999999999996e-06 2024-09-14T06:16:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589214+0000", 2024-09-14T06:16:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:16:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605216+0000", 2024-09-14T06:16:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016002267000000001 2024-09-14T06:16:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605290+0000", 2024-09-14T06:16:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3662e-05 2024-09-14T06:16:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605378+0000", 2024-09-14T06:16:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8529000000000006e-05 2024-09-14T06:16:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4380 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 2326528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589221+0000", 2024-09-14T06:16:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.3753036939999994, 2024-09-14T06:16:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016607628999999999, 2024-09-14T06:16:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4380 2024-09-14T06:16:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589221+0000", 2024-09-14T06:16:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589221+0000", 2024-09-14T06:16:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589222+0000", 2024-09-14T06:16:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6100000000000001e-07 2024-09-14T06:16:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589224+0000", 2024-09-14T06:16:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1619999999999998e-06 2024-09-14T06:16:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589230+0000", 2024-09-14T06:16:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4080000000000003e-06 2024-09-14T06:16:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589235+0000", 2024-09-14T06:16:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:16:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605688+0000", 2024-09-14T06:16:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016452784000000002 2024-09-14T06:16:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605761+0000", 2024-09-14T06:16:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3702999999999996e-05 2024-09-14T06:16:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605829+0000", 2024-09-14T06:16:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7576999999999994e-05 2024-09-14T06:16:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4381 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 2523136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589243+0000", 2024-09-14T06:16:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.3752824780000008, 2024-09-14T06:16:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016792926999999999, 2024-09-14T06:16:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4381 2024-09-14T06:16:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589243+0000", 2024-09-14T06:16:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589243+0000", 2024-09-14T06:16:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589243+0000", 2024-09-14T06:16:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0900000000000002e-07 2024-09-14T06:16:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589245+0000", 2024-09-14T06:16:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2450000000000001e-06 2024-09-14T06:16:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589252+0000", 2024-09-14T06:16:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6699999999999997e-06 2024-09-14T06:16:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589259+0000", 2024-09-14T06:16:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0400000000000004e-06 2024-09-14T06:16:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605876+0000", 2024-09-14T06:16:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016617226999999998 2024-09-14T06:16:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.605956+0000", 2024-09-14T06:16:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9430000000000004e-05 2024-09-14T06:16:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606036+0000", 2024-09-14T06:16:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9806000000000001e-05 2024-09-14T06:16:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4382 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 2719744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589267+0000", 2024-09-14T06:16:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.3752582310000001, 2024-09-14T06:16:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016998154000000001, 2024-09-14T06:16:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4382 2024-09-14T06:16:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589267+0000", 2024-09-14T06:16:36.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589267+0000", 2024-09-14T06:16:36.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589267+0000", 2024-09-14T06:16:36.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7199999999999999e-07 2024-09-14T06:16:36.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589269+0000", 2024-09-14T06:16:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.956e-06 2024-09-14T06:16:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589277+0000", 2024-09-14T06:16:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9570000000000002e-06 2024-09-14T06:16:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589296+0000", 2024-09-14T06:16:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9185e-05 2024-09-14T06:16:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606080+0000", 2024-09-14T06:16:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016783513 2024-09-14T06:16:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606153+0000", 2024-09-14T06:16:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2622999999999999e-05 2024-09-14T06:16:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606265+0000", 2024-09-14T06:16:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000112448 2024-09-14T06:16:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4383 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 2916352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589306+0000", 2024-09-14T06:16:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.3752193310000003, 2024-09-14T06:16:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017123392000000001, 2024-09-14T06:16:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4383 2024-09-14T06:16:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589306+0000", 2024-09-14T06:16:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589306+0000", 2024-09-14T06:16:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589306+0000", 2024-09-14T06:16:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:16:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589313+0000", 2024-09-14T06:16:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1130000000000003e-06 2024-09-14T06:16:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589321+0000", 2024-09-14T06:16:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3370000000000002e-06 2024-09-14T06:16:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589326+0000", 2024-09-14T06:16:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.48e-06 2024-09-14T06:16:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606319+0000", 2024-09-14T06:16:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016993063999999999 2024-09-14T06:16:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606380+0000", 2024-09-14T06:16:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0921000000000002e-05 2024-09-14T06:16:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606429+0000", 2024-09-14T06:16:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8965000000000003e-05 2024-09-14T06:16:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4384 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589335+0000", 2024-09-14T06:16:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.3751904429999993, 2024-09-14T06:16:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017257641000000001, 2024-09-14T06:16:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4384 2024-09-14T06:16:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589335+0000", 2024-09-14T06:16:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589335+0000", 2024-09-14T06:16:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589335+0000", 2024-09-14T06:16:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:16:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589344+0000", 2024-09-14T06:16:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6319999999999997e-06 2024-09-14T06:16:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589351+0000", 2024-09-14T06:16:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1590000000000004e-06 2024-09-14T06:16:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589356+0000", 2024-09-14T06:16:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3070000000000002e-06 2024-09-14T06:16:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606463+0000", 2024-09-14T06:16:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017106857 2024-09-14T06:16:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606534+0000", 2024-09-14T06:16:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0728000000000005e-05 2024-09-14T06:16:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606592+0000", 2024-09-14T06:16:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8508000000000003e-05 2024-09-14T06:16:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4385 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589365+0000", 2024-09-14T06:16:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.3751600170000007, 2024-09-14T06:16:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017375774, 2024-09-14T06:16:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4385 2024-09-14T06:16:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589365+0000", 2024-09-14T06:16:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589365+0000", 2024-09-14T06:16:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589366+0000", 2024-09-14T06:16:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:16:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589368+0000", 2024-09-14T06:16:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.199e-06 2024-09-14T06:16:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589376+0000", 2024-09-14T06:16:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3720000000000005e-06 2024-09-14T06:16:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589381+0000", 2024-09-14T06:16:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2850000000000002e-06 2024-09-14T06:16:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606641+0000", 2024-09-14T06:16:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017259872999999998 2024-09-14T06:16:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606692+0000", 2024-09-14T06:16:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0803000000000001e-05 2024-09-14T06:16:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606741+0000", 2024-09-14T06:16:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8733999999999998e-05 2024-09-14T06:16:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4386 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589389+0000", 2024-09-14T06:16:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.3751358749999998, 2024-09-14T06:16:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017485302000000001, 2024-09-14T06:16:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4386 2024-09-14T06:16:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589389+0000", 2024-09-14T06:16:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589389+0000", 2024-09-14T06:16:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589390+0000", 2024-09-14T06:16:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6199999999999998e-07 2024-09-14T06:16:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589392+0000", 2024-09-14T06:16:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.968e-06 2024-09-14T06:16:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589398+0000", 2024-09-14T06:16:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6000000000000003e-06 2024-09-14T06:16:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589403+0000", 2024-09-14T06:16:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0390000000000004e-06 2024-09-14T06:16:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606773+0000", 2024-09-14T06:16:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017369643000000001 2024-09-14T06:16:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606824+0000", 2024-09-14T06:16:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0871999999999997e-05 2024-09-14T06:16:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606875+0000", 2024-09-14T06:16:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0717999999999998e-05 2024-09-14T06:16:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4387 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589411+0000", 2024-09-14T06:16:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.3751143110000008, 2024-09-14T06:16:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017590590999999999, 2024-09-14T06:16:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4387 2024-09-14T06:16:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589411+0000", 2024-09-14T06:16:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589411+0000", 2024-09-14T06:16:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589411+0000", 2024-09-14T06:16:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4299999999999998e-07 2024-09-14T06:16:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589413+0000", 2024-09-14T06:16:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1339999999999998e-06 2024-09-14T06:16:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589420+0000", 2024-09-14T06:16:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6059999999999999e-06 2024-09-14T06:16:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589425+0000", 2024-09-14T06:16:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0409999999999997e-06 2024-09-14T06:16:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606904+0000", 2024-09-14T06:16:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017479416000000001 2024-09-14T06:16:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.606955+0000", 2024-09-14T06:16:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0853000000000003e-05 2024-09-14T06:16:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607001+0000", 2024-09-14T06:16:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6097999999999998e-05 2024-09-14T06:16:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4388 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589432+0000", 2024-09-14T06:16:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.3750926910000008, 2024-09-14T06:16:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017698337000000001, 2024-09-14T06:16:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4388 2024-09-14T06:16:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589432+0000", 2024-09-14T06:16:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589432+0000", 2024-09-14T06:16:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589433+0000", 2024-09-14T06:16:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2300000000000002e-07 2024-09-14T06:16:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589435+0000", 2024-09-14T06:16:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.902e-06 2024-09-14T06:16:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589441+0000", 2024-09-14T06:16:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2060000000000004e-06 2024-09-14T06:16:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589446+0000", 2024-09-14T06:16:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6500000000000004e-06 2024-09-14T06:16:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607037+0000", 2024-09-14T06:16:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017591496000000002 2024-09-14T06:16:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607084+0000", 2024-09-14T06:16:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6873999999999999e-05 2024-09-14T06:16:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607131+0000", 2024-09-14T06:16:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6786000000000002e-05 2024-09-14T06:16:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4389 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589453+0000", 2024-09-14T06:16:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.3750721890000008, 2024-09-14T06:16:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017801510999999999, 2024-09-14T06:16:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4389 2024-09-14T06:16:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589453+0000", 2024-09-14T06:16:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589453+0000", 2024-09-14T06:16:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589453+0000", 2024-09-14T06:16:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4499999999999997e-07 2024-09-14T06:16:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589455+0000", 2024-09-14T06:16:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0370000000000001e-06 2024-09-14T06:16:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589462+0000", 2024-09-14T06:16:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1360000000000002e-06 2024-09-14T06:16:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589466+0000", 2024-09-14T06:16:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.843e-06 2024-09-14T06:16:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607149+0000", 2024-09-14T06:16:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017682356 2024-09-14T06:16:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607209+0000", 2024-09-14T06:16:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9902999999999999e-05 2024-09-14T06:16:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607254+0000", 2024-09-14T06:16:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5791000000000003e-05 2024-09-14T06:16:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.059 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:36.061 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:16:36.061 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:36.060+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:36.062 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:36.064+0000 7fc937e66640 -1 reset not still connected to 0x557a85be5ba0 2024-09-14T06:16:36.164 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:36.164+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:36.263 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:36.260+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:36.325 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:36.325 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:36.325 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:36.325 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:36.341 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:16:36.387 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:36.384+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:36.465 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:36.468+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:36.499 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:36.499 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:36.499 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:36.499 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:36.499 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:36.512 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:16:36.538 INFO:teuthology.orchestra.run.smithi039.stdout:801 punch from 0x367570 to 0x372140, (0xabd0 bytes) 2024-09-14T06:16:36.546 INFO:teuthology.orchestra.run.smithi039.stdout:802 write 0x5e15ad0 thru 0x5e199e5 (0x3f16 bytes) 2024-09-14T06:16:36.565 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:36.560+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:36.585 INFO:teuthology.orchestra.run.smithi039.stdout:803 read 0xdfc90df thru 0xdfc9ac1 (0x9e3 bytes) 2024-09-14T06:16:36.585 INFO:teuthology.orchestra.run.smithi039.stdout:804 write 0xd71d3d1 thru 0xd723b55 (0x6785 bytes) 2024-09-14T06:16:36.605 INFO:teuthology.orchestra.run.smithi039.stdout:805 write 0xbc04cca thru 0xbc0fd44 (0xb07b bytes) 2024-09-14T06:16:36.615 INFO:teuthology.orchestra.run.smithi039.stdout:806 trunc from 0xe7c9710 to 0x818ccd9 2024-09-14T06:16:36.646 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:36.646 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:36.646 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:36.646 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:36.646 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.646 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:36.646 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:36.646 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 32.568216913999997, 2024-09-14T06:16:36.646 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.56009052699999995, 2024-09-14T06:16:36.646 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.646 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.647 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:36.647 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.647 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:36.647 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:36.647 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 1 2024-09-14T06:16:36.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.647 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.647 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.647 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.647 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:36.647 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.647 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.648 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.648 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:36.648 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.648 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.648 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.648 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077065+0000", 2024-09-14T06:16:36.648 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6429999999999999e-06 2024-09-14T06:16:36.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.648 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.648 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.648 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077079+0000", 2024-09-14T06:16:36.648 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4447000000000001e-05 2024-09-14T06:16:36.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.649 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.649 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.649 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077103+0000", 2024-09-14T06:16:36.649 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3856e-05 2024-09-14T06:16:36.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.649 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.649 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.649 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077120+0000", 2024-09-14T06:16:36.649 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7337e-05 2024-09-14T06:16:36.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.649 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.649 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.650 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077255+0000", 2024-09-14T06:16:36.650 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000134652 2024-09-14T06:16:36.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.650 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.650 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:16:36.650 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077274+0000", 2024-09-14T06:16:36.650 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9003000000000001e-05 2024-09-14T06:16:36.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.650 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.650 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.650 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634272+0000", 2024-09-14T06:16:36.650 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.55699813099999995 2024-09-14T06:16:36.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634591+0000", 2024-09-14T06:16:36.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031910800000000002 2024-09-14T06:16:36.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:36.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634892+0000", 2024-09-14T06:16:36.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00030066400000000002 2024-09-14T06:16:36.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:36.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636345+0000", 2024-09-14T06:16:36.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001452896 2024-09-14T06:16:36.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.652 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:36.652 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636974+0000", 2024-09-14T06:16:36.652 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00062927800000000004 2024-09-14T06:16:36.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.652 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:36.652 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637071+0000", 2024-09-14T06:16:36.652 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7178999999999999e-05 2024-09-14T06:16:36.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.652 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.652 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637153+0000", 2024-09-14T06:16:36.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1333000000000002e-05 2024-09-14T06:16:36.653 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.653 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.653 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.653 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.653 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13136 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.653 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.091799+0000", 2024-09-14T06:16:36.653 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55347997100000002, 2024-09-14T06:16:36.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022628747000000001, 2024-09-14T06:16:36.653 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.653 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.653 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.654 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.654 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.654 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.654 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13136 2024-09-14T06:16:36.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.654 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.654 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.654 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.654 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091799+0000", 2024-09-14T06:16:36.654 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.654 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.654 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.654 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091799+0000", 2024-09-14T06:16:36.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.655 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091800+0000", 2024-09-14T06:16:36.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.68e-07 2024-09-14T06:16:36.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.655 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091802+0000", 2024-09-14T06:16:36.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6699999999999998e-06 2024-09-14T06:16:36.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.656 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091810+0000", 2024-09-14T06:16:36.656 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6750000000000002e-06 2024-09-14T06:16:36.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.656 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.656 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091816+0000", 2024-09-14T06:16:36.656 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1600000000000003e-06 2024-09-14T06:16:36.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.656 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.656 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.110345+0000", 2024-09-14T06:16:36.656 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018528393000000001 2024-09-14T06:16:36.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.657 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.657 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.111299+0000", 2024-09-14T06:16:36.657 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00095451100000000001 2024-09-14T06:16:36.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.657 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.657 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114428+0000", 2024-09-14T06:16:36.657 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031287699999999999 2024-09-14T06:16:36.657 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.657 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.657 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.658 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13150 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.658 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092128+0000", 2024-09-14T06:16:36.658 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.553151485, 2024-09-14T06:16:36.658 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.021923827999999999, 2024-09-14T06:16:36.658 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.658 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.658 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.658 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.658 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.658 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.658 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13150 2024-09-14T06:16:36.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.658 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.659 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.659 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.659 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092128+0000", 2024-09-14T06:16:36.659 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.659 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.659 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.659 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.659 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092128+0000", 2024-09-14T06:16:36.659 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.659 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.659 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.659 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.659 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092128+0000", 2024-09-14T06:16:36.659 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:16:36.659 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.660 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.660 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.660 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092130+0000", 2024-09-14T06:16:36.660 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1730000000000002e-06 2024-09-14T06:16:36.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.660 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.660 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.660 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092136+0000", 2024-09-14T06:16:36.660 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7780000000000004e-06 2024-09-14T06:16:36.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.660 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.660 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.660 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092140+0000", 2024-09-14T06:16:36.661 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5859999999999998e-06 2024-09-14T06:16:36.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.661 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.661 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.661 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.113894+0000", 2024-09-14T06:16:36.661 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.021753660000000001 2024-09-14T06:16:36.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.661 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.661 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.661 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.113970+0000", 2024-09-14T06:16:36.661 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6334999999999993e-05 2024-09-14T06:16:36.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.661 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114051+0000", 2024-09-14T06:16:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0884999999999996e-05 2024-09-14T06:16:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13153 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092161+0000", 2024-09-14T06:16:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55311823999999998, 2024-09-14T06:16:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022150551000000001, 2024-09-14T06:16:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.663 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.663 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.663 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.663 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.663 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13153 2024-09-14T06:16:36.663 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.663 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.663 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.663 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.663 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092161+0000", 2024-09-14T06:16:36.663 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.663 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.663 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.664 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.664 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092161+0000", 2024-09-14T06:16:36.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.664 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.664 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.664 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092161+0000", 2024-09-14T06:16:36.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.68e-07 2024-09-14T06:16:36.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.664 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.664 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.664 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092164+0000", 2024-09-14T06:16:36.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:16:36.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.665 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.665 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092172+0000", 2024-09-14T06:16:36.665 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8029999999999997e-06 2024-09-14T06:16:36.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.665 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.665 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092178+0000", 2024-09-14T06:16:36.665 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6210000000000001e-06 2024-09-14T06:16:36.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.665 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.665 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114130+0000", 2024-09-14T06:16:36.666 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.021952701000000002 2024-09-14T06:16:36.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.666 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.666 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114217+0000", 2024-09-14T06:16:36.666 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6527999999999996e-05 2024-09-14T06:16:36.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.666 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.666 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114311+0000", 2024-09-14T06:16:36.666 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4394000000000005e-05 2024-09-14T06:16:36.666 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.666 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.666 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.667 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.667 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13154 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.667 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092189+0000", 2024-09-14T06:16:36.667 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55309012199999996, 2024-09-14T06:16:36.667 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022359604000000002, 2024-09-14T06:16:36.667 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.667 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.667 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.667 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.667 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.667 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.667 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13154 2024-09-14T06:16:36.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.668 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.668 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.668 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.668 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092189+0000", 2024-09-14T06:16:36.668 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.668 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.668 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.668 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092189+0000", 2024-09-14T06:16:36.668 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.668 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.668 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.669 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092189+0000", 2024-09-14T06:16:36.669 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9700000000000002e-07 2024-09-14T06:16:36.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.669 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.669 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.669 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092192+0000", 2024-09-14T06:16:36.669 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.317e-06 2024-09-14T06:16:36.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.669 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.669 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.669 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092198+0000", 2024-09-14T06:16:36.669 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.387e-06 2024-09-14T06:16:36.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.669 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.670 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.670 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092203+0000", 2024-09-14T06:16:36.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.259e-06 2024-09-14T06:16:36.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.670 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.670 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.670 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114338+0000", 2024-09-14T06:16:36.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022135207 2024-09-14T06:16:36.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.670 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.670 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.670 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114451+0000", 2024-09-14T06:16:36.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000112518 2024-09-14T06:16:36.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114548+0000", 2024-09-14T06:16:36.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7518999999999999e-05 2024-09-14T06:16:36.671 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.671 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.671 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.671 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13155 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.671 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092211+0000", 2024-09-14T06:16:36.671 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55306850799999996, 2024-09-14T06:16:36.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.02255184, 2024-09-14T06:16:36.671 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.672 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.672 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.672 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.672 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.672 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.672 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13155 2024-09-14T06:16:36.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.672 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.672 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.672 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092211+0000", 2024-09-14T06:16:36.672 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092211+0000", 2024-09-14T06:16:36.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092211+0000", 2024-09-14T06:16:36.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8500000000000002e-07 2024-09-14T06:16:36.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092213+0000", 2024-09-14T06:16:36.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1380000000000001e-06 2024-09-14T06:16:36.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.674 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.674 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092220+0000", 2024-09-14T06:16:36.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5250000000000002e-06 2024-09-14T06:16:36.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.674 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.674 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092224+0000", 2024-09-14T06:16:36.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.843e-06 2024-09-14T06:16:36.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.675 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.675 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.675 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114580+0000", 2024-09-14T06:16:36.675 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022355455999999999 2024-09-14T06:16:36.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.675 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.675 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.675 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114673+0000", 2024-09-14T06:16:36.675 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2987999999999998e-05 2024-09-14T06:16:36.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.675 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.675 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.675 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114762+0000", 2024-09-14T06:16:36.676 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9505000000000004e-05 2024-09-14T06:16:36.676 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.676 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.676 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.676 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.676 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13156 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 409600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.676 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092232+0000", 2024-09-14T06:16:36.676 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55304742200000001, 2024-09-14T06:16:36.676 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022738728999999999, 2024-09-14T06:16:36.676 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.676 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.676 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.677 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.677 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.677 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.677 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13156 2024-09-14T06:16:36.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.677 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.677 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.677 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092232+0000", 2024-09-14T06:16:36.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.677 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.677 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092232+0000", 2024-09-14T06:16:36.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.678 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092232+0000", 2024-09-14T06:16:36.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8099999999999998e-07 2024-09-14T06:16:36.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.678 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092234+0000", 2024-09-14T06:16:36.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0379999999999998e-06 2024-09-14T06:16:36.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092240+0000", 2024-09-14T06:16:36.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2859999999999996e-06 2024-09-14T06:16:36.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092248+0000", 2024-09-14T06:16:36.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5140000000000004e-06 2024-09-14T06:16:36.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114792+0000", 2024-09-14T06:16:36.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022543758000000001 2024-09-14T06:16:36.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114865+0000", 2024-09-14T06:16:36.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3652000000000006e-05 2024-09-14T06:16:36.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114970+0000", 2024-09-14T06:16:36.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010509999999999999 2024-09-14T06:16:36.680 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.680 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.680 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.681 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13157 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 524288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.681 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092256+0000", 2024-09-14T06:16:36.681 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55302349500000003, 2024-09-14T06:16:36.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022964834999999999, 2024-09-14T06:16:36.681 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.681 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.681 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.681 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.681 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.681 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.681 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13157 2024-09-14T06:16:36.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.682 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.682 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.682 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092256+0000", 2024-09-14T06:16:36.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.682 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.682 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092256+0000", 2024-09-14T06:16:36.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092256+0000", 2024-09-14T06:16:36.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6199999999999998e-07 2024-09-14T06:16:36.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092258+0000", 2024-09-14T06:16:36.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3530000000000002e-06 2024-09-14T06:16:36.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092265+0000", 2024-09-14T06:16:36.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5069999999999997e-06 2024-09-14T06:16:36.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092270+0000", 2024-09-14T06:16:36.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2530000000000004e-06 2024-09-14T06:16:36.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115000+0000", 2024-09-14T06:16:36.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022729787000000001 2024-09-14T06:16:36.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115074+0000", 2024-09-14T06:16:36.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4506999999999997e-05 2024-09-14T06:16:36.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115220+0000", 2024-09-14T06:16:36.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000145966 2024-09-14T06:16:36.685 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.685 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.685 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.685 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13158 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.685 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092282+0000", 2024-09-14T06:16:36.686 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55299662299999996, 2024-09-14T06:16:36.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.023233365999999998, 2024-09-14T06:16:36.686 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.686 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.686 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.686 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.686 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.686 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.686 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13158 2024-09-14T06:16:36.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.686 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092282+0000", 2024-09-14T06:16:36.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092282+0000", 2024-09-14T06:16:36.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092283+0000", 2024-09-14T06:16:36.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:16:36.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092285+0000", 2024-09-14T06:16:36.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3460000000000001e-06 2024-09-14T06:16:36.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092291+0000", 2024-09-14T06:16:36.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0549999999999996e-06 2024-09-14T06:16:36.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092296+0000", 2024-09-14T06:16:36.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1800000000000004e-06 2024-09-14T06:16:36.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115253+0000", 2024-09-14T06:16:36.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022956764000000001 2024-09-14T06:16:36.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115347+0000", 2024-09-14T06:16:36.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3762000000000001e-05 2024-09-14T06:16:36.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115516+0000", 2024-09-14T06:16:36.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016881200000000001 2024-09-14T06:16:36.689 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.690 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.690 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.690 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13159 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.690 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092307+0000", 2024-09-14T06:16:36.690 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55297248700000001, 2024-09-14T06:16:36.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.023865384999999999, 2024-09-14T06:16:36.690 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.690 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.690 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.690 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.690 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.691 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.691 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13159 2024-09-14T06:16:36.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.691 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092307+0000", 2024-09-14T06:16:36.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092307+0000", 2024-09-14T06:16:36.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092307+0000", 2024-09-14T06:16:36.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:16:36.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092309+0000", 2024-09-14T06:16:36.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3080000000000002e-06 2024-09-14T06:16:36.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092316+0000", 2024-09-14T06:16:36.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7120000000000003e-06 2024-09-14T06:16:36.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092321+0000", 2024-09-14T06:16:36.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7729999999999999e-06 2024-09-14T06:16:36.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115546+0000", 2024-09-14T06:16:36.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.023225000999999999 2024-09-14T06:16:36.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115621+0000", 2024-09-14T06:16:36.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4981999999999995e-05 2024-09-14T06:16:36.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.694 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116172+0000", 2024-09-14T06:16:36.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00055117199999999995 2024-09-14T06:16:36.694 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.694 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.694 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.694 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13160 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.694 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092329+0000", 2024-09-14T06:16:36.695 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55295033000000005, 2024-09-14T06:16:36.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024062849000000001, 2024-09-14T06:16:36.695 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.695 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.695 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.695 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.695 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.695 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.695 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13160 2024-09-14T06:16:36.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.695 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092329+0000", 2024-09-14T06:16:36.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092329+0000", 2024-09-14T06:16:36.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092329+0000", 2024-09-14T06:16:36.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8299999999999997e-07 2024-09-14T06:16:36.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092331+0000", 2024-09-14T06:16:36.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:16:36.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092338+0000", 2024-09-14T06:16:36.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:16:36.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092351+0000", 2024-09-14T06:16:36.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3059e-05 2024-09-14T06:16:36.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116204+0000", 2024-09-14T06:16:36.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.023853268 2024-09-14T06:16:36.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116287+0000", 2024-09-14T06:16:36.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3140999999999996e-05 2024-09-14T06:16:36.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116392+0000", 2024-09-14T06:16:36.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010451900000000001 2024-09-14T06:16:36.699 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.699 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.699 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.699 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13161 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.699 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092359+0000", 2024-09-14T06:16:36.699 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.552919932, 2024-09-14T06:16:36.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024248515000000002, 2024-09-14T06:16:36.699 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.699 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.699 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.699 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.700 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.700 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.700 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13161 2024-09-14T06:16:36.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.700 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.700 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.700 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092359+0000", 2024-09-14T06:16:36.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.700 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.700 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092359+0000", 2024-09-14T06:16:36.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092360+0000", 2024-09-14T06:16:36.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:16:36.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092368+0000", 2024-09-14T06:16:36.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1140000000000006e-06 2024-09-14T06:16:36.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092375+0000", 2024-09-14T06:16:36.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4399999999999999e-06 2024-09-14T06:16:36.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092380+0000", 2024-09-14T06:16:36.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3170000000000001e-06 2024-09-14T06:16:36.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116426+0000", 2024-09-14T06:16:36.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024045492000000002 2024-09-14T06:16:36.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116513+0000", 2024-09-14T06:16:36.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6936999999999998e-05 2024-09-14T06:16:36.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116608+0000", 2024-09-14T06:16:36.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4765000000000005e-05 2024-09-14T06:16:36.703 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.703 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.703 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.704 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13162 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.704 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092389+0000", 2024-09-14T06:16:36.704 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55289012000000004, 2024-09-14T06:16:36.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024432156, 2024-09-14T06:16:36.704 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.704 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.704 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.704 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.704 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.704 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.704 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13162 2024-09-14T06:16:36.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.704 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092389+0000", 2024-09-14T06:16:36.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092389+0000", 2024-09-14T06:16:36.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092389+0000", 2024-09-14T06:16:36.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:16:36.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092392+0000", 2024-09-14T06:16:36.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3460000000000001e-06 2024-09-14T06:16:36.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092398+0000", 2024-09-14T06:16:36.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6789999999999999e-06 2024-09-14T06:16:36.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092403+0000", 2024-09-14T06:16:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9389999999999996e-06 2024-09-14T06:16:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116657+0000", 2024-09-14T06:16:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024253896 2024-09-14T06:16:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116732+0000", 2024-09-14T06:16:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4727000000000005e-05 2024-09-14T06:16:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116821+0000", 2024-09-14T06:16:36.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:16:36.708 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.708 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.708 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.708 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13163 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.708 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092410+0000", 2024-09-14T06:16:36.708 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55286866199999996, 2024-09-14T06:16:36.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024727810999999999, 2024-09-14T06:16:36.708 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.709 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.709 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.709 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.709 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.709 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.709 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13163 2024-09-14T06:16:36.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.709 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092410+0000", 2024-09-14T06:16:36.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092410+0000", 2024-09-14T06:16:36.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092411+0000", 2024-09-14T06:16:36.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2899999999999999e-07 2024-09-14T06:16:36.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092413+0000", 2024-09-14T06:16:36.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3649999999999998e-06 2024-09-14T06:16:36.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092420+0000", 2024-09-14T06:16:36.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5729999999999996e-06 2024-09-14T06:16:36.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092425+0000", 2024-09-14T06:16:36.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0669999999999999e-06 2024-09-14T06:16:36.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116850+0000", 2024-09-14T06:16:36.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024424833999999999 2024-09-14T06:16:36.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117058+0000", 2024-09-14T06:16:36.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020800199999999999 2024-09-14T06:16:36.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117138+0000", 2024-09-14T06:16:36.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0541e-05 2024-09-14T06:16:36.712 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.712 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.713 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.713 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13164 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.713 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092432+0000", 2024-09-14T06:16:36.713 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55284698600000004, 2024-09-14T06:16:36.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024878806999999999, 2024-09-14T06:16:36.713 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.713 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.713 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.713 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.713 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.713 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.713 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13164 2024-09-14T06:16:36.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.714 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092432+0000", 2024-09-14T06:16:36.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092432+0000", 2024-09-14T06:16:36.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092432+0000", 2024-09-14T06:16:36.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:16:36.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092434+0000", 2024-09-14T06:16:36.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.968e-06 2024-09-14T06:16:36.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092440+0000", 2024-09-14T06:16:36.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.964e-06 2024-09-14T06:16:36.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092445+0000", 2024-09-14T06:16:36.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9529999999999998e-06 2024-09-14T06:16:36.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117166+0000", 2024-09-14T06:16:36.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024720925000000001 2024-09-14T06:16:36.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117239+0000", 2024-09-14T06:16:36.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2650000000000004e-05 2024-09-14T06:16:36.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117311+0000", 2024-09-14T06:16:36.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1945000000000003e-05 2024-09-14T06:16:36.717 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.717 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.717 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.717 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13165 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.717 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092453+0000", 2024-09-14T06:16:36.717 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55282630499999996, 2024-09-14T06:16:36.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025052076999999999, 2024-09-14T06:16:36.718 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.718 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.718 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.718 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.718 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.718 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.718 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13165 2024-09-14T06:16:36.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.718 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092453+0000", 2024-09-14T06:16:36.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092453+0000", 2024-09-14T06:16:36.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092453+0000", 2024-09-14T06:16:36.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7300000000000001e-07 2024-09-14T06:16:36.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092455+0000", 2024-09-14T06:16:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.204e-06 2024-09-14T06:16:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092461+0000", 2024-09-14T06:16:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0299999999999999e-06 2024-09-14T06:16:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092466+0000", 2024-09-14T06:16:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7820000000000001e-06 2024-09-14T06:16:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117343+0000", 2024-09-14T06:16:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024876334999999999 2024-09-14T06:16:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117416+0000", 2024-09-14T06:16:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3368000000000006e-05 2024-09-14T06:16:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117505+0000", 2024-09-14T06:16:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8885e-05 2024-09-14T06:16:36.721 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13166 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092490+0000", 2024-09-14T06:16:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55278914199999996, 2024-09-14T06:16:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025236825000000001, 2024-09-14T06:16:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.722 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13166 2024-09-14T06:16:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092490+0000", 2024-09-14T06:16:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092490+0000", 2024-09-14T06:16:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092490+0000", 2024-09-14T06:16:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6199999999999999e-07 2024-09-14T06:16:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092492+0000", 2024-09-14T06:16:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7910000000000001e-06 2024-09-14T06:16:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092497+0000", 2024-09-14T06:16:36.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4190000000000001e-06 2024-09-14T06:16:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092501+0000", 2024-09-14T06:16:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0230000000000003e-06 2024-09-14T06:16:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117533+0000", 2024-09-14T06:16:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025031623999999999 2024-09-14T06:16:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117638+0000", 2024-09-14T06:16:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010451900000000001 2024-09-14T06:16:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117727+0000", 2024-09-14T06:16:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9086999999999996e-05 2024-09-14T06:16:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13167 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092507+0000", 2024-09-14T06:16:36.726 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55277165399999995, 2024-09-14T06:16:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025412396, 2024-09-14T06:16:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13167 2024-09-14T06:16:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092507+0000", 2024-09-14T06:16:36.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092507+0000", 2024-09-14T06:16:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092508+0000", 2024-09-14T06:16:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.65e-07 2024-09-14T06:16:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092509+0000", 2024-09-14T06:16:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.426e-06 2024-09-14T06:16:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092512+0000", 2024-09-14T06:16:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.416e-06 2024-09-14T06:16:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092515+0000", 2024-09-14T06:16:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.971e-06 2024-09-14T06:16:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117755+0000", 2024-09-14T06:16:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.02523999 2024-09-14T06:16:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117830+0000", 2024-09-14T06:16:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4386000000000002e-05 2024-09-14T06:16:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117920+0000", 2024-09-14T06:16:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9942e-05 2024-09-14T06:16:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.730 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.731 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.731 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13168 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.731 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092519+0000", 2024-09-14T06:16:36.731 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55275952100000003, 2024-09-14T06:16:36.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025607092000000001, 2024-09-14T06:16:36.731 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.731 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.731 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.731 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.731 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.731 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.732 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13168 2024-09-14T06:16:36.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.732 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092519+0000", 2024-09-14T06:16:36.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092519+0000", 2024-09-14T06:16:36.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092520+0000", 2024-09-14T06:16:36.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4999999999999999e-07 2024-09-14T06:16:36.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092521+0000", 2024-09-14T06:16:36.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2440000000000001e-06 2024-09-14T06:16:36.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092524+0000", 2024-09-14T06:16:36.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3189999999999999e-06 2024-09-14T06:16:36.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092527+0000", 2024-09-14T06:16:36.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.638e-06 2024-09-14T06:16:36.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117948+0000", 2024-09-14T06:16:36.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025421293000000001 2024-09-14T06:16:36.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.118025+0000", 2024-09-14T06:16:36.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6410999999999997e-05 2024-09-14T06:16:36.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.118127+0000", 2024-09-14T06:16:36.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000101937 2024-09-14T06:16:36.735 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.736 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.736 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.736 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13169 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:36.736 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092531+0000", 2024-09-14T06:16:36.736 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.55274832399999996, 2024-09-14T06:16:36.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.030222784999999999, 2024-09-14T06:16:36.736 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:36.736 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:36.736 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:36.736 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:36.736 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:36.736 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:36.737 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13169 2024-09-14T06:16:36.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.737 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:36.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:36.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092531+0000", 2024-09-14T06:16:36.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:36.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092531+0000", 2024-09-14T06:16:36.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:36.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:36.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092531+0000", 2024-09-14T06:16:36.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.41e-07 2024-09-14T06:16:36.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:36.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092545+0000", 2024-09-14T06:16:36.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4256999999999999e-05 2024-09-14T06:16:36.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:36.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092553+0000", 2024-09-14T06:16:36.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.374e-06 2024-09-14T06:16:36.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:36.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092558+0000", 2024-09-14T06:16:36.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2830000000000001e-06 2024-09-14T06:16:36.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:36.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.122584+0000", 2024-09-14T06:16:36.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.030026091000000001 2024-09-14T06:16:36.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:36.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.122672+0000", 2024-09-14T06:16:36.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7880999999999994e-05 2024-09-14T06:16:36.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:36.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:36.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:36.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.122753+0000", 2024-09-14T06:16:36.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1657999999999996e-05 2024-09-14T06:16:36.740 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.740 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.740 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.740 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:36.740 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:36.740 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:36.741 INFO:teuthology.orchestra.run.smithi039.stdout:809 read 0x42df2d9 thru 0x42ea98b (0xb6b3 bytes) 2024-09-14T06:16:36.742 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:16:36.743 INFO:teuthology.orchestra.run.smithi039.stdout:811 writesame 0xc124584 thru 0xc13050c data_size 0x1(0xbf89 bytes) 2024-09-14T06:16:36.743 INFO:teuthology.orchestra.run.smithi039.stdout:812 punch from 0x758bf4 to 0x76559b, (0xc9a7 bytes) 2024-09-14T06:16:36.743 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:36.736+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:36.744 INFO:teuthology.orchestra.run.smithi039.stdout:813 write 0xd59a6a9 thru 0xd5a7aae (0xd406 bytes) 2024-09-14T06:16:36.747 INFO:teuthology.orchestra.run.smithi039.stdout:814 punch from 0x1cdf882 to 0x1ce4d61, (0x54df bytes) 2024-09-14T06:16:36.748 INFO:teuthology.orchestra.run.smithi039.stdout:815 write 0xc5894a8 thru 0xc59526b (0xbdc4 bytes) 2024-09-14T06:16:36.751 INFO:teuthology.orchestra.run.smithi039.stdout:817 write 0x1f8076a thru 0x1f8abe3 (0xa47a bytes) 2024-09-14T06:16:36.754 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:36.754 INFO:teuthology.orchestra.run.smithi039.stdout:820 writesame 0x23ce699 thru 0x23dbf65 data_size 0x1(0xd8cd bytes) 2024-09-14T06:16:36.821 INFO:teuthology.orchestra.run.smithi039.stdout:821 writesame 0x1271f5e thru 0x1277cef data_size 0x3b(0x5d92 bytes) 2024-09-14T06:16:36.843 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:36.840+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:36.859 INFO:teuthology.orchestra.run.smithi039.stdout:822 read 0x8188982 thru 0x8197db4 (0xf433 bytes) 2024-09-14T06:16:36.860 INFO:teuthology.orchestra.run.smithi039.stdout:823 writesame 0xd1c4ec8 thru 0xd1ca967 data_size 0x28(0x5aa0 bytes) 2024-09-14T06:16:36.863 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:36.863 INFO:teuthology.orchestra.run.smithi039.stdout:825 write 0x6fd06c1 thru 0x6fd665f (0x5f9f bytes) 2024-09-14T06:16:36.882 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:36.882 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:36.882 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:36.882 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:36.898 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:16:36.901 INFO:teuthology.orchestra.run.smithi039.stdout:827 write 0x5c8ac74 thru 0x5c8d0a8 (0x2435 bytes) 2024-09-14T06:16:36.929 INFO:teuthology.orchestra.run.smithi039.stdout:829 write 0xd091fb8 thru 0xd09dc2f (0xbc78 bytes) 2024-09-14T06:16:36.941 INFO:teuthology.orchestra.run.smithi039.stdout:830 trunc from 0xd5a7aaf to 0x647057b 2024-09-14T06:16:36.943 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:36.940+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:36.997 INFO:teuthology.orchestra.run.smithi039.stdout:831 read 0xa32114 thru 0xa3b2b9 (0x91a6 bytes) 2024-09-14T06:16:36.998 INFO:teuthology.orchestra.run.smithi039.stdout:832 read 0x1759cef thru 0x176284b (0x8b5d bytes) 2024-09-14T06:16:36.999 INFO:teuthology.orchestra.run.smithi039.stdout:835 read 0x27e316e thru 0x27eac31 (0x7ac4 bytes) 2024-09-14T06:16:37.000 INFO:teuthology.orchestra.run.smithi039.stdout:837 read 0x428076f thru 0x428988c (0x911e bytes) 2024-09-14T06:16:37.024 INFO:teuthology.orchestra.run.smithi039.stdout:841 writesame 0xb3c111a thru 0xb3cc6fe data_size 0x43(0xb5e5 bytes) 2024-09-14T06:16:37.035 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:37.035 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:37.035 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:37.035 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:37.035 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:37.043 INFO:teuthology.orchestra.run.smithi039.stdout:842 write 0xdb5fea4 thru 0xdb6edf4 (0xef51 bytes) 2024-09-14T06:16:37.044 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:37.040+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:37.049 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:16:37.051 INFO:teuthology.orchestra.run.smithi039.stdout:843 writesame 0x109c355 thru 0x10a49b0 data_size 0x2(0x865c bytes) 2024-09-14T06:16:37.064 INFO:teuthology.orchestra.run.smithi039.stdout:844 read 0x948cd4c thru 0x94989df (0xbc94 bytes) 2024-09-14T06:16:37.064 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:37.064 INFO:teuthology.orchestra.run.smithi039.stdout:847 punch from 0x1c141d4 to 0x1c17aeb, (0x3917 bytes) 2024-09-14T06:16:37.064 INFO:teuthology.orchestra.run.smithi039.stdout:848 write 0x52a170f thru 0x52a6f7e (0x5870 bytes) 2024-09-14T06:16:37.068 INFO:teuthology.orchestra.run.smithi039.stdout:850 read 0xd884362 thru 0xd89168d (0xd32c bytes) 2024-09-14T06:16:37.079 INFO:teuthology.orchestra.run.smithi039.stdout:851 trunc from 0xdb6edf5 to 0x5350e1a 2024-09-14T06:16:37.103 INFO:teuthology.orchestra.run.smithi039.stdout:854 read 0x1d8ba32 thru 0x1d9b6bf (0xfc8e bytes) 2024-09-14T06:16:37.103 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:37.104 INFO:teuthology.orchestra.run.smithi039.stdout:856 punch from 0x2ba24c3 to 0x2bac6bd, (0xa1fa bytes) 2024-09-14T06:16:37.104 INFO:teuthology.orchestra.run.smithi039.stdout:857 read 0x2631173 thru 0x2639aad (0x893b bytes) 2024-09-14T06:16:37.105 INFO:teuthology.orchestra.run.smithi039.stdout:859 trunc from 0x5350e1a to 0x62a39bc 2024-09-14T06:16:37.141 INFO:teuthology.orchestra.run.smithi039.stdout:860 write 0xe7810ba thru 0xe7852f2 (0x4239 bytes) 2024-09-14T06:16:37.145 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:37.148+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:37.159 INFO:teuthology.orchestra.run.smithi039.stdout:864 trunc from 0xe7852f3 to 0x6b78ab5 2024-09-14T06:16:37.188 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:37.190 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:37.190 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:37.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:37.191 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:37.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.191 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.5 30/29 [PushOp(3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head, version: 28'5, data_included: [0~212992,360448~311296], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head@28'5, size: 1032192, copy_subset: [(0, 1032192)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1032192, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:16:37.191 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:37.191 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.9121836549999998, 2024-09-14T06:16:37.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041630249999999999, 2024-09-14T06:16:37.191 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.191 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.191 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.191 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:37.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:37.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275888+0000", 2024-09-14T06:16:37.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.367e-06 2024-09-14T06:16:37.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277098+0000", 2024-09-14T06:16:37.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001210661 2024-09-14T06:16:37.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277122+0000", 2024-09-14T06:16:37.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3413000000000001e-05 2024-09-14T06:16:37.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277141+0000", 2024-09-14T06:16:37.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9133000000000001e-05 2024-09-14T06:16:37.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277364+0000", 2024-09-14T06:16:37.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000223365 2024-09-14T06:16:37.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.194 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277405+0000", 2024-09-14T06:16:37.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1053000000000001e-05 2024-09-14T06:16:37.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.194 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.280049+0000", 2024-09-14T06:16:37.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026440330000000001 2024-09-14T06:16:37.195 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.195 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.195 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.195 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.16 30/29 [PushOp(3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head, version: 28'3, data_included: [0~212992,344064~327680], data_size: 540672, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head@28'3, size: 1032192, copy_subset: [(0, 1032192)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1032192, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:16:37.195 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.287894+0000", 2024-09-14T06:16:37.195 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.900175462, 2024-09-14T06:16:37.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032539600000000002, 2024-09-14T06:16:37.195 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.195 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.195 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.195 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.287894+0000", 2024-09-14T06:16:37.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.287894+0000", 2024-09-14T06:16:37.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.287899+0000", 2024-09-14T06:16:37.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7280000000000003e-06 2024-09-14T06:16:37.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.288805+0000", 2024-09-14T06:16:37.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00090622700000000001 2024-09-14T06:16:37.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.288855+0000", 2024-09-14T06:16:37.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9835e-05 2024-09-14T06:16:37.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.288902+0000", 2024-09-14T06:16:37.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6854999999999998e-05 2024-09-14T06:16:37.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.289120+0000", 2024-09-14T06:16:37.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000217495 2024-09-14T06:16:37.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.289142+0000", 2024-09-14T06:16:37.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2870999999999998e-05 2024-09-14T06:16:37.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.291148+0000", 2024-09-14T06:16:37.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020059489999999999 2024-09-14T06:16:37.199 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.199 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.199 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.199 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3865 3.5 3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head [stat,truncate 1998848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:37.199 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.502395+0000", 2024-09-14T06:16:37.199 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.6856749479999991, 2024-09-14T06:16:37.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030449240000000001, 2024-09-14T06:16:37.199 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.199 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.199 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.199 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.200 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.200 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.200 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3865 2024-09-14T06:16:37.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.200 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502395+0000", 2024-09-14T06:16:37.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502395+0000", 2024-09-14T06:16:37.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502397+0000", 2024-09-14T06:16:37.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:16:37.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502403+0000", 2024-09-14T06:16:37.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3409999999999999e-06 2024-09-14T06:16:37.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502426+0000", 2024-09-14T06:16:37.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3181e-05 2024-09-14T06:16:37.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502437+0000", 2024-09-14T06:16:37.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0749e-05 2024-09-14T06:16:37.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502510+0000", 2024-09-14T06:16:37.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3606e-05 2024-09-14T06:16:37.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502595+0000", 2024-09-14T06:16:37.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5018999999999994e-05 2024-09-14T06:16:37.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:37.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502745+0000", 2024-09-14T06:16:37.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00014936600000000001 2024-09-14T06:16:37.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:37.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.504081+0000", 2024-09-14T06:16:37.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0013359310000000001 2024-09-14T06:16:37.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:37.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505355+0000", 2024-09-14T06:16:37.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001273887 2024-09-14T06:16:37.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505402+0000", 2024-09-14T06:16:37.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7515000000000002e-05 2024-09-14T06:16:37.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505440+0000", 2024-09-14T06:16:37.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7805999999999997e-05 2024-09-14T06:16:37.204 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.205 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.205 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.205 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3868 3.16 3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head [stat,truncate 1998848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:37.205 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:37.205 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.6855882849999997, 2024-09-14T06:16:37.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034445919999999998, 2024-09-14T06:16:37.205 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.205 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.205 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.205 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.205 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.205 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.206 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3868 2024-09-14T06:16:37.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.206 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:37.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:37.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:37.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1900000000000002e-07 2024-09-14T06:16:37.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502501+0000", 2024-09-14T06:16:37.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8340999999999999e-05 2024-09-14T06:16:37.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502512+0000", 2024-09-14T06:16:37.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1347e-05 2024-09-14T06:16:37.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.208 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.208 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502518+0000", 2024-09-14T06:16:37.208 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.947e-06 2024-09-14T06:16:37.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.208 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.208 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.208 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502570+0000", 2024-09-14T06:16:37.208 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1858999999999999e-05 2024-09-14T06:16:37.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.208 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.208 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.208 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502633+0000", 2024-09-14T06:16:37.208 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2974999999999999e-05 2024-09-14T06:16:37.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:16:37.209 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502795+0000", 2024-09-14T06:16:37.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016262899999999999 2024-09-14T06:16:37.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:37.209 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505179+0000", 2024-09-14T06:16:37.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002383844 2024-09-14T06:16:37.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:37.209 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505676+0000", 2024-09-14T06:16:37.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00049634899999999999 2024-09-14T06:16:37.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.210 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.210 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.210 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505816+0000", 2024-09-14T06:16:37.210 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00014026300000000001 2024-09-14T06:16:37.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.210 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.210 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.210 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505926+0000", 2024-09-14T06:16:37.210 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000110319 2024-09-14T06:16:37.210 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.210 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.210 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.210 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.210 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3902 3.5 3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:37.211 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.569016+0000", 2024-09-14T06:16:37.211 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.6190544140000007, 2024-09-14T06:16:37.211 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003356197, 2024-09-14T06:16:37.211 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.211 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.211 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.211 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.211 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.211 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.211 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3902 2024-09-14T06:16:37.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.211 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.211 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.211 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.212 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569016+0000", 2024-09-14T06:16:37.212 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.212 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.212 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.212 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569016+0000", 2024-09-14T06:16:37.212 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.212 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.212 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.212 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569019+0000", 2024-09-14T06:16:37.212 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2279999999999998e-06 2024-09-14T06:16:37.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.213 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.213 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.213 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569029+0000", 2024-09-14T06:16:37.213 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8079999999999999e-06 2024-09-14T06:16:37.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.213 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.213 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.213 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569059+0000", 2024-09-14T06:16:37.213 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0198000000000001e-05 2024-09-14T06:16:37.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.213 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.213 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.213 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569074+0000", 2024-09-14T06:16:37.213 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4968e-05 2024-09-14T06:16:37.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.214 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.214 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.214 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569165+0000", 2024-09-14T06:16:37.214 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0867999999999998e-05 2024-09-14T06:16:37.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.214 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.214 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.214 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569282+0000", 2024-09-14T06:16:37.214 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000117602 2024-09-14T06:16:37.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.214 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.214 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:37.214 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569405+0000", 2024-09-14T06:16:37.214 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122471 2024-09-14T06:16:37.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.215 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:37.215 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.570585+0000", 2024-09-14T06:16:37.215 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001180459 2024-09-14T06:16:37.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.215 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:37.215 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.572315+0000", 2024-09-14T06:16:37.215 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0017295329999999999 2024-09-14T06:16:37.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.215 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.215 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.572345+0000", 2024-09-14T06:16:37.215 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0784999999999997e-05 2024-09-14T06:16:37.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.216 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.216 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.216 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.572372+0000", 2024-09-14T06:16:37.216 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6276999999999999e-05 2024-09-14T06:16:37.216 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.216 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.216 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.216 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.216 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3916 3.16 3:6ab8b1d8:::rbd_data.1152bd92e04f.0000000000000002:head [stat,truncate 1590169] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:37.216 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:37.216 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.6088688540000007, 2024-09-14T06:16:37.216 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022958078999999999, 2024-09-14T06:16:37.217 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.217 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.217 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.217 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.217 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.217 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.217 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3916 2024-09-14T06:16:37.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.217 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.217 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.217 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.217 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:37.217 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.218 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.218 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.218 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:37.218 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.218 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.218 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.218 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579203+0000", 2024-09-14T06:16:37.218 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.748e-06 2024-09-14T06:16:37.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.218 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.218 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.218 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579208+0000", 2024-09-14T06:16:37.218 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:16:37.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.219 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.219 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.219 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579223+0000", 2024-09-14T06:16:37.219 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.508e-05 2024-09-14T06:16:37.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.219 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.219 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.219 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579232+0000", 2024-09-14T06:16:37.219 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9719999999999998e-06 2024-09-14T06:16:37.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.219 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.219 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.220 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579307+0000", 2024-09-14T06:16:37.220 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5028000000000001e-05 2024-09-14T06:16:37.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.220 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.220 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.220 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579412+0000", 2024-09-14T06:16:37.220 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010525400000000001 2024-09-14T06:16:37.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.220 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.220 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:16:37.220 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579535+0000", 2024-09-14T06:16:37.220 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122726 2024-09-14T06:16:37.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.220 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.221 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:37.221 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.581415+0000", 2024-09-14T06:16:37.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018800360000000001 2024-09-14T06:16:37.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.221 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.221 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:37.221 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602017+0000", 2024-09-14T06:16:37.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.020602425000000001 2024-09-14T06:16:37.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.221 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.221 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.221 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602096+0000", 2024-09-14T06:16:37.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8968999999999996e-05 2024-09-14T06:16:37.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.222 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.222 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602159+0000", 2024-09-14T06:16:37.222 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2824999999999996e-05 2024-09-14T06:16:37.222 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.222 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.222 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.222 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4704 3.5 3:a428c282:::rbd_data.1152bd92e04f.0000000000000047:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:37.222 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.943845+0000", 2024-09-14T06:16:37.222 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.2442249489999995, 2024-09-14T06:16:37.222 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003139476, 2024-09-14T06:16:37.222 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.223 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.223 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.223 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.223 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.223 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.223 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4704 2024-09-14T06:16:37.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.223 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.223 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.223 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.943845+0000", 2024-09-14T06:16:37.223 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.224 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.224 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.943845+0000", 2024-09-14T06:16:37.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.224 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.224 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.943846+0000", 2024-09-14T06:16:37.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2049999999999999e-06 2024-09-14T06:16:37.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.224 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.224 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.943851+0000", 2024-09-14T06:16:37.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2909999999999998e-06 2024-09-14T06:16:37.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.225 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.225 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.225 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944219+0000", 2024-09-14T06:16:37.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036789100000000002 2024-09-14T06:16:37.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.225 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.225 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.225 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944232+0000", 2024-09-14T06:16:37.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2564e-05 2024-09-14T06:16:37.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.225 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.225 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.225 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944331+0000", 2024-09-14T06:16:37.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8628999999999994e-05 2024-09-14T06:16:37.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.226 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.226 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.226 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944481+0000", 2024-09-14T06:16:37.226 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015043700000000001 2024-09-14T06:16:37.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.226 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.226 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:37.226 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944672+0000", 2024-09-14T06:16:37.226 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000191237 2024-09-14T06:16:37.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.226 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.226 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:37.226 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.945897+0000", 2024-09-14T06:16:37.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0012252020000000001 2024-09-14T06:16:37.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:37.227 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.946788+0000", 2024-09-14T06:16:37.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000890134 2024-09-14T06:16:37.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.227 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.946894+0000", 2024-09-14T06:16:37.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000106062 2024-09-14T06:16:37.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.228 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.946984+0000", 2024-09-14T06:16:37.228 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0823999999999996e-05 2024-09-14T06:16:37.228 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.228 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.228 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.228 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4873 3.e 3:76656e66:::rbd_data.1152bd92e04f.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=2b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:37.228 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:37.228 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.9944223149999996, 2024-09-14T06:16:37.228 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048796120000000002, 2024-09-14T06:16:37.228 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.228 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.228 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.228 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.229 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.229 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.229 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4873 2024-09-14T06:16:37.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.229 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.229 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.229 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.229 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:37.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.229 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.229 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.229 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:37.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.230 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.230 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193651+0000", 2024-09-14T06:16:37.230 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:16:37.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.230 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.230 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193663+0000", 2024-09-14T06:16:37.230 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2047e-05 2024-09-14T06:16:37.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.230 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.230 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193692+0000", 2024-09-14T06:16:37.231 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9346e-05 2024-09-14T06:16:37.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.231 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193708+0000", 2024-09-14T06:16:37.231 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6123000000000001e-05 2024-09-14T06:16:37.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.231 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193798+0000", 2024-09-14T06:16:37.231 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9909999999999998e-05 2024-09-14T06:16:37.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.231 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193863+0000", 2024-09-14T06:16:37.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4583000000000002e-05 2024-09-14T06:16:37.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.232 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:37.232 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.194717+0000", 2024-09-14T06:16:37.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00085473299999999999 2024-09-14T06:16:37.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.232 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:37.232 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198400+0000", 2024-09-14T06:16:37.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036830890000000001 2024-09-14T06:16:37.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.232 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:37.233 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198461+0000", 2024-09-14T06:16:37.233 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0844000000000002e-05 2024-09-14T06:16:37.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.233 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.233 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.233 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198496+0000", 2024-09-14T06:16:37.233 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4820000000000002e-05 2024-09-14T06:16:37.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.233 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.233 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.233 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198527+0000", 2024-09-14T06:16:37.233 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1229999999999997e-05 2024-09-14T06:16:37.233 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.233 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.234 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.234 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.234 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:6925 3.5 3:a15e9af1:::rbd_data.1152f87cbac6.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:16:37.234 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:37.234 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.0732907770000004, 2024-09-14T06:16:37.234 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0049096490000000003, 2024-09-14T06:16:37.234 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.234 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.234 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.234 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.234 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.234 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.234 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 6925 2024-09-14T06:16:37.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.235 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.235 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.235 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:37.235 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.235 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.235 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:37.235 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.235 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.236 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114780+0000", 2024-09-14T06:16:37.236 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.09e-07 2024-09-14T06:16:37.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.236 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.236 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.236 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114784+0000", 2024-09-14T06:16:37.236 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4669999999999998e-06 2024-09-14T06:16:37.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.236 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.236 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.236 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114799+0000", 2024-09-14T06:16:37.236 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4491000000000001e-05 2024-09-14T06:16:37.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.236 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.237 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.237 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114810+0000", 2024-09-14T06:16:37.237 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1243000000000001e-05 2024-09-14T06:16:37.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.237 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.237 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115515+0000", 2024-09-14T06:16:37.237 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00070521899999999998 2024-09-14T06:16:37.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.237 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.237 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115610+0000", 2024-09-14T06:16:37.237 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4260999999999998e-05 2024-09-14T06:16:37.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:37.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115742+0000", 2024-09-14T06:16:37.238 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000132414 2024-09-14T06:16:37.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.238 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:37.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.116650+0000", 2024-09-14T06:16:37.238 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00090814900000000002 2024-09-14T06:16:37.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.238 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:37.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119568+0000", 2024-09-14T06:16:37.238 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029178440000000002 2024-09-14T06:16:37.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.239 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.239 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119636+0000", 2024-09-14T06:16:37.239 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8365e-05 2024-09-14T06:16:37.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.239 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.239 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119689+0000", 2024-09-14T06:16:37.239 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2386999999999998e-05 2024-09-14T06:16:37.239 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.239 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.239 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.240 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:7151 3.e 3:7793b17d:::rbd_data.1152f87cbac6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:16:37.240 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:37.240 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.8584729869999999, 2024-09-14T06:16:37.240 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.029556783, 2024-09-14T06:16:37.240 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.240 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.240 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.240 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.240 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.240 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.240 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 7151 2024-09-14T06:16:37.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.240 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.240 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.241 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.241 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:37.241 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.241 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.241 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.241 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:37.241 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.241 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.241 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.241 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329599+0000", 2024-09-14T06:16:37.241 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5270000000000001e-06 2024-09-14T06:16:37.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.242 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.242 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329610+0000", 2024-09-14T06:16:37.242 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0431000000000001e-05 2024-09-14T06:16:37.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.242 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.242 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329634+0000", 2024-09-14T06:16:37.242 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3617000000000001e-05 2024-09-14T06:16:37.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.242 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.242 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329644+0000", 2024-09-14T06:16:37.242 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.095e-05 2024-09-14T06:16:37.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329712+0000", 2024-09-14T06:16:37.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7067e-05 2024-09-14T06:16:37.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329827+0000", 2024-09-14T06:16:37.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011498 2024-09-14T06:16:37.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:37.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.334271+0000", 2024-09-14T06:16:37.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044445839999999997 2024-09-14T06:16:37.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.244 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.244 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:37.244 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.358967+0000", 2024-09-14T06:16:37.244 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024696356999999999 2024-09-14T06:16:37.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.244 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.244 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:37.244 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359027+0000", 2024-09-14T06:16:37.244 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0006000000000001e-05 2024-09-14T06:16:37.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.244 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.244 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.244 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359099+0000", 2024-09-14T06:16:37.245 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1052999999999999e-05 2024-09-14T06:16:37.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.245 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.245 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.245 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359154+0000", 2024-09-14T06:16:37.245 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5210999999999997e-05 2024-09-14T06:16:37.245 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.245 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.245 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.245 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.245 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:7151 3.e e32/29 ondisk, result = 0)", 2024-09-14T06:16:37.245 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:37.245 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.8385285529999997, 2024-09-14T06:16:37.246 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0096251340000000005, 2024-09-14T06:16:37.246 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.246 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.246 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.246 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.246 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.246 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.246 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:37.246 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.246 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.246 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.246 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:37.246 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.247 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.247 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.247 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349543+0000", 2024-09-14T06:16:37.247 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:16:37.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.247 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.247 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.247 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349552+0000", 2024-09-14T06:16:37.247 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1039999999999996e-06 2024-09-14T06:16:37.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.247 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.247 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.247 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349579+0000", 2024-09-14T06:16:37.248 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6449999999999999e-05 2024-09-14T06:16:37.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.248 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.248 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349593+0000", 2024-09-14T06:16:37.248 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4099e-05 2024-09-14T06:16:37.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.248 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.248 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359003+0000", 2024-09-14T06:16:37.248 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0094096119999999995 2024-09-14T06:16:37.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.248 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.249 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359014+0000", 2024-09-14T06:16:37.249 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0871e-05 2024-09-14T06:16:37.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.249 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.249 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.249 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359167+0000", 2024-09-14T06:16:37.249 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015300700000000001 2024-09-14T06:16:37.249 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.249 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.249 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.249 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.249 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15516 3.1 e36/29 3:860a8b0d:::rbd_data.115223d7013d.00000000000000af:head v 36'26, mlcod=36'24)", 2024-09-14T06:16:37.249 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:37.250 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.93077420300000002, 2024-09-14T06:16:37.250 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035459770000000001, 2024-09-14T06:16:37.250 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.250 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.250 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.250 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.250 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.250 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.250 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:37.250 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.250 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.250 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.251 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:37.251 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.251 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.251 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.251 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257297+0000", 2024-09-14T06:16:37.251 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4900000000000004e-07 2024-09-14T06:16:37.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.251 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.251 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.251 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257304+0000", 2024-09-14T06:16:37.251 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:16:37.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.251 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.251 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.252 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257321+0000", 2024-09-14T06:16:37.252 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6311e-05 2024-09-14T06:16:37.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.252 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.252 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.252 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257329+0000", 2024-09-14T06:16:37.252 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0399999999999993e-06 2024-09-14T06:16:37.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.252 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.252 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.252 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257414+0000", 2024-09-14T06:16:37.252 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4846999999999996e-05 2024-09-14T06:16:37.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.253 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.253 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257436+0000", 2024-09-14T06:16:37.253 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2048999999999998e-05 2024-09-14T06:16:37.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.253 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.253 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.260796+0000", 2024-09-14T06:16:37.253 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033607559999999999 2024-09-14T06:16:37.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.253 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.253 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.260842+0000", 2024-09-14T06:16:37.253 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5226999999999999e-05 2024-09-14T06:16:37.253 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.254 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.254 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.254 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.254 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15558 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'116, mlcod=36'109)", 2024-09-14T06:16:37.254 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.274168+0000", 2024-09-14T06:16:37.254 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.91390188400000005, 2024-09-14T06:16:37.254 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030495710000000001, 2024-09-14T06:16:37.254 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.254 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.254 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.254 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.254 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.254 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.255 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.274168+0000", 2024-09-14T06:16:37.255 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.255 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.255 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.274168+0000", 2024-09-14T06:16:37.255 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.255 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.255 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.274168+0000", 2024-09-14T06:16:37.255 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.65e-07 2024-09-14T06:16:37.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.256 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.256 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.274175+0000", 2024-09-14T06:16:37.256 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8940000000000004e-06 2024-09-14T06:16:37.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.256 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.256 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.256 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.274183+0000", 2024-09-14T06:16:37.256 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2949999999999996e-06 2024-09-14T06:16:37.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.256 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.256 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.256 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.274188+0000", 2024-09-14T06:16:37.256 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0869999999999997e-06 2024-09-14T06:16:37.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.257 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.257 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.257 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.274671+0000", 2024-09-14T06:16:37.257 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000483019 2024-09-14T06:16:37.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.257 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.257 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.257 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.274695+0000", 2024-09-14T06:16:37.257 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3969999999999999e-05 2024-09-14T06:16:37.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.257 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.257 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.258 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.277169+0000", 2024-09-14T06:16:37.258 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024740420000000001 2024-09-14T06:16:37.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.258 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.258 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.258 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.277218+0000", 2024-09-14T06:16:37.258 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8899000000000001e-05 2024-09-14T06:16:37.258 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.258 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.258 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.258 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.258 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15613 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'121, mlcod=36'116)", 2024-09-14T06:16:37.258 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:37.259 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.89516569800000001, 2024-09-14T06:16:37.259 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053619389999999996, 2024-09-14T06:16:37.259 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.259 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.259 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.259 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.259 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.259 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.259 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:37.259 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.259 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.259 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.259 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:37.260 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.260 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.260 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.260 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292905+0000", 2024-09-14T06:16:37.260 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6800000000000009e-07 2024-09-14T06:16:37.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.260 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.260 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.260 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292915+0000", 2024-09-14T06:16:37.260 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:16:37.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.260 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.261 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.261 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292927+0000", 2024-09-14T06:16:37.261 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-09-14T06:16:37.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.261 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.261 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292935+0000", 2024-09-14T06:16:37.261 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2459999999999997e-06 2024-09-14T06:16:37.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.261 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.261 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292992+0000", 2024-09-14T06:16:37.261 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6873999999999998e-05 2024-09-14T06:16:37.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.262 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.262 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.262 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293016+0000", 2024-09-14T06:16:37.262 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4037e-05 2024-09-14T06:16:37.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.262 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.262 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.262 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298217+0000", 2024-09-14T06:16:37.262 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052008710000000001 2024-09-14T06:16:37.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.262 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.262 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.262 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298266+0000", 2024-09-14T06:16:37.262 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9691000000000002e-05 2024-09-14T06:16:37.263 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.263 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.263 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.263 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.263 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15615 3.11 e36/29 3:8d07d636:::rbd_data.115223d7013d.0000000000000002:head v 36'122, mlcod=36'116)", 2024-09-14T06:16:37.263 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:37.263 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.89426933900000005, 2024-09-14T06:16:37.263 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053069229999999998, 2024-09-14T06:16:37.263 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.263 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.263 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.263 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.263 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.264 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.264 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:37.264 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.264 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.264 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.264 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:37.264 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.264 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.264 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.264 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:37.264 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5799999999999998e-07 2024-09-14T06:16:37.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.264 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.265 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.265 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294003+0000", 2024-09-14T06:16:37.265 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020149600000000001 2024-09-14T06:16:37.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.265 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.265 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.265 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294016+0000", 2024-09-14T06:16:37.265 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3424999999999999e-05 2024-09-14T06:16:37.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.265 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.265 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.265 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294024+0000", 2024-09-14T06:16:37.265 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5240000000000003e-06 2024-09-14T06:16:37.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.266 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.266 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294673+0000", 2024-09-14T06:16:37.266 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064906700000000003 2024-09-14T06:16:37.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.266 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.266 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294698+0000", 2024-09-14T06:16:37.266 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5301999999999999e-05 2024-09-14T06:16:37.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.266 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.266 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299064+0000", 2024-09-14T06:16:37.266 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043652170000000002 2024-09-14T06:16:37.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.267 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.267 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.267 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299108+0000", 2024-09-14T06:16:37.267 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4033999999999999e-05 2024-09-14T06:16:37.267 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.267 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.267 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.267 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.267 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15619 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'124, mlcod=36'119)", 2024-09-14T06:16:37.267 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:37.267 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.89021539100000002, 2024-09-14T06:16:37.267 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061545389999999997, 2024-09-14T06:16:37.268 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.268 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.268 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.268 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.268 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.268 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.268 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:37.268 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.268 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.268 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.268 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:37.268 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.269 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.269 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.269 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:37.269 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3399999999999998e-07 2024-09-14T06:16:37.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.269 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.269 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.269 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297861+0000", 2024-09-14T06:16:37.269 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7640000000000002e-06 2024-09-14T06:16:37.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.269 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.269 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.269 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297876+0000", 2024-09-14T06:16:37.270 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4881e-05 2024-09-14T06:16:37.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.270 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.270 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.270 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297884+0000", 2024-09-14T06:16:37.270 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.588e-06 2024-09-14T06:16:37.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.270 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.270 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.270 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298294+0000", 2024-09-14T06:16:37.270 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00041011200000000002 2024-09-14T06:16:37.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.270 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.270 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.271 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298317+0000", 2024-09-14T06:16:37.271 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3417999999999998e-05 2024-09-14T06:16:37.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.271 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.271 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303972+0000", 2024-09-14T06:16:37.271 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056548680000000004 2024-09-14T06:16:37.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.271 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.271 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304009+0000", 2024-09-14T06:16:37.271 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7073999999999998e-05 2024-09-14T06:16:37.271 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.271 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.272 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.272 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15621 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'125, mlcod=36'119)", 2024-09-14T06:16:37.272 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:37.272 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.88870437400000002, 2024-09-14T06:16:37.272 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0069496169999999999, 2024-09-14T06:16:37.272 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.272 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.272 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.272 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.272 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.273 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:37.273 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.273 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.273 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:37.273 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.273 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.273 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:37.273 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2200000000000003e-07 2024-09-14T06:16:37.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.274 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.274 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299372+0000", 2024-09-14T06:16:37.274 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3850000000000001e-06 2024-09-14T06:16:37.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.274 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.274 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299387+0000", 2024-09-14T06:16:37.274 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4905e-05 2024-09-14T06:16:37.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.274 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.274 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299393+0000", 2024-09-14T06:16:37.274 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9090000000000003e-06 2024-09-14T06:16:37.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.275 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303591+0000", 2024-09-14T06:16:37.275 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041976640000000003 2024-09-14T06:16:37.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.275 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303619+0000", 2024-09-14T06:16:37.275 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7651e-05 2024-09-14T06:16:37.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.275 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306287+0000", 2024-09-14T06:16:37.276 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026686689999999998 2024-09-14T06:16:37.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.276 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.276 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.276 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306315+0000", 2024-09-14T06:16:37.276 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7712000000000002e-05 2024-09-14T06:16:37.276 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.276 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.276 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.276 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.276 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15623 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'126, mlcod=36'119)", 2024-09-14T06:16:37.276 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:37.277 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.885397561, 2024-09-14T06:16:37.277 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003658859, 2024-09-14T06:16:37.277 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.277 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.277 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.277 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.277 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.277 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.277 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:37.277 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.277 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.277 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.277 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:37.278 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.278 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.278 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.278 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302673+0000", 2024-09-14T06:16:37.278 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4199999999999995e-07 2024-09-14T06:16:37.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.278 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.278 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.278 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302679+0000", 2024-09-14T06:16:37.278 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7599999999999999e-06 2024-09-14T06:16:37.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.278 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.279 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.279 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302689+0000", 2024-09-14T06:16:37.279 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0335e-05 2024-09-14T06:16:37.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.279 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.279 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302697+0000", 2024-09-14T06:16:37.279 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3420000000000002e-06 2024-09-14T06:16:37.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.279 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.279 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304047+0000", 2024-09-14T06:16:37.279 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0013503530000000001 2024-09-14T06:16:37.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.280 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.280 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304059+0000", 2024-09-14T06:16:37.280 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2285e-05 2024-09-14T06:16:37.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.280 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.280 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.280 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306318+0000", 2024-09-14T06:16:37.280 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022586580000000002 2024-09-14T06:16:37.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.280 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.280 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.280 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306331+0000", 2024-09-14T06:16:37.280 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3383999999999999e-05 2024-09-14T06:16:37.280 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.280 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.280 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.281 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.281 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15626 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'127, mlcod=36'120)", 2024-09-14T06:16:37.281 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:37.281 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.88503316600000004, 2024-09-14T06:16:37.281 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038671249999999999, 2024-09-14T06:16:37.281 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.281 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.281 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.281 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.281 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.281 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.281 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:37.282 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.282 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.282 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.282 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:37.282 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.282 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.282 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.282 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:37.282 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0100000000000005e-07 2024-09-14T06:16:37.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.282 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.282 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.282 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303043+0000", 2024-09-14T06:16:37.283 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.293e-06 2024-09-14T06:16:37.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.283 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.283 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.283 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303072+0000", 2024-09-14T06:16:37.283 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9135000000000001e-05 2024-09-14T06:16:37.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.283 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.283 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.283 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303081+0000", 2024-09-14T06:16:37.283 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7080000000000003e-06 2024-09-14T06:16:37.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.283 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.283 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.283 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.305965+0000", 2024-09-14T06:16:37.284 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028841769999999999 2024-09-14T06:16:37.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.284 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.284 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.284 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.305993+0000", 2024-09-14T06:16:37.284 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8121000000000001e-05 2024-09-14T06:16:37.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.284 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.284 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.284 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306861+0000", 2024-09-14T06:16:37.284 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00086771099999999996 2024-09-14T06:16:37.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.284 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.284 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.285 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306904+0000", 2024-09-14T06:16:37.285 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3378999999999999e-05 2024-09-14T06:16:37.285 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.285 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.285 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.285 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.285 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:16521 3.e e36/34 3:77c8e3e5:::rbd_data.1152fc09e7e6.0000000000000003:head v 36'83, mlcod=36'81)", 2024-09-14T06:16:37.285 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:37.285 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.26205107399999999, 2024-09-14T06:16:37.285 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044948899999999996, 2024-09-14T06:16:37.285 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.285 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926020+0000", 2024-09-14T06:16:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.545e-06 2024-09-14T06:16:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927192+0000", 2024-09-14T06:16:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011711919999999999 2024-09-14T06:16:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927215+0000", 2024-09-14T06:16:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3677e-05 2024-09-14T06:16:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927226+0000", 2024-09-14T06:16:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0665e-05 2024-09-14T06:16:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927314+0000", 2024-09-14T06:16:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8314999999999997e-05 2024-09-14T06:16:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927344+0000", 2024-09-14T06:16:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9941e-05 2024-09-14T06:16:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.930411+0000", 2024-09-14T06:16:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030670649999999999 2024-09-14T06:16:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.930514+0000", 2024-09-14T06:16:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010249 2024-09-14T06:16:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.289 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:37.290 INFO:teuthology.orchestra.run.smithi039.stdout:866 trunc from 0x6b78ab5 to 0x859f5b4 2024-09-14T06:16:37.290 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:37.290 INFO:teuthology.orchestra.run.smithi039.stdout:871 read 0x74dc629 thru 0x74e5ed1 (0x98a9 bytes) 2024-09-14T06:16:37.290 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:37.290 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:37.290 INFO:teuthology.orchestra.run.smithi039.stdout:874 read 0x43db109 thru 0x43e53b7 (0xa2af bytes) 2024-09-14T06:16:37.291 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:16:37.292 INFO:teuthology.orchestra.run.smithi039.stdout:876 read 0x807b8ef thru 0x8085ab8 (0xa1ca bytes) 2024-09-14T06:16:37.292 INFO:teuthology.orchestra.run.smithi039.stdout:878 write 0x52f6f4 thru 0x538ee3 (0x97f0 bytes) 2024-09-14T06:16:37.292 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:37.293 INFO:teuthology.orchestra.run.smithi039.stdout:880 read 0x15d9200 thru 0x15e85dc (0xf3dd bytes) 2024-09-14T06:16:37.293 INFO:teuthology.orchestra.run.smithi039.stdout:881 read 0x31bc3a thru 0x32360f (0x79d6 bytes) 2024-09-14T06:16:37.293 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:37.293 INFO:teuthology.orchestra.run.smithi039.stdout:884 write 0xab8330b thru 0xab8bd37 (0x8a2d bytes) 2024-09-14T06:16:37.293 INFO:teuthology.orchestra.run.smithi039.stdout:885 write 0x58da18d thru 0x58e83e8 (0xe25c bytes) 2024-09-14T06:16:37.293 INFO:teuthology.orchestra.run.smithi039.stdout:886 trunc from 0xab8bd38 to 0xccc48b 2024-09-14T06:16:37.293 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:37.299 INFO:teuthology.orchestra.run.smithi039.stdout:888 write 0x59f7d6e thru 0x59fb6be (0x3951 bytes) 2024-09-14T06:16:37.300 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:37.296+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:37.303 INFO:teuthology.orchestra.run.smithi039.stdout:890 trunc from 0x59fb6bf to 0x5a73018 2024-09-14T06:16:37.333 INFO:teuthology.orchestra.run.smithi039.stdout:891 write 0x81b2e96 thru 0x81bba51 (0x8bbc bytes) 2024-09-14T06:16:37.339 INFO:teuthology.orchestra.run.smithi039.stdout:892 write 0x3018647 thru 0x301f4c2 (0x6e7c bytes) 2024-09-14T06:16:37.343 INFO:teuthology.orchestra.run.smithi039.stdout:893 read 0x580605a thru 0x580738e (0x1335 bytes) 2024-09-14T06:16:37.344 INFO:teuthology.orchestra.run.smithi039.stdout:894 writesame 0x1eea031 thru 0x1eec6d7 data_size 0x1(0x26a7 bytes) 2024-09-14T06:16:37.353 INFO:teuthology.orchestra.run.smithi039.stdout:896 trunc from 0x81bba52 to 0x715d3ad 2024-09-14T06:16:37.366 INFO:teuthology.orchestra.run.smithi039.stdout:898 punch from 0x3cc2565 to 0x3ccf17c, (0xcc17 bytes) 2024-09-14T06:16:37.372 INFO:teuthology.orchestra.run.smithi039.stdout:899 trunc from 0x715d3ad to 0xa2c7e95 2024-09-14T06:16:37.384 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:37.384 INFO:teuthology.orchestra.run.smithi039.stdout:904 write 0x9271cb8 thru 0x927c085 (0xa3ce bytes) 2024-09-14T06:16:37.395 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:37.392+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:37.413 INFO:teuthology.orchestra.run.smithi039.stdout:905 write 0xdfe9315 thru 0xdfeadbc (0x1aa8 bytes) 2024-09-14T06:16:37.416 INFO:teuthology.orchestra.run.smithi039.stdout:906 punch from 0x69b0d68 to 0x69b70f1, (0x6389 bytes) 2024-09-14T06:16:37.416 INFO:teuthology.orchestra.run.smithi039.stdout:907 write 0x2d49855 thru 0x2d4e089 (0x4835 bytes) 2024-09-14T06:16:37.420 INFO:teuthology.orchestra.run.smithi039.stdout:908 read 0x51c6e9e thru 0x51cb793 (0x48f6 bytes) 2024-09-14T06:16:37.420 INFO:teuthology.orchestra.run.smithi039.stdout:909 write 0x874f6dc thru 0x87571fa (0x7b1f bytes) 2024-09-14T06:16:37.434 INFO:teuthology.orchestra.run.smithi039.stdout:910 trunc from 0xdfeadbd to 0xc18339c 2024-09-14T06:16:37.442 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:37.442 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:37.443 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:37.443 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:37.451 INFO:teuthology.orchestra.run.smithi039.stdout:911 read 0x6aca754 thru 0x6ad209a (0x7947 bytes) 2024-09-14T06:16:37.452 INFO:teuthology.orchestra.run.smithi039.stdout:912 write 0x9c65003 thru 0x9c6d2be (0x82bc bytes) 2024-09-14T06:16:37.457 INFO:teuthology.orchestra.run.smithi039.stdout:913 trunc from 0xc18339c to 0xc91342e 2024-09-14T06:16:37.459 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:16:37.470 INFO:teuthology.orchestra.run.smithi039.stdout:914 writesame 0x5c2b235 thru 0x5c31d7b data_size 0x1(0x6b47 bytes) 2024-09-14T06:16:37.486 INFO:teuthology.orchestra.run.smithi039.stdout:915 trunc from 0xc91342e to 0xe2cdef4 2024-09-14T06:16:37.496 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:37.492+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:37.497 INFO:teuthology.orchestra.run.smithi039.stdout:916 punch from 0xbbdcbd2 to 0xbbdee51, (0x227f bytes) 2024-09-14T06:16:37.497 INFO:teuthology.orchestra.run.smithi039.stdout:917 write 0x6d9c0bc thru 0x6daa191 (0xe0d6 bytes) 2024-09-14T06:16:37.501 INFO:teuthology.orchestra.run.smithi039.stdout:918 read 0x9a73e9d thru 0x9a76cff (0x2e63 bytes) 2024-09-14T06:16:37.501 INFO:teuthology.orchestra.run.smithi039.stdout:919 punch from 0x7f064eb to 0x7f155a7, (0xf0bc bytes) 2024-09-14T06:16:37.501 INFO:teuthology.orchestra.run.smithi039.stdout:920 read 0xb35d8e7 thru 0xb3619b5 (0x40cf bytes) 2024-09-14T06:16:37.501 INFO:teuthology.orchestra.run.smithi039.stdout:921 read 0x2bb8f00 thru 0x2bbc366 (0x3467 bytes) 2024-09-14T06:16:37.501 INFO:teuthology.orchestra.run.smithi039.stdout:922 read 0x659a500 thru 0x659ccc0 (0x27c1 bytes) 2024-09-14T06:16:37.502 INFO:teuthology.orchestra.run.smithi039.stdout:924 punch from 0x43a29b5 to 0x43a618d, (0x37d8 bytes) 2024-09-14T06:16:37.502 INFO:teuthology.orchestra.run.smithi039.stdout:truncating to largest ever: 0xed2b727 2024-09-14T06:16:37.503 INFO:teuthology.orchestra.run.smithi039.stdout:925 trunc from 0xe2cdef4 to 0xed2b727 2024-09-14T06:16:37.516 INFO:teuthology.orchestra.run.smithi039.stdout:927 punch from 0x625afb3 to 0x6269c02, (0xec4f bytes) 2024-09-14T06:16:37.516 INFO:teuthology.orchestra.run.smithi039.stdout:928 write 0xce6a933 thru 0xce6d926 (0x2ff4 bytes) 2024-09-14T06:16:37.541 INFO:teuthology.orchestra.run.smithi039.stdout:929 trunc from 0xed2b727 to 0x257a969 2024-09-14T06:16:37.570 INFO:teuthology.orchestra.run.smithi039.stdout:930 read 0x11b21d0 thru 0x11bf484 (0xd2b5 bytes) 2024-09-14T06:16:37.576 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:16:37.577 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:16:37.577 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T06:16:37.577 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T06:16:37.578 INFO:teuthology.orchestra.run.smithi039.stdout:931 trunc from 0x257a969 to 0x56ff6a7 2024-09-14T06:16:37.578 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:16:37.589 INFO:teuthology.orchestra.run.smithi039.stdout:935 punch from 0x3c8b44b to 0x3c90c62, (0x5817 bytes) 2024-09-14T06:16:37.589 INFO:teuthology.orchestra.run.smithi039.stdout:937 read 0x173676c thru 0x173b46c (0x4d01 bytes) 2024-09-14T06:16:37.611 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:37.611 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:37.611 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:37.611 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:37.611 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:37.611 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:37.608+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:37.613 INFO:teuthology.orchestra.run.smithi039.stdout:939 write 0xc8f7fa5 thru 0xc8fffca (0x8026 bytes) 2024-09-14T06:16:37.630 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:16:37.634 INFO:teuthology.orchestra.run.smithi039.stdout:941 writesame 0xe0eb7af thru 0xe0f1554 data_size 0x1(0x5da6 bytes) 2024-09-14T06:16:37.634 INFO:teuthology.orchestra.run.smithi039.stdout:942 read 0xd9a44eb thru 0xd9a82bb (0x3dd1 bytes) 2024-09-14T06:16:37.634 INFO:teuthology.orchestra.run.smithi039.stdout:943 read 0x7452a1b thru 0x745d2a8 (0xa88e bytes) 2024-09-14T06:16:37.634 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:37.634 INFO:teuthology.orchestra.run.smithi039.stdout:946 read 0x56f6be3 thru 0x56fde65 (0x7283 bytes) 2024-09-14T06:16:37.635 INFO:teuthology.orchestra.run.smithi039.stdout:947 write 0xd6967eb thru 0xd6a1e36 (0xb64c bytes) 2024-09-14T06:16:37.643 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:37.643 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:37.643 INFO:teuthology.orchestra.run.smithi039.stdout:953 writesame 0x6772286 thru 0x677880c data_size 0x2f(0x6587 bytes) 2024-09-14T06:16:37.646 INFO:teuthology.orchestra.run.smithi039.stdout:954 punch from 0x9313f30 to 0x9316e18, (0x2ee8 bytes) 2024-09-14T06:16:37.646 INFO:teuthology.orchestra.run.smithi039.stdout:956 write 0x4ce8d16 thru 0x4cf137d (0x8668 bytes) 2024-09-14T06:16:37.650 INFO:teuthology.orchestra.run.smithi039.stdout:957 writesame 0x70d1347 thru 0x70dea27 data_size 0x1(0xd6e1 bytes) 2024-09-14T06:16:37.688 INFO:teuthology.orchestra.run.smithi039.stdout:958 read 0x233ab68 thru 0x233dfe9 (0x3482 bytes) 2024-09-14T06:16:37.689 INFO:teuthology.orchestra.run.smithi039.stdout:961 write 0x1ef1d0 thru 0x1f298d (0x37be bytes) 2024-09-14T06:16:37.698 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:37.700+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:37.701 INFO:teuthology.orchestra.run.smithi039.stdout:962 trunc from 0xe0f1555 to 0x8079abd 2024-09-14T06:16:37.716 INFO:teuthology.orchestra.run.smithi039.stdout:963 read 0x63922a9 thru 0x639a09d (0x7df5 bytes) 2024-09-14T06:16:37.717 INFO:teuthology.orchestra.run.smithi039.stdout:965 trunc from 0x8079abd to 0x7eb44d0 2024-09-14T06:16:37.746 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:16:37.753 INFO:teuthology.orchestra.run.smithi039.stdout:966 write 0xe37b1b9 thru 0xe37d60c (0x2454 bytes) 2024-09-14T06:16:37.771 INFO:teuthology.orchestra.run.smithi039.stdout:967 writesame 0xe90c0a1 thru 0xe91969c data_size 0x1e(0xd5fc bytes) 2024-09-14T06:16:37.771 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:37.771 INFO:teuthology.orchestra.run.smithi039.stdout:969 punch from 0x6b9c50d to 0x6ba221b, (0x5d0e bytes) 2024-09-14T06:16:37.772 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:37.772 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:37.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:37.772 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:37.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.772 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11217 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 925696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.772 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:37.772 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.7791606199999999, 2024-09-14T06:16:37.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049306313999999997, 2024-09-14T06:16:37.773 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.773 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.773 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.773 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.773 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.773 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.773 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11217 2024-09-14T06:16:37.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.773 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:37.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:37.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:37.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.29e-07 2024-09-14T06:16:37.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991345+0000", 2024-09-14T06:16:37.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1480000000000001e-06 2024-09-14T06:16:37.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991349+0000", 2024-09-14T06:16:37.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9840000000000003e-06 2024-09-14T06:16:37.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991359+0000", 2024-09-14T06:16:37.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4050000000000006e-06 2024-09-14T06:16:37.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.030826+0000", 2024-09-14T06:16:37.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.039467214 2024-09-14T06:16:37.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.030894+0000", 2024-09-14T06:16:37.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8400999999999998e-05 2024-09-14T06:16:37.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040650+0000", 2024-09-14T06:16:37.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0097559329999999996 2024-09-14T06:16:37.776 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.776 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.776 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.777 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11262 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 81920~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.777 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:37.777 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.778974584, 2024-09-14T06:16:37.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049588879000000002, 2024-09-14T06:16:37.777 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.777 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.777 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.777 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.777 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.777 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.777 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11262 2024-09-14T06:16:37.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.778 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:37.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:37.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:37.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9599999999999999e-07 2024-09-14T06:16:37.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991533+0000", 2024-09-14T06:16:37.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.362e-06 2024-09-14T06:16:37.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991539+0000", 2024-09-14T06:16:37.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:16:37.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991544+0000", 2024-09-14T06:16:37.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:16:37.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040755+0000", 2024-09-14T06:16:37.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049210938000000003 2024-09-14T06:16:37.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040833+0000", 2024-09-14T06:16:37.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7812999999999995e-05 2024-09-14T06:16:37.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041119+0000", 2024-09-14T06:16:37.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0002855 2024-09-14T06:16:37.781 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.781 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.781 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.781 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11263 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.781 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:37.781 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.778951806, 2024-09-14T06:16:37.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049944574999999998, 2024-09-14T06:16:37.782 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.782 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.782 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.782 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.782 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.782 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.782 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11263 2024-09-14T06:16:37.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.782 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:37.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:37.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:37.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.51e-07 2024-09-14T06:16:37.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991560+0000", 2024-09-14T06:16:37.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6719999999999998e-06 2024-09-14T06:16:37.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991567+0000", 2024-09-14T06:16:37.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1470000000000004e-06 2024-09-14T06:16:37.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991572+0000", 2024-09-14T06:16:37.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.237e-06 2024-09-14T06:16:37.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041149+0000", 2024-09-14T06:16:37.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049576988000000002 2024-09-14T06:16:37.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041277+0000", 2024-09-14T06:16:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000127687 2024-09-14T06:16:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041497+0000", 2024-09-14T06:16:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022039299999999999 2024-09-14T06:16:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11264 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 327680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.7789254649999999, 2024-09-14T06:16:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050248567000000001, 2024-09-14T06:16:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11264 2024-09-14T06:16:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:37.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.787 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:37.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.787 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:37.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.41e-07 2024-09-14T06:16:37.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991581+0000", 2024-09-14T06:16:37.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3740000000000001e-06 2024-09-14T06:16:37.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991585+0000", 2024-09-14T06:16:37.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9400000000000004e-06 2024-09-14T06:16:37.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991587+0000", 2024-09-14T06:16:37.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.694e-06 2024-09-14T06:16:37.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041527+0000", 2024-09-14T06:16:37.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049939349000000001 2024-09-14T06:16:37.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041598+0000", 2024-09-14T06:16:37.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1633999999999999e-05 2024-09-14T06:16:37.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041827+0000", 2024-09-14T06:16:37.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000229235 2024-09-14T06:16:37.789 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.790 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.790 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.790 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.790 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:37.790 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.778912125, 2024-09-14T06:16:37.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:16:37.790 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.790 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.790 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.790 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.790 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.790 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.791 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:16:37.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.791 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.791 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.791 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:37.791 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.791 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.791 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:37.791 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:37.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:16:37.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:16:37.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:16:37.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:16:37.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:16:37.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:16:37.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:16:37.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:16:37.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:16:37.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:16:37.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:16:37.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:16:37.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:16:37.794 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.794 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.794 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.794 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.794 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:37.794 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.778899053, 2024-09-14T06:16:37.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:16:37.794 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.794 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.795 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.795 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.795 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.795 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.795 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:16:37.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.795 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:37.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:37.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.796 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:16:37.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:16:37.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.796 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:16:37.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:16:37.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.796 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:16:37.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:16:37.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:16:37.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:16:37.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:16:37.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:16:37.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:16:37.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:16:37.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:16:37.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:16:37.798 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.798 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.798 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.798 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.798 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:37.798 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.778881621, 2024-09-14T06:16:37.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:16:37.798 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.798 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.798 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.799 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.799 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.799 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.799 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:16:37.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.799 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:37.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:37.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.800 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:37.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:16:37.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.800 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:16:37.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:16:37.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:16:37.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:16:37.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:16:37.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:16:37.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:16:37.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:16:37.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:16:37.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:16:37.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:16:37.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:16:37.802 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.802 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.802 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.802 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.803 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:37.803 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.7788605159999999, 2024-09-14T06:16:37.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:16:37.803 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.803 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.803 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.803 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.803 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.803 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.803 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:16:37.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.803 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:37.804 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.804 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.804 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:37.804 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.804 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.804 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:37.804 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:16:37.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.804 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.804 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:16:37.805 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:16:37.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.805 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.805 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:16:37.805 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:16:37.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.805 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.805 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:16:37.805 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:16:37.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.805 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.805 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:16:37.805 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:16:37.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.806 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.806 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.806 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:16:37.806 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:16:37.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.806 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.806 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.806 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:16:37.806 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:16:37.806 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.806 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.806 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.7788394080000001, 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.807 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:37.808 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.808 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.808 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.808 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:37.808 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.808 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.808 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.808 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:37.808 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:16:37.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.808 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.808 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.808 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:16:37.809 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:16:37.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.809 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.809 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.809 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:16:37.809 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:16:37.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.809 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.809 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.809 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:16:37.809 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:16:37.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.809 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.810 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.810 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:16:37.810 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:16:37.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.810 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.810 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.810 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:16:37.810 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:16:37.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.810 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.810 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.810 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:16:37.810 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:16:37.810 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.811 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.811 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.811 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13590 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 491520~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.811 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:37.811 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.6328645260000001, 2024-09-14T06:16:37.811 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042553466999999998, 2024-09-14T06:16:37.811 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.811 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.811 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.811 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.811 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.811 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.811 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13590 2024-09-14T06:16:37.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.812 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.812 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.812 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.812 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:37.812 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.812 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.812 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.812 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:37.812 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.812 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.813 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.813 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:37.813 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.34e-07 2024-09-14T06:16:37.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.813 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.813 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.813 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137642+0000", 2024-09-14T06:16:37.813 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0150000000000002e-06 2024-09-14T06:16:37.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.813 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.813 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.813 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137649+0000", 2024-09-14T06:16:37.813 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4010000000000002e-06 2024-09-14T06:16:37.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.814 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.814 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137653+0000", 2024-09-14T06:16:37.814 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:16:37.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.814 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.814 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.179985+0000", 2024-09-14T06:16:37.814 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042331580000000001 2024-09-14T06:16:37.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.814 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.814 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180076+0000", 2024-09-14T06:16:37.814 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0796000000000003e-05 2024-09-14T06:16:37.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.815 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.815 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180193+0000", 2024-09-14T06:16:37.815 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000117471 2024-09-14T06:16:37.815 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.815 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.815 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.815 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13592 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 606208~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.815 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:37.815 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.6328399579999999, 2024-09-14T06:16:37.815 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042709440000000001, 2024-09-14T06:16:37.815 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.815 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.816 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.816 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.816 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.816 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.816 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13592 2024-09-14T06:16:37.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.816 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.816 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.816 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.816 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:37.816 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.816 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.816 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.816 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:37.817 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.817 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.817 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.817 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.817 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137665+0000", 2024-09-14T06:16:37.817 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4299999999999998e-07 2024-09-14T06:16:37.817 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.817 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.817 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.817 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137667+0000", 2024-09-14T06:16:37.817 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5780000000000001e-06 2024-09-14T06:16:37.817 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.817 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.817 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.817 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137675+0000", 2024-09-14T06:16:37.817 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0870000000000001e-06 2024-09-14T06:16:37.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.818 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.818 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.818 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137680+0000", 2024-09-14T06:16:37.818 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4999999999999999e-06 2024-09-14T06:16:37.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.818 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.818 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.818 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180223+0000", 2024-09-14T06:16:37.818 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042542808000000001 2024-09-14T06:16:37.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.818 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.818 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.818 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180294+0000", 2024-09-14T06:16:37.818 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1497999999999999e-05 2024-09-14T06:16:37.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.819 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.819 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.819 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180374+0000", 2024-09-14T06:16:37.819 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9525999999999997e-05 2024-09-14T06:16:37.819 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.819 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.819 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.819 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.819 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13594 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 729088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.819 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:37.819 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.632816394, 2024-09-14T06:16:37.819 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042830604000000001, 2024-09-14T06:16:37.819 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.820 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.820 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.820 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.820 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.820 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.820 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13594 2024-09-14T06:16:37.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.820 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.820 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.820 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.820 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:37.820 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.820 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.821 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.821 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:37.821 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.821 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.821 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.821 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:37.821 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0699999999999998e-07 2024-09-14T06:16:37.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.821 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.821 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.821 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137690+0000", 2024-09-14T06:16:37.821 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.012e-06 2024-09-14T06:16:37.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.822 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.822 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137698+0000", 2024-09-14T06:16:37.822 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4379999999999998e-06 2024-09-14T06:16:37.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.822 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.822 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137703+0000", 2024-09-14T06:16:37.822 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1250000000000001e-06 2024-09-14T06:16:37.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.822 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.822 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180404+0000", 2024-09-14T06:16:37.822 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042701465000000001 2024-09-14T06:16:37.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.823 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.823 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180456+0000", 2024-09-14T06:16:37.823 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1227999999999998e-05 2024-09-14T06:16:37.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.823 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.823 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180519+0000", 2024-09-14T06:16:37.823 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-09-14T06:16:37.823 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.823 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.823 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.824 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13595 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 851968~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.824 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:37.824 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.632793889, 2024-09-14T06:16:37.824 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043012433000000003, 2024-09-14T06:16:37.824 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.824 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.824 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.824 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.824 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.824 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.824 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13595 2024-09-14T06:16:37.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.824 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.825 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.825 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:37.825 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.825 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.825 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:37.825 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.825 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.825 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137711+0000", 2024-09-14T06:16:37.825 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:16:37.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.826 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.826 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137713+0000", 2024-09-14T06:16:37.826 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5170000000000002e-06 2024-09-14T06:16:37.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.826 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.826 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.826 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137720+0000", 2024-09-14T06:16:37.826 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-09-14T06:16:37.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.826 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.826 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.826 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137726+0000", 2024-09-14T06:16:37.826 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9580000000000004e-06 2024-09-14T06:16:37.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.826 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.827 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.827 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180551+0000", 2024-09-14T06:16:37.827 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042825611999999999 2024-09-14T06:16:37.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.827 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.827 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180623+0000", 2024-09-14T06:16:37.827 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1673000000000003e-05 2024-09-14T06:16:37.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.827 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.827 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180723+0000", 2024-09-14T06:16:37.827 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.9928999999999998e-05 2024-09-14T06:16:37.827 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.828 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.828 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.828 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.828 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13597 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 974848~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.828 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:37.828 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.6327710039999999, 2024-09-14T06:16:37.828 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043180442999999999, 2024-09-14T06:16:37.828 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.828 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.828 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.828 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.828 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.828 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.828 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13597 2024-09-14T06:16:37.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.829 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.829 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.829 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:37.829 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.829 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.829 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:37.829 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.830 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.830 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137734+0000", 2024-09-14T06:16:37.830 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4499999999999997e-07 2024-09-14T06:16:37.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.830 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.830 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137736+0000", 2024-09-14T06:16:37.830 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0470000000000001e-06 2024-09-14T06:16:37.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.830 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.830 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137742+0000", 2024-09-14T06:16:37.830 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2269999999999998e-06 2024-09-14T06:16:37.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.831 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.831 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.831 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137747+0000", 2024-09-14T06:16:37.831 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6670000000000004e-06 2024-09-14T06:16:37.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.831 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.831 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.831 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180753+0000", 2024-09-14T06:16:37.831 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043006132000000002 2024-09-14T06:16:37.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.831 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.831 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.831 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180820+0000", 2024-09-14T06:16:37.831 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7516999999999997e-05 2024-09-14T06:16:37.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.832 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.832 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.832 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180914+0000", 2024-09-14T06:16:37.832 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3407999999999997e-05 2024-09-14T06:16:37.832 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.832 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.832 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.832 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.832 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13684 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 999424~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.832 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:37.832 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.630538461, 2024-09-14T06:16:37.832 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042643702999999998, 2024-09-14T06:16:37.833 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.833 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.833 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.833 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.833 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.833 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.833 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13684 2024-09-14T06:16:37.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.833 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.833 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.833 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:37.833 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.834 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.834 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:37.834 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.834 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.834 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139967+0000", 2024-09-14T06:16:37.834 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1200000000000003e-07 2024-09-14T06:16:37.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.834 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.834 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141080+0000", 2024-09-14T06:16:37.834 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011133029999999999 2024-09-14T06:16:37.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.835 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.835 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141084+0000", 2024-09-14T06:16:37.835 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.544e-06 2024-09-14T06:16:37.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.835 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.835 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141088+0000", 2024-09-14T06:16:37.835 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0280000000000002e-06 2024-09-14T06:16:37.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.835 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.835 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182457+0000", 2024-09-14T06:16:37.836 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041368690999999999 2024-09-14T06:16:37.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.836 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.836 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.836 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182528+0000", 2024-09-14T06:16:37.836 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0881000000000001e-05 2024-09-14T06:16:37.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.836 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.836 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.836 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182610+0000", 2024-09-14T06:16:37.836 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1644000000000006e-05 2024-09-14T06:16:37.836 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.836 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.836 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.837 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.837 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13721 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 532480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.837 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:37.837 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.629237727, 2024-09-14T06:16:37.837 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042577526999999997, 2024-09-14T06:16:37.837 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.837 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.837 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.837 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.837 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.837 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.837 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13721 2024-09-14T06:16:37.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.838 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.838 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.838 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.838 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:37.838 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.838 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.838 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.838 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:37.838 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.838 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.838 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.838 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:37.839 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2999999999999999e-07 2024-09-14T06:16:37.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.839 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.839 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.839 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141268+0000", 2024-09-14T06:16:37.839 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0130000000000001e-06 2024-09-14T06:16:37.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.839 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.839 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.839 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141271+0000", 2024-09-14T06:16:37.839 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0970000000000002e-06 2024-09-14T06:16:37.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.839 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.839 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.840 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141274+0000", 2024-09-14T06:16:37.840 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5809999999999999e-06 2024-09-14T06:16:37.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.840 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.840 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.840 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183522+0000", 2024-09-14T06:16:37.840 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042248591000000002 2024-09-14T06:16:37.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.840 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.840 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.840 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183691+0000", 2024-09-14T06:16:37.840 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016885399999999999 2024-09-14T06:16:37.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.841 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.841 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.841 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183844+0000", 2024-09-14T06:16:37.841 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015316099999999999 2024-09-14T06:16:37.841 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.841 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.841 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.841 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.841 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13722 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 655360~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.841 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:37.841 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.6292274249999998, 2024-09-14T06:16:37.841 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042741610999999999, 2024-09-14T06:16:37.842 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.842 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.842 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.842 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.842 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.842 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.842 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13722 2024-09-14T06:16:37.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.842 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.842 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.842 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.842 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:37.842 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.842 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.843 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.843 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:37.843 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.843 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.843 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.843 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:37.843 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:16:37.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.843 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.843 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.843 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141278+0000", 2024-09-14T06:16:37.843 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.066e-06 2024-09-14T06:16:37.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.844 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.844 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.844 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141282+0000", 2024-09-14T06:16:37.844 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7100000000000001e-06 2024-09-14T06:16:37.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.844 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.844 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.844 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141285+0000", 2024-09-14T06:16:37.844 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.554e-06 2024-09-14T06:16:37.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.844 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.844 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.845 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183824+0000", 2024-09-14T06:16:37.845 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042539043999999998 2024-09-14T06:16:37.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.845 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.845 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.845 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183906+0000", 2024-09-14T06:16:37.845 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2298000000000004e-05 2024-09-14T06:16:37.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.845 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.845 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.845 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184019+0000", 2024-09-14T06:16:37.845 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000112724 2024-09-14T06:16:37.845 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.845 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.846 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.846 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.846 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13723 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 778240~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.846 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:37.846 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.629216456, 2024-09-14T06:16:37.846 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042995901000000003, 2024-09-14T06:16:37.846 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.846 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.846 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.846 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.846 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.846 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.847 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13723 2024-09-14T06:16:37.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.847 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.847 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.847 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.847 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:37.847 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.847 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.847 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.847 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:37.847 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.847 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.847 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.848 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:37.848 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9799999999999999e-07 2024-09-14T06:16:37.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.848 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.848 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.848 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141289+0000", 2024-09-14T06:16:37.848 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0589999999999999e-06 2024-09-14T06:16:37.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.848 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.848 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.848 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141292+0000", 2024-09-14T06:16:37.848 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.208e-06 2024-09-14T06:16:37.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.848 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.849 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.849 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141295+0000", 2024-09-14T06:16:37.849 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5160000000000001e-06 2024-09-14T06:16:37.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.849 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.849 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.849 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184069+0000", 2024-09-14T06:16:37.849 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042774176999999997 2024-09-14T06:16:37.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.849 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.849 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.849 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184169+0000", 2024-09-14T06:16:37.849 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000100073 2024-09-14T06:16:37.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.849 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.850 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.850 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184284+0000", 2024-09-14T06:16:37.850 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011457 2024-09-14T06:16:37.850 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.850 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.850 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.850 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.850 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13724 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 901120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.850 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:16:37.850 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.629206062, 2024-09-14T06:16:37.850 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043197179000000002, 2024-09-14T06:16:37.850 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.850 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.851 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.851 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.851 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.851 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.851 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13724 2024-09-14T06:16:37.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.851 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.851 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.851 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.851 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:16:37.851 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.851 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.851 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.852 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:16:37.852 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.852 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.852 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.852 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.852 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141299+0000", 2024-09-14T06:16:37.852 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.03e-07 2024-09-14T06:16:37.852 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.852 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.852 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.852 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141300+0000", 2024-09-14T06:16:37.852 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0389999999999999e-06 2024-09-14T06:16:37.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.853 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.853 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.853 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141303+0000", 2024-09-14T06:16:37.853 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0479999999999999e-06 2024-09-14T06:16:37.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.853 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.853 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.853 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141305+0000", 2024-09-14T06:16:37.853 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5739999999999998e-06 2024-09-14T06:16:37.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.853 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.853 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.854 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184317+0000", 2024-09-14T06:16:37.854 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043011701999999999 2024-09-14T06:16:37.854 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.854 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.854 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.854 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184394+0000", 2024-09-14T06:16:37.854 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7324999999999996e-05 2024-09-14T06:16:37.854 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.854 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.854 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.854 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184496+0000", 2024-09-14T06:16:37.854 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010118799999999999 2024-09-14T06:16:37.854 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.855 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.855 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.855 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.855 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13726 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 1024000~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:37.855 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:37.855 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 1.6291957889999999, 2024-09-14T06:16:37.855 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043397185999999997, 2024-09-14T06:16:37.855 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:37.855 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:37.855 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:37.855 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:37.855 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:37.855 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:37.855 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13726 2024-09-14T06:16:37.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.856 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:37.856 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.856 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:37.856 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:37.856 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.856 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.856 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:37.856 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:37.856 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:37.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.856 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.856 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:37.857 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:37.857 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:16:37.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.857 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.857 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:37.857 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141310+0000", 2024-09-14T06:16:37.857 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:16:37.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.857 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.857 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:37.857 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141313+0000", 2024-09-14T06:16:37.857 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0309999999999999e-06 2024-09-14T06:16:37.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.858 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.858 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:37.858 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141315+0000", 2024-09-14T06:16:37.858 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.52e-06 2024-09-14T06:16:37.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.858 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.858 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:37.858 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184527+0000", 2024-09-14T06:16:37.858 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043211855 2024-09-14T06:16:37.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.858 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.858 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:37.858 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184606+0000", 2024-09-14T06:16:37.859 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8287000000000004e-05 2024-09-14T06:16:37.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.859 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.859 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:37.859 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184706+0000", 2024-09-14T06:16:37.859 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010021700000000001 2024-09-14T06:16:37.859 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.859 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.859 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.859 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.859 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.859 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:37.860 INFO:teuthology.orchestra.run.smithi039.stdout:970 trunc from 0xe91969d to 0xa6c4844 2024-09-14T06:16:37.861 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:16:37.862 INFO:teuthology.orchestra.run.smithi039.stdout:971 trunc from 0xa6c4844 to 0x85cef31 2024-09-14T06:16:37.862 INFO:teuthology.orchestra.run.smithi039.stdout:973 trunc from 0x85cef31 to 0x633e64b 2024-09-14T06:16:37.862 INFO:teuthology.orchestra.run.smithi039.stdout:974 write 0x67c6eb4 thru 0x67d4875 (0xd9c2 bytes) 2024-09-14T06:16:37.862 INFO:teuthology.orchestra.run.smithi039.stdout:976 punch from 0x2c501e to 0x2c7393, (0x2375 bytes) 2024-09-14T06:16:37.862 INFO:teuthology.orchestra.run.smithi039.stdout:977 read 0x1a8bc18 thru 0x1a92f4d (0x7336 bytes) 2024-09-14T06:16:37.863 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:37.860+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:37.875 INFO:teuthology.orchestra.run.smithi039.stdout:978 write 0x83d8313 thru 0x83e0acf (0x87bd bytes) 2024-09-14T06:16:37.896 INFO:teuthology.orchestra.run.smithi039.stdout:979 write 0xa689848 thru 0xa68f346 (0x5aff bytes) 2024-09-14T06:16:37.900 INFO:teuthology.orchestra.run.smithi039.stdout:982 write 0xac27ad thru 0xac4ec6 (0x271a bytes) 2024-09-14T06:16:37.902 INFO:teuthology.orchestra.run.smithi039.stdout:983 write 0x35325d6 thru 0x3534d0f (0x273a bytes) 2024-09-14T06:16:37.903 INFO:teuthology.orchestra.run.smithi039.stdout:984 read 0x6d224f5 thru 0x6d271a0 (0x4cac bytes) 2024-09-14T06:16:37.915 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:37.915 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:37.918 INFO:teuthology.orchestra.run.smithi039.stdout:985 writesame 0xee0bf87 thru 0xee1a947 data_size 0x6d(0xe9c1 bytes) 2024-09-14T06:16:37.918 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:37.929 INFO:teuthology.orchestra.run.smithi039.stdout:987 trunc from 0xee1a948 to 0x7a7449c 2024-09-14T06:16:37.940 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:16:37.940 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:37.940 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 36, 2024-09-14T06:16:37.940 INFO:teuthology.orchestra.run.smithi039.stdout: "fsid": "038563b1-268f-461f-b6a6-4b4b41ad0236", 2024-09-14T06:16:37.940 INFO:teuthology.orchestra.run.smithi039.stdout: "created": "2024-09-14T06:15:49.493035+0000", 2024-09-14T06:16:37.940 INFO:teuthology.orchestra.run.smithi039.stdout: "modified": "2024-09-14T06:16:35.545326+0000", 2024-09-14T06:16:37.941 INFO:teuthology.orchestra.run.smithi039.stdout: "last_up_change": "2024-09-14T06:16:02.014989+0000", 2024-09-14T06:16:37.941 INFO:teuthology.orchestra.run.smithi039.stdout: "last_in_change": "2024-09-14T06:16:26.221200+0000", 2024-09-14T06:16:37.941 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T06:16:37.941 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_num": 5806080, 2024-09-14T06:16:37.941 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_set": [ 2024-09-14T06:16:37.941 INFO:teuthology.orchestra.run.smithi039.stdout: "nodeep-scrub", 2024-09-14T06:16:37.941 INFO:teuthology.orchestra.run.smithi039.stdout: "noscrub", 2024-09-14T06:16:37.941 INFO:teuthology.orchestra.run.smithi039.stdout: "pglog_hardlimit", 2024-09-14T06:16:37.941 INFO:teuthology.orchestra.run.smithi039.stdout: "purged_snapdirs", 2024-09-14T06:16:37.941 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_deletes", 2024-09-14T06:16:37.941 INFO:teuthology.orchestra.run.smithi039.stdout: "sortbitwise" 2024-09-14T06:16:37.941 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:16:37.941 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_version": 5, 2024-09-14T06:16:37.941 INFO:teuthology.orchestra.run.smithi039.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T06:16:37.942 INFO:teuthology.orchestra.run.smithi039.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T06:16:37.942 INFO:teuthology.orchestra.run.smithi039.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T06:16:37.942 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_snapshot": "", 2024-09-14T06:16:37.942 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_max": 3, 2024-09-14T06:16:37.942 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd": 8, 2024-09-14T06:16:37.942 INFO:teuthology.orchestra.run.smithi039.stdout: "require_min_compat_client": "luminous", 2024-09-14T06:16:37.942 INFO:teuthology.orchestra.run.smithi039.stdout: "min_compat_client": "jewel", 2024-09-14T06:16:37.942 INFO:teuthology.orchestra.run.smithi039.stdout: "require_osd_release": "squid", 2024-09-14T06:16:37.942 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_crimson": false, 2024-09-14T06:16:37.942 INFO:teuthology.orchestra.run.smithi039.stdout: "pools": [ 2024-09-14T06:16:37.942 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.942 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 1, 2024-09-14T06:16:37.942 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": ".mgr", 2024-09-14T06:16:37.942 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-09-14T06:16:02.429225+0000", 2024-09-14T06:16:37.942 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-14T06:16:37.943 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool", 2024-09-14T06:16:37.943 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-09-14T06:16:37.943 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-09-14T06:16:37.943 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-09-14T06:16:37.943 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-09-14T06:16:37.943 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:16:37.943 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:16:37.943 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:16:37.943 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:16:37.943 INFO:teuthology.orchestra.run.smithi039.stdout: "is_stretch_pool": false, 2024-09-14T06:16:37.943 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-09-14T06:16:37.943 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:16:37.943 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 1, 2024-09-14T06:16:37.943 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 1, 2024-09-14T06:16:37.944 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 1, 2024-09-14T06:16:37.944 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 1, 2024-09-14T06:16:37.944 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 1, 2024-09-14T06:16:37.944 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-09-14T06:16:37.944 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-09-14T06:16:37.944 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-09-14T06:16:37.944 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-09-14T06:16:37.944 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-09-14T06:16:37.944 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-09-14T06:16:37.944 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-09-14T06:16:37.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.944 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "18", 2024-09-14T06:16:37.944 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-09-14T06:16:37.945 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:16:37.945 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:16:37.945 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-09-14T06:16:37.945 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:16:37.945 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 0, 2024-09-14T06:16:37.945 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 0, 2024-09-14T06:16:37.945 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-09-14T06:16:37.945 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-09-14T06:16:37.945 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-09-14T06:16:37.945 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-09-14T06:16:37.945 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-09-14T06:16:37.945 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-09-14T06:16:37.945 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-09-14T06:16:37.946 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-09-14T06:16:37.946 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-09-14T06:16:37.946 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-09-14T06:16:37.946 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-09-14T06:16:37.946 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:16:37.946 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:16:37.946 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:16:37.946 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-09-14T06:16:37.946 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-09-14T06:16:37.946 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-09-14T06:16:37.946 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-09-14T06:16:37.946 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-09-14T06:16:37.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.946 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-09-14T06:16:37.947 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-09-14T06:16:37.947 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-09-14T06:16:37.947 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:16:37.947 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:16:37.947 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:16:37.947 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:16:37.947 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-09-14T06:16:37.947 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-09-14T06:16:37.947 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-09-14T06:16:37.947 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-09-14T06:16:37.947 INFO:teuthology.orchestra.run.smithi039.stdout: "options": { 2024-09-14T06:16:37.947 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_max": 32, 2024-09-14T06:16:37.947 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_min": 1 2024-09-14T06:16:37.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.948 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-09-14T06:16:37.948 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": {} 2024-09-14T06:16:37.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.948 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-09-14T06:16:37.948 INFO:teuthology.orchestra.run.smithi039.stdout: "score_type": "Fair distribution", 2024-09-14T06:16:37.948 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 6.9000000953674316, 2024-09-14T06:16:37.948 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 6.9000000953674316, 2024-09-14T06:16:37.948 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T06:16:37.948 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 2, 2024-09-14T06:16:37.948 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 2, 2024-09-14T06:16:37.948 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:16:37.948 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 1, 2024-09-14T06:16:37.948 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:16:37.948 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.949 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-14T06:16:37.949 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": "rbd", 2024-09-14T06:16:37.949 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-09-14T06:16:08.129454+0000", 2024-09-14T06:16:37.949 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 8193, 2024-09-14T06:16:37.949 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:16:37.949 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-09-14T06:16:37.949 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-09-14T06:16:37.949 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-09-14T06:16:37.949 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-09-14T06:16:37.949 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:16:37.949 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:16:37.950 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:16:37.950 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:16:37.950 INFO:teuthology.orchestra.run.smithi039.stdout: "is_stretch_pool": false, 2024-09-14T06:16:37.950 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-09-14T06:16:37.950 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:16:37.950 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 8, 2024-09-14T06:16:37.950 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 8, 2024-09-14T06:16:37.950 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 8, 2024-09-14T06:16:37.950 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 8, 2024-09-14T06:16:37.950 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 8, 2024-09-14T06:16:37.950 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-09-14T06:16:37.950 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-09-14T06:16:37.950 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-09-14T06:16:37.950 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-09-14T06:16:37.951 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-09-14T06:16:37.951 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-09-14T06:16:37.951 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-09-14T06:16:37.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.951 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "22", 2024-09-14T06:16:37.951 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-09-14T06:16:37.951 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:16:37.951 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:16:37.951 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-09-14T06:16:37.951 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:16:37.951 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 2, 2024-09-14T06:16:37.951 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 22, 2024-09-14T06:16:37.951 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-09-14T06:16:37.951 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-09-14T06:16:37.952 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-09-14T06:16:37.952 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-09-14T06:16:37.952 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-09-14T06:16:37.952 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-09-14T06:16:37.952 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-09-14T06:16:37.952 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-09-14T06:16:37.952 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-09-14T06:16:37.952 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-09-14T06:16:37.952 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-09-14T06:16:37.952 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:16:37.952 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:16:37.952 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:16:37.952 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-09-14T06:16:37.952 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-09-14T06:16:37.952 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-09-14T06:16:37.953 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-09-14T06:16:37.953 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-09-14T06:16:37.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.953 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-09-14T06:16:37.953 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-09-14T06:16:37.953 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-09-14T06:16:37.953 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:16:37.953 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:16:37.953 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:16:37.953 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:16:37.953 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-09-14T06:16:37.953 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-09-14T06:16:37.953 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-09-14T06:16:37.954 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-09-14T06:16:37.954 INFO:teuthology.orchestra.run.smithi039.stdout: "options": {}, 2024-09-14T06:16:37.954 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-09-14T06:16:37.954 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd": {} 2024-09-14T06:16:37.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.954 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-09-14T06:16:37.954 INFO:teuthology.orchestra.run.smithi039.stdout: "score_type": "Fair distribution", 2024-09-14T06:16:37.954 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 4.380000114440918, 2024-09-14T06:16:37.954 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 4.380000114440918, 2024-09-14T06:16:37.954 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 1, 2024-09-14T06:16:37.954 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 4.380000114440918, 2024-09-14T06:16:37.954 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 4.380000114440918, 2024-09-14T06:16:37.954 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:16:37.954 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 1, 2024-09-14T06:16:37.955 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:16:37.955 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.955 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 3, 2024-09-14T06:16:37.955 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": "pool_client.0", 2024-09-14T06:16:37.955 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-09-14T06:16:18.928556+0000", 2024-09-14T06:16:37.955 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 8193, 2024-09-14T06:16:37.955 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:16:37.955 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-09-14T06:16:37.955 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-09-14T06:16:37.955 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-09-14T06:16:37.955 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-09-14T06:16:37.955 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:16:37.956 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:16:37.956 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:16:37.956 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:16:37.956 INFO:teuthology.orchestra.run.smithi039.stdout: "is_stretch_pool": false, 2024-09-14T06:16:37.956 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-09-14T06:16:37.956 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:16:37.956 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 32, 2024-09-14T06:16:37.956 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 32, 2024-09-14T06:16:37.956 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 32, 2024-09-14T06:16:37.956 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 32, 2024-09-14T06:16:37.956 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 32, 2024-09-14T06:16:37.956 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-09-14T06:16:37.956 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-09-14T06:16:37.956 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-09-14T06:16:37.957 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-09-14T06:16:37.957 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-09-14T06:16:37.957 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-09-14T06:16:37.957 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-09-14T06:16:37.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.957 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "36", 2024-09-14T06:16:37.957 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-09-14T06:16:37.957 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:16:37.957 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:16:37.957 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-09-14T06:16:37.957 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:16:37.957 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 8, 2024-09-14T06:16:37.957 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 36, 2024-09-14T06:16:37.957 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-09-14T06:16:37.958 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-09-14T06:16:37.958 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-09-14T06:16:37.958 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-09-14T06:16:37.958 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-09-14T06:16:37.958 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-09-14T06:16:37.958 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-09-14T06:16:37.958 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-09-14T06:16:37.958 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-09-14T06:16:37.958 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-09-14T06:16:37.958 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-09-14T06:16:37.958 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:16:37.958 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:16:37.958 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:16:37.958 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-09-14T06:16:37.959 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-09-14T06:16:37.959 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-09-14T06:16:37.959 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-09-14T06:16:37.959 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-09-14T06:16:37.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.959 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-09-14T06:16:37.959 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-09-14T06:16:37.959 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-09-14T06:16:37.959 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:16:37.959 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:16:37.959 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:16:37.959 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:16:37.959 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-09-14T06:16:37.960 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-09-14T06:16:37.960 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-09-14T06:16:37.960 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-09-14T06:16:37.960 INFO:teuthology.orchestra.run.smithi039.stdout: "options": {}, 2024-09-14T06:16:37.960 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-09-14T06:16:37.960 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd": {} 2024-09-14T06:16:37.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.960 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-09-14T06:16:37.960 INFO:teuthology.orchestra.run.smithi039.stdout: "score_type": "Fair distribution", 2024-09-14T06:16:37.960 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 1.75, 2024-09-14T06:16:37.960 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 1.75, 2024-09-14T06:16:37.960 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 1.1399999856948853, 2024-09-14T06:16:37.960 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 2, 2024-09-14T06:16:37.960 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 2, 2024-09-14T06:16:37.961 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:16:37.961 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 1, 2024-09-14T06:16:37.961 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:16:37.961 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.961 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.961 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:16:37.961 INFO:teuthology.orchestra.run.smithi039.stdout: "osds": [ 2024-09-14T06:16:37.961 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.961 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 0, 2024-09-14T06:16:37.961 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "c669ea76-ed19-4c12-9e56-d79d62846f95", 2024-09-14T06:16:37.961 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:16:37.961 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:16:37.961 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:16:37.962 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:16:37.962 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:16:37.962 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:16:37.962 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:16:37.962 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 34, 2024-09-14T06:16:37.962 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:16:37.962 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:16:37.962 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:16:37.962 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.962 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.962 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6808", 2024-09-14T06:16:37.962 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:16:37.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.963 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.963 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6809", 2024-09-14T06:16:37.963 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:16:37.963 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.963 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.963 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:16:37.963 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.963 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.963 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6810", 2024-09-14T06:16:37.963 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:16:37.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.964 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.964 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6811", 2024-09-14T06:16:37.964 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:16:37.964 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.964 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.964 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:16:37.964 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.964 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.964 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6814", 2024-09-14T06:16:37.964 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:16:37.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.965 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.965 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6815", 2024-09-14T06:16:37.965 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:16:37.965 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.965 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.965 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:16:37.965 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.965 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.965 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6812", 2024-09-14T06:16:37.965 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:16:37.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.966 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.966 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6813", 2024-09-14T06:16:37.966 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:16:37.966 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.966 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.966 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6809/1157559201", 2024-09-14T06:16:37.966 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6811/1157559201", 2024-09-14T06:16:37.966 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6815/1157559201", 2024-09-14T06:16:37.966 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6813/1157559201", 2024-09-14T06:16:37.966 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:16:37.966 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:16:37.966 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:16:37.966 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.967 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 1, 2024-09-14T06:16:37.967 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b", 2024-09-14T06:16:37.967 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:16:37.967 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:16:37.967 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:16:37.967 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:16:37.967 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:16:37.967 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:16:37.967 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:16:37.967 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 23, 2024-09-14T06:16:37.967 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:16:37.967 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:16:37.968 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:16:37.968 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.968 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.968 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6816", 2024-09-14T06:16:37.968 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:16:37.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.968 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.968 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6817", 2024-09-14T06:16:37.968 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:16:37.968 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.968 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.968 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:16:37.969 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.969 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.969 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6818", 2024-09-14T06:16:37.969 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:16:37.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.969 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.969 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6819", 2024-09-14T06:16:37.969 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:16:37.969 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.969 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.969 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:16:37.970 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.970 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.970 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6822", 2024-09-14T06:16:37.970 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:16:37.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.970 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.970 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6823", 2024-09-14T06:16:37.970 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:16:37.970 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.970 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.970 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:16:37.970 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.971 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.971 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6820", 2024-09-14T06:16:37.971 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:16:37.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.971 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.971 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6821", 2024-09-14T06:16:37.971 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:16:37.971 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.971 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.971 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6817/4062089212", 2024-09-14T06:16:37.971 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6819/4062089212", 2024-09-14T06:16:37.972 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6823/4062089212", 2024-09-14T06:16:37.972 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6821/4062089212", 2024-09-14T06:16:37.972 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:16:37.972 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:16:37.972 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:16:37.972 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.972 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 2, 2024-09-14T06:16:37.972 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "9bda988f-e615-4545-849b-318c408c4c42", 2024-09-14T06:16:37.972 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:16:37.972 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:16:37.972 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:16:37.972 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 0, 2024-09-14T06:16:37.973 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:16:37.973 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:16:37.973 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:16:37.973 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 29, 2024-09-14T06:16:37.973 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:16:37.973 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:16:37.973 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:16:37.973 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.973 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.973 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6824", 2024-09-14T06:16:37.973 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:16:37.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.974 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.974 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6825", 2024-09-14T06:16:37.974 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:16:37.974 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.974 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.974 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:16:37.974 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.974 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.974 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6826", 2024-09-14T06:16:37.974 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:16:37.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.975 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.975 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6827", 2024-09-14T06:16:37.975 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:16:37.975 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.975 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.975 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:16:37.975 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.975 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.975 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6830", 2024-09-14T06:16:37.975 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:16:37.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.976 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.976 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6831", 2024-09-14T06:16:37.976 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:16:37.976 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.976 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.976 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:16:37.976 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.976 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.976 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6828", 2024-09-14T06:16:37.976 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:16:37.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.977 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.977 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.977 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6829", 2024-09-14T06:16:37.977 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:16:37.977 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.977 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.977 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6825/1724773882", 2024-09-14T06:16:37.977 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6827/1724773882", 2024-09-14T06:16:37.977 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6831/1724773882", 2024-09-14T06:16:37.977 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6829/1724773882", 2024-09-14T06:16:37.977 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:16:37.977 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:16:37.977 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:16:37.978 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.978 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 3, 2024-09-14T06:16:37.978 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "8ea6cd69-2ef1-42ea-b339-7394bf8f4a87", 2024-09-14T06:16:37.978 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:16:37.978 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:16:37.978 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:16:37.978 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:16:37.978 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:16:37.978 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:16:37.978 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:16:37.978 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 34, 2024-09-14T06:16:37.979 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:16:37.979 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:16:37.979 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:16:37.979 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.979 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.979 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6800", 2024-09-14T06:16:37.979 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:16:37.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.979 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.979 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6801", 2024-09-14T06:16:37.979 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:16:37.979 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.979 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.980 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:16:37.980 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.980 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.980 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6802", 2024-09-14T06:16:37.980 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:16:37.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.980 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.980 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6803", 2024-09-14T06:16:37.980 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:16:37.980 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.981 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.981 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:16:37.981 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.981 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.981 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6806", 2024-09-14T06:16:37.981 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:16:37.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.981 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.981 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6807", 2024-09-14T06:16:37.981 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:16:37.982 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.982 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.982 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:16:37.982 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.982 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.982 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6804", 2024-09-14T06:16:37.982 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:16:37.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.982 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.982 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6805", 2024-09-14T06:16:37.982 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:16:37.983 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.983 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.983 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6801/3775031293", 2024-09-14T06:16:37.983 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6803/3775031293", 2024-09-14T06:16:37.983 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6807/3775031293", 2024-09-14T06:16:37.983 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6805/3775031293", 2024-09-14T06:16:37.983 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:16:37.983 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:16:37.983 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:16:37.983 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.983 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 4, 2024-09-14T06:16:37.983 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "fea45e8a-e8ea-4963-b93f-0651b84c9e74", 2024-09-14T06:16:37.984 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:16:37.984 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:16:37.984 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:16:37.984 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:16:37.984 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:16:37.984 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:16:37.984 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:16:37.984 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 29, 2024-09-14T06:16:37.984 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:16:37.984 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:16:37.984 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:16:37.984 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.984 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6824", 2024-09-14T06:16:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:16:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6825", 2024-09-14T06:16:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:16:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:16:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.986 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6826", 2024-09-14T06:16:37.986 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:16:37.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.986 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.986 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6827", 2024-09-14T06:16:37.986 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:16:37.986 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.986 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.986 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:16:37.986 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.986 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.987 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6830", 2024-09-14T06:16:37.987 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:16:37.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.987 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.987 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6831", 2024-09-14T06:16:37.987 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:16:37.987 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.987 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.987 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:16:37.987 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.987 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.987 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6828", 2024-09-14T06:16:37.988 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:16:37.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.988 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.988 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6829", 2024-09-14T06:16:37.988 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:16:37.988 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.988 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.988 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6825/2632317380", 2024-09-14T06:16:37.988 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6827/2632317380", 2024-09-14T06:16:37.988 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6831/2632317380", 2024-09-14T06:16:37.988 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6829/2632317380", 2024-09-14T06:16:37.988 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:16:37.988 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:16:37.988 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:16:37.989 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.989 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 5, 2024-09-14T06:16:37.989 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "b601e852-a180-477f-86fc-6a213fa866f0", 2024-09-14T06:16:37.989 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:16:37.989 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:16:37.989 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:16:37.989 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:16:37.989 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:16:37.989 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:16:37.989 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:16:37.989 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 29, 2024-09-14T06:16:37.990 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:16:37.990 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:16:37.990 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:16:37.990 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.990 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.990 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6816", 2024-09-14T06:16:37.990 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:16:37.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.990 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.990 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6817", 2024-09-14T06:16:37.990 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:16:37.990 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.991 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.991 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:16:37.991 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.991 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.991 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6818", 2024-09-14T06:16:37.991 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:16:37.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.991 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.991 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6819", 2024-09-14T06:16:37.991 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:16:37.991 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.991 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.992 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:16:37.992 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.992 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.992 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6822", 2024-09-14T06:16:37.992 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:16:37.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.992 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.992 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6823", 2024-09-14T06:16:37.992 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:16:37.992 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.992 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.993 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:16:37.993 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.993 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.993 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6820", 2024-09-14T06:16:37.993 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:16:37.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.993 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.993 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6821", 2024-09-14T06:16:37.993 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:16:37.993 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.993 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.994 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6817/3501664149", 2024-09-14T06:16:37.994 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6819/3501664149", 2024-09-14T06:16:37.994 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6823/3501664149", 2024-09-14T06:16:37.994 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6821/3501664149", 2024-09-14T06:16:37.994 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:16:37.994 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:16:37.994 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:16:37.994 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.994 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 6, 2024-09-14T06:16:37.994 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "1113afe0-f524-41b1-8e51-eb5164602d38", 2024-09-14T06:16:37.994 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:16:37.994 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:16:37.995 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:16:37.995 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:16:37.995 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:16:37.995 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:16:37.995 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:16:37.995 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 34, 2024-09-14T06:16:37.995 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:16:37.995 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:16:37.995 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:16:37.995 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.995 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.995 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6800", 2024-09-14T06:16:37.996 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:16:37.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.996 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.996 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6801", 2024-09-14T06:16:37.996 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:16:37.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.996 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.996 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:16:37.996 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.996 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.997 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6802", 2024-09-14T06:16:37.997 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:16:37.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.997 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.997 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6803", 2024-09-14T06:16:37.997 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:16:37.997 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.997 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.997 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:16:37.997 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.997 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.997 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6806", 2024-09-14T06:16:37.998 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:16:37.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.998 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.998 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6807", 2024-09-14T06:16:37.998 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:16:37.998 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.998 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.998 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:16:37.998 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:37.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.998 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:37.999 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6804", 2024-09-14T06:16:37.999 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:16:37.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:37.999 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:37.999 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6805", 2024-09-14T06:16:37.999 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:16:37.999 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:37.999 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:37.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:37.999 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6801/1395936437", 2024-09-14T06:16:37.999 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6803/1395936437", 2024-09-14T06:16:37.999 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6807/1395936437", 2024-09-14T06:16:37.999 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6805/1395936437", 2024-09-14T06:16:37.999 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:16:38.000 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:16:38.000 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:16:38.000 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:38.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.000 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 7, 2024-09-14T06:16:38.000 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "c9162545-318b-4591-be64-cc8ace9097fe", 2024-09-14T06:16:38.000 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:16:38.000 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:16:38.000 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:16:38.000 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:16:38.000 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:16:38.000 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:16:38.001 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:16:38.001 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 23, 2024-09-14T06:16:38.001 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:16:38.001 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:16:38.001 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:16:38.001 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:38.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.001 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:38.001 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6808", 2024-09-14T06:16:38.001 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:16:38.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.001 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:38.001 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6809", 2024-09-14T06:16:38.001 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:16:38.002 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:38.002 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:38.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.002 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:16:38.002 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:38.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.002 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:38.002 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6810", 2024-09-14T06:16:38.002 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:16:38.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.002 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:38.002 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6811", 2024-09-14T06:16:38.002 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:16:38.002 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:38.003 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:38.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.003 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:16:38.003 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:38.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.003 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:38.003 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6814", 2024-09-14T06:16:38.003 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:16:38.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.003 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:38.003 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6815", 2024-09-14T06:16:38.003 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:16:38.003 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:38.003 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:38.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.004 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:16:38.004 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:38.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.004 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:38.004 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6812", 2024-09-14T06:16:38.004 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:16:38.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.004 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:38.004 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6813", 2024-09-14T06:16:38.004 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:16:38.004 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:38.004 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:38.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.005 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6809/711112216", 2024-09-14T06:16:38.005 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6811/711112216", 2024-09-14T06:16:38.005 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6815/711112216", 2024-09-14T06:16:38.005 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6813/711112216", 2024-09-14T06:16:38.008 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:16:38.008 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:16:38.008 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:16:38.008 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:38.008 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:38.008 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:16:38.008 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_xinfo": [ 2024-09-14T06:16:38.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.009 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 0, 2024-09-14T06:16:38.009 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:16:38.009 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:16:38.009 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:16:38.009 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:16:38.009 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:16:38.009 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.705575+0000", 2024-09-14T06:16:38.009 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:16:38.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.009 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 1, 2024-09-14T06:16:38.009 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:16:38.009 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:16:38.009 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:16:38.009 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:16:38.010 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:16:38.010 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.806946+0000", 2024-09-14T06:16:38.010 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:16:38.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.010 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 2, 2024-09-14T06:16:38.010 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:16:38.010 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:16:38.010 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:16:38.010 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:16:38.010 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:16:38.010 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.853831+0000", 2024-09-14T06:16:38.010 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:16:38.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.011 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 3, 2024-09-14T06:16:38.011 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:16:38.011 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:16:38.011 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:16:38.011 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:16:38.011 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:16:38.011 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.690315+0000", 2024-09-14T06:16:38.011 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:16:38.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.011 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 4, 2024-09-14T06:16:38.011 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:16:38.011 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:16:38.011 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:16:38.011 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:16:38.012 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:16:38.012 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.494526+0000", 2024-09-14T06:16:38.012 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:16:38.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.012 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 5, 2024-09-14T06:16:38.012 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:16:38.012 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:16:38.012 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:16:38.012 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:16:38.012 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:16:38.012 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.546122+0000", 2024-09-14T06:16:38.012 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:16:38.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.013 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 6, 2024-09-14T06:16:38.013 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:16:38.013 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:16:38.013 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:16:38.013 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:16:38.013 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:16:38.013 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.173691+0000", 2024-09-14T06:16:38.013 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:16:38.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:38.013 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 7, 2024-09-14T06:16:38.013 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:16:38.013 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:16:38.014 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:16:38.014 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:16:38.014 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:16:38.014 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.148509+0000", 2024-09-14T06:16:38.014 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:16:38.014 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:38.014 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:16:38.014 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap": [], 2024-09-14T06:16:38.014 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap_items": [], 2024-09-14T06:16:38.014 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap_primaries": [], 2024-09-14T06:16:38.014 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_temp": [], 2024-09-14T06:16:38.014 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_temp": [], 2024-09-14T06:16:38.014 INFO:teuthology.orchestra.run.smithi039.stdout: "blocklist": {}, 2024-09-14T06:16:38.014 INFO:teuthology.orchestra.run.smithi039.stdout: "range_blocklist": {}, 2024-09-14T06:16:38.014 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profiles": { 2024-09-14T06:16:38.015 INFO:teuthology.orchestra.run.smithi039.stdout: "default": { 2024-09-14T06:16:38.015 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-failure-domain": "osd", 2024-09-14T06:16:38.015 INFO:teuthology.orchestra.run.smithi039.stdout: "k": "2", 2024-09-14T06:16:38.015 INFO:teuthology.orchestra.run.smithi039.stdout: "m": "1", 2024-09-14T06:16:38.015 INFO:teuthology.orchestra.run.smithi039.stdout: "plugin": "jerasure", 2024-09-14T06:16:38.015 INFO:teuthology.orchestra.run.smithi039.stdout: "technique": "reed_sol_van" 2024-09-14T06:16:38.015 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:38.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:38.015 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps_queue": [], 2024-09-14T06:16:38.015 INFO:teuthology.orchestra.run.smithi039.stdout: "new_removed_snaps": [], 2024-09-14T06:16:38.015 INFO:teuthology.orchestra.run.smithi039.stdout: "new_purged_snaps": [], 2024-09-14T06:16:38.015 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_node_flags": {}, 2024-09-14T06:16:38.015 INFO:teuthology.orchestra.run.smithi039.stdout: "device_class_flags": {}, 2024-09-14T06:16:38.015 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode": { 2024-09-14T06:16:38.015 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode_enabled": false, 2024-09-14T06:16:38.015 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_bucket_count": 0, 2024-09-14T06:16:38.016 INFO:teuthology.orchestra.run.smithi039.stdout: "degraded_stretch_mode": 0, 2024-09-14T06:16:38.016 INFO:teuthology.orchestra.run.smithi039.stdout: "recovering_stretch_mode": 0, 2024-09-14T06:16:38.016 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode_bucket": 0 2024-09-14T06:16:38.016 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:38.016 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:38.017 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:38.017 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:38.017 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:16:38.017 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:38.017 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:38.018 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:38.018 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:38.018 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:38.018 INFO:teuthology.orchestra.run.smithi039.stdout:992 read 0x7203e6a thru 0x7205cf3 (0x1e8a bytes) 2024-09-14T06:16:38.018 INFO:teuthology.orchestra.run.smithi039.stdout:993 punch from 0x2f61502 to 0x2f64882, (0x3380 bytes) 2024-09-14T06:16:38.018 INFO:teuthology.orchestra.run.smithi039.stdout:996 write 0x3e81b89 thru 0x3e843a9 (0x2821 bytes) 2024-09-14T06:16:38.019 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:38.019 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:38.019 INFO:teuthology.orchestra.run.smithi039.stdout:997 write 0xcf11441 thru 0xcf1e9bb (0xd57b bytes) 2024-09-14T06:16:38.021 INFO:tasks.thrashosds.thrasher:j is {'epoch': 36, 'fsid': '038563b1-268f-461f-b6a6-4b4b41ad0236', 'created': '2024-09-14T06:15:49.493035+0000', 'modified': '2024-09-14T06:16:35.545326+0000', 'last_up_change': '2024-09-14T06:16:02.014989+0000', 'last_in_change': '2024-09-14T06:16:26.221200+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:16:02.429225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:16:08.129454+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.380000114440918, 'score_stable': 4.380000114440918, 'optimal_score': 1, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:16:18.928556+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 8, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'c669ea76-ed19-4c12-9e56-d79d62846f95', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6808', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6809', 'nonce': 1157559201}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6810', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6811', 'nonce': 1157559201}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6814', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6815', 'nonce': 1157559201}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6812', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6813', 'nonce': 1157559201}]}, 'public_addr': '172.21.15.39:6809/1157559201', 'cluster_addr': '172.21.15.39:6811/1157559201', 'heartbeat_back_addr': '172.21.15.39:6815/1157559201', 'heartbeat_front_addr': '172.21.15.39:6813/1157559201', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6816', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6817', 'nonce': 4062089212}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6818', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6819', 'nonce': 4062089212}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6822', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6823', 'nonce': 4062089212}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6820', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6821', 'nonce': 4062089212}]}, 'public_addr': '172.21.15.39:6817/4062089212', 'cluster_addr': '172.21.15.39:6819/4062089212', 'heartbeat_back_addr': '172.21.15.39:6823/4062089212', 'heartbeat_front_addr': '172.21.15.39:6821/4062089212', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9bda988f-e615-4545-849b-318c408c4c42', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 29, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6824', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6825', 'nonce': 1724773882}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6826', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6827', 'nonce': 1724773882}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6830', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6831', 'nonce': 1724773882}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6828', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6829', 'nonce': 1724773882}]}, 'public_addr': '172.21.15.39:6825/1724773882', 'cluster_addr': '172.21.15.39:6827/1724773882', 'heartbeat_back_addr': '172.21.15.39:6831/1724773882', 'heartbeat_front_addr': '172.21.15.39:6829/1724773882', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8ea6cd69-2ef1-42ea-b339-7394bf8f4a87', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6800', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6801', 'nonce': 3775031293}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6802', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6803', 'nonce': 3775031293}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6806', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6807', 'nonce': 3775031293}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6804', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6805', 'nonce': 3775031293}]}, 'public_addr': '172.21.15.39:6801/3775031293', 'cluster_addr': '172.21.15.39:6803/3775031293', 'heartbeat_back_addr': '172.21.15.39:6807/3775031293', 'heartbeat_front_addr': '172.21.15.39:6805/3775031293', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fea45e8a-e8ea-4963-b93f-0651b84c9e74', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 29, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6824', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6825', 'nonce': 2632317380}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6826', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6827', 'nonce': 2632317380}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6830', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6831', 'nonce': 2632317380}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6828', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6829', 'nonce': 2632317380}]}, 'public_addr': '172.21.15.87:6825/2632317380', 'cluster_addr': '172.21.15.87:6827/2632317380', 'heartbeat_back_addr': '172.21.15.87:6831/2632317380', 'heartbeat_front_addr': '172.21.15.87:6829/2632317380', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b601e852-a180-477f-86fc-6a213fa866f0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 29, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6816', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6817', 'nonce': 3501664149}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6818', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6819', 'nonce': 3501664149}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6822', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6823', 'nonce': 3501664149}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6820', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6821', 'nonce': 3501664149}]}, 'public_addr': '172.21.15.87:6817/3501664149', 'cluster_addr': '172.21.15.87:6819/3501664149', 'heartbeat_back_addr': '172.21.15.87:6823/3501664149', 'heartbeat_front_addr': '172.21.15.87:6821/3501664149', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '1113afe0-f524-41b1-8e51-eb5164602d38', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6800', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6801', 'nonce': 1395936437}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6802', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6803', 'nonce': 1395936437}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6806', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6807', 'nonce': 1395936437}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6804', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6805', 'nonce': 1395936437}]}, 'public_addr': '172.21.15.87:6801/1395936437', 'cluster_addr': '172.21.15.87:6803/1395936437', 'heartbeat_back_addr': '172.21.15.87:6807/1395936437', 'heartbeat_front_addr': '172.21.15.87:6805/1395936437', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c9162545-318b-4591-be64-cc8ace9097fe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6808', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6809', 'nonce': 711112216}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6810', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6811', 'nonce': 711112216}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6814', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6815', 'nonce': 711112216}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6812', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6813', 'nonce': 711112216}]}, 'public_addr': '172.21.15.87:6809/711112216', 'cluster_addr': '172.21.15.87:6811/711112216', 'heartbeat_back_addr': '172.21.15.87:6815/711112216', 'heartbeat_front_addr': '172.21.15.87:6813/711112216', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.705575+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.806946+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.853831+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.690315+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.494526+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.546122+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.173691+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.148509+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T06:16:38.021 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:16:38.022 INFO:teuthology.orchestra.run.smithi039.stdout:998 writesame 0x61cf7bd thru 0x61d64ab data_size 0x1(0x6cef bytes) 2024-09-14T06:16:38.022 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:38.022 INFO:teuthology.orchestra.run.smithi039.stdout:1000 write 0x86cd4a6 thru 0x86d81f5 (0xad50 bytes) 2024-09-14T06:16:38.022 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:38.022 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:38.023 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:38.020+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:38.023 INFO:teuthology.orchestra.run.smithi039.stdout:1002 punch from 0x8e8a9ea to 0x8e8e794, (0x3daa bytes) 2024-09-14T06:16:38.024 INFO:teuthology.orchestra.run.smithi039.stdout:1003 write 0xeda907 thru 0xedb945 (0x103f bytes) 2024-09-14T06:16:38.024 INFO:teuthology.orchestra.run.smithi039.stdout:1004 trunc from 0xcf1e9bc to 0xa1d7bfa 2024-09-14T06:16:38.024 INFO:teuthology.orchestra.run.smithi039.stdout:1005 read 0x210edec thru 0x211dd31 (0xef46 bytes) 2024-09-14T06:16:38.024 INFO:teuthology.orchestra.run.smithi039.stdout:1006 read 0xcc78f5 thru 0xccc083 (0x478f bytes) 2024-09-14T06:16:38.024 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:38.024 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:38.025 INFO:teuthology.orchestra.run.smithi039.stdout:1008 trunc from 0xa1d7bfa to 0x3bc7c8e 2024-09-14T06:16:38.025 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:16:38.036 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:38.037 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:38.039 INFO:teuthology.orchestra.run.smithi039.stdout:1009 read 0x3368495 thru 0x3373074 (0xabe0 bytes) 2024-09-14T06:16:38.040 INFO:teuthology.orchestra.run.smithi039.stdout:1010 read 0x4366d1 thru 0x4423ca (0xbcfa bytes) 2024-09-14T06:16:38.055 INFO:teuthology.orchestra.run.smithi039.stdout:1013 write 0x7e5dfc0 thru 0x7e6365e (0x569f bytes) 2024-09-14T06:16:38.062 INFO:teuthology.orchestra.run.smithi039.stdout:1014 read 0x66e23eb thru 0x66e7474 (0x508a bytes) 2024-09-14T06:16:38.062 INFO:teuthology.orchestra.run.smithi039.stdout:1015 read 0x55ef903 thru 0x55f2b1a (0x3218 bytes) 2024-09-14T06:16:38.062 INFO:teuthology.orchestra.run.smithi039.stdout:1016 write 0x32a4d53 thru 0x32a8424 (0x36d2 bytes) 2024-09-14T06:16:38.064 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:38.064 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:38.064 INFO:teuthology.orchestra.run.smithi039.stdout:1017 read 0x3356 thru 0x1024d (0xcef8 bytes) 2024-09-14T06:16:38.065 INFO:teuthology.orchestra.run.smithi039.stdout:1018 read 0x47657f8 thru 0x476ac23 (0x542c bytes) 2024-09-14T06:16:38.066 INFO:teuthology.orchestra.run.smithi039.stdout:1019 write 0x3617019 thru 0x362282a (0xb812 bytes) 2024-09-14T06:16:38.078 INFO:teuthology.orchestra.run.smithi039.stdout:1020 trunc from 0x7e6365f to 0xea4e413 2024-09-14T06:16:38.085 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:38.085 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:38.086 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:38.086 INFO:teuthology.orchestra.run.smithi039.stdout:1022 punch from 0x3b846ad to 0x3b93286, (0xebd9 bytes) 2024-09-14T06:16:38.086 INFO:teuthology.orchestra.run.smithi039.stdout:1026 read 0x751e95 thru 0x753b41 (0x1cad bytes) 2024-09-14T06:16:38.087 INFO:teuthology.orchestra.run.smithi039.stdout:1027 write 0xb4294e1 thru 0xb434568 (0xb088 bytes) 2024-09-14T06:16:38.089 INFO:teuthology.orchestra.run.smithi039.stdout:1028 read 0x3f78dc2 thru 0x3f799c7 (0xc06 bytes) 2024-09-14T06:16:38.089 INFO:teuthology.orchestra.run.smithi039.stdout:1030 write 0x517eca5 thru 0x5180793 (0x1aef bytes) 2024-09-14T06:16:38.092 INFO:teuthology.orchestra.run.smithi039.stdout:1031 read 0xe0276d8 thru 0xe02873e (0x1067 bytes) 2024-09-14T06:16:38.092 INFO:teuthology.orchestra.run.smithi039.stdout:1032 writesame 0x529d545 thru 0x52a331a data_size 0x1(0x5dd6 bytes) 2024-09-14T06:16:38.100 INFO:teuthology.orchestra.run.smithi039.stdout:1033 punch from 0xd3bb2a1 to 0xd3c9f8c, (0xeceb bytes) 2024-09-14T06:16:38.100 INFO:teuthology.orchestra.run.smithi039.stdout:1034 writesame 0x8dc0069 thru 0x8dc4552 data_size 0x1(0x44ea bytes) 2024-09-14T06:16:38.113 INFO:teuthology.orchestra.run.smithi039.stdout:1035 write 0xbfba701 thru 0xbfc99db (0xf2db bytes) 2024-09-14T06:16:38.117 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:38.112+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:38.118 INFO:teuthology.orchestra.run.smithi039.stdout:1036 read 0xa7a9b43 thru 0xa7b3230 (0x96ee bytes) 2024-09-14T06:16:38.118 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:38.118 INFO:teuthology.orchestra.run.smithi039.stdout:1038 write 0x9764628 thru 0x976e301 (0x9cda bytes) 2024-09-14T06:16:38.120 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:38.120 INFO:teuthology.orchestra.run.smithi039.stdout:1040 punch from 0xdff2075 to 0xdffc2df, (0xa26a bytes) 2024-09-14T06:16:38.121 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:38.121 INFO:teuthology.orchestra.run.smithi039.stdout:1043 read 0x1d46808 thru 0x1d4d450 (0x6c49 bytes) 2024-09-14T06:16:38.121 INFO:teuthology.orchestra.run.smithi039.stdout:1044 punch from 0xa123d0f to 0xa128ce5, (0x4fd6 bytes) 2024-09-14T06:16:38.121 INFO:teuthology.orchestra.run.smithi039.stdout:1045 read 0xdf4000a thru 0xdf40012 (0x9 bytes) 2024-09-14T06:16:38.122 INFO:teuthology.orchestra.run.smithi039.stdout:1046 punch from 0x310c750 to 0x311264c, (0x5efc bytes) 2024-09-14T06:16:38.122 INFO:teuthology.orchestra.run.smithi039.stdout:1047 writesame 0x316d7cf thru 0x3172a98 data_size 0x1(0x52ca bytes) 2024-09-14T06:16:38.134 INFO:teuthology.orchestra.run.smithi039.stdout:1049 writesame 0xeb13f66 thru 0xeb1a5bd data_size 0x64(0x6658 bytes) 2024-09-14T06:16:38.134 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:38.134 INFO:teuthology.orchestra.run.smithi039.stdout:1053 read 0xcf8c03b thru 0xcf94d86 (0x8d4c bytes) 2024-09-14T06:16:38.134 INFO:teuthology.orchestra.run.smithi039.stdout:1054 write 0x5692645 thru 0x5698447 (0x5e03 bytes) 2024-09-14T06:16:38.137 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:38.155 INFO:teuthology.orchestra.run.smithi039.stdout:1057 trunc from 0xeb1a5be to 0x1f4b7ce 2024-09-14T06:16:38.191 INFO:teuthology.orchestra.run.smithi039.stdout:1058 write 0x6f08175 thru 0x6f09265 (0x10f1 bytes) 2024-09-14T06:16:38.202 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:38.202 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:1060 writesame 0x790d136 thru 0x7917dd9 data_size 0x1d(0xaca4 bytes) 2024-09-14T06:16:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:1061 write 0x59ed0b4 thru 0x59f16e7 (0x4634 bytes) 2024-09-14T06:16:38.207 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:38.208 INFO:teuthology.orchestra.run.smithi039.stdout:1063 write 0x3f9603 thru 0x3fcf33 (0x3931 bytes) 2024-09-14T06:16:38.210 INFO:teuthology.orchestra.run.smithi039.stdout:1064 trunc from 0x7917dda to 0x70c6cb0 2024-09-14T06:16:38.217 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:16:38.218 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:38.216+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:38.219 INFO:teuthology.orchestra.run.smithi039.stdout:1065 punch from 0x7061b99 to 0x7066f1a, (0x5381 bytes) 2024-09-14T06:16:38.219 INFO:teuthology.orchestra.run.smithi039.stdout:1066 read 0xefc4ef thru 0xf0030e (0x3e20 bytes) 2024-09-14T06:16:38.221 INFO:teuthology.orchestra.run.smithi039.stdout:1067 trunc from 0x70c6cb0 to 0x813c2a8 2024-09-14T06:16:38.229 INFO:teuthology.orchestra.run.smithi039.stdout:1068 trunc from 0x813c2a8 to 0x6927737 2024-09-14T06:16:38.239 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:38.239 INFO:teuthology.orchestra.run.smithi039.stdout:1071 read 0x1706614 thru 0x171575a (0xf147 bytes) 2024-09-14T06:16:38.254 INFO:teuthology.orchestra.run.smithi039.stdout:1072 writesame 0xa8aca23 thru 0xa8bc910 data_size 0x6(0xfeee bytes) 2024-09-14T06:16:38.255 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:38.255 INFO:teuthology.orchestra.run.smithi039.stdout:1074 write 0x105fd7a thru 0x106b1cd (0xb454 bytes) 2024-09-14T06:16:38.257 INFO:teuthology.orchestra.run.smithi039.stdout:1075 read 0x86ad270 thru 0x86b15bc (0x434d bytes) 2024-09-14T06:16:38.257 INFO:teuthology.orchestra.run.smithi039.stdout:1076 read 0x879e7e3 thru 0x87a2495 (0x3cb3 bytes) 2024-09-14T06:16:38.257 INFO:teuthology.orchestra.run.smithi039.stdout:1077 punch from 0x9ff9069 to 0xa003ace, (0xaa65 bytes) 2024-09-14T06:16:38.261 INFO:teuthology.orchestra.run.smithi039.stdout:1078 trunc from 0xa8bc911 to 0xd4e349b 2024-09-14T06:16:38.269 INFO:teuthology.orchestra.run.smithi039.stdout:1079 read 0x8cdcc33 thru 0x8ce7abb (0xae89 bytes) 2024-09-14T06:16:38.269 INFO:teuthology.orchestra.run.smithi039.stdout:1080 write 0x986af2e thru 0x987ae13 (0xfee6 bytes) 2024-09-14T06:16:38.276 INFO:teuthology.orchestra.run.smithi039.stdout:1081 writesame 0x223e209 thru 0x2242493 data_size 0x1(0x428b bytes) 2024-09-14T06:16:38.291 INFO:teuthology.orchestra.run.smithi039.stdout:1082 writesame 0x10fd98b thru 0x10fd9b1 data_size 0x1(0x27 bytes) 2024-09-14T06:16:38.293 INFO:teuthology.orchestra.run.smithi039.stdout:1083 write 0x800cfd thru 0x809503 (0x8807 bytes) 2024-09-14T06:16:38.296 INFO:teuthology.orchestra.run.smithi039.stdout:1084 write 0x26aeeeb thru 0x26b9691 (0xa7a7 bytes) 2024-09-14T06:16:38.299 INFO:teuthology.orchestra.run.smithi039.stdout:1085 writesame 0xa4c5c87 thru 0xa4c74f9 data_size 0x1(0x1873 bytes) 2024-09-14T06:16:38.310 INFO:teuthology.orchestra.run.smithi039.stdout:1088 punch from 0x87ea3d2 to 0x87f7aff, (0xd72d bytes) 2024-09-14T06:16:38.310 INFO:teuthology.orchestra.run.smithi039.stdout:1089 write 0xaa29c89 thru 0xaa394bd (0xf835 bytes) 2024-09-14T06:16:38.319 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:38.320+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:38.330 INFO:teuthology.orchestra.run.smithi039.stdout:1090 write 0xe954688 thru 0xe959d78 (0x56f1 bytes) 2024-09-14T06:16:38.335 INFO:teuthology.orchestra.run.smithi039.stdout:1092 write 0x2e478f7 thru 0x2e50694 (0x8d9e bytes) 2024-09-14T06:16:38.338 INFO:teuthology.orchestra.run.smithi039.stdout:1094 read 0x7d9257c thru 0x7d95207 (0x2c8c bytes) 2024-09-14T06:16:38.339 INFO:teuthology.orchestra.run.smithi039.stdout:1095 write 0x879cc5 thru 0x881779 (0x7ab5 bytes) 2024-09-14T06:16:38.341 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:38.341 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:38.342 INFO:teuthology.orchestra.run.smithi039.stdout:1096 write 0x3ea135b thru 0x3ea1675 (0x31b bytes) 2024-09-14T06:16:38.345 INFO:teuthology.orchestra.run.smithi039.stdout:1097 writesame 0x67ebcea thru 0x67ec270 data_size 0x1(0x587 bytes) 2024-09-14T06:16:38.349 INFO:teuthology.orchestra.run.smithi039.stdout:1098 writesame 0xe52c032 thru 0xe53af82 data_size 0x1(0xef51 bytes) 2024-09-14T06:16:38.355 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:16:38.378 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:16:38.378 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:16:38.379 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:38.386 INFO:teuthology.orchestra.run.smithi039.stdout:1101 trunc from 0xe959d79 to 0xb3c6b95 2024-09-14T06:16:38.400 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2024-09-14T06:16:36.391014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":104023070,"num_objects":188,"num_object_clones":57,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":188,"num_whiteouts":61,"num_read":11382,"num_read_kb":56175,"num_write":2354,"num_write_kb":34151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1840849,"num_keys_recovered":9,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1863,"ondisk_log_size":1863,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270556,"kb_used_data":8520,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754704164,"statfs":{"total":773094113280,"available":772817063936,"internally_reserved":0,"allocated":8724480,"data_stored":36362759,"data_compressed":867612,"data_compressed_allocated":2330624,"data_compressed_original":32223400,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39427268,"num_objects":136,"num_object_clones":57,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":61,"num_read":5586,"num_read_kb":27875,"num_write":1395,"num_write_kb":11865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":542873,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.879554"},"pg_stats":[{"pgid":"3.1f","version":"33'122","reported_seq":498,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.533701+0000","last_change":"2024-09-14T06:16:22.204741+0000","last_active":"2024-09-14T06:16:31.533701+0000","last_peered":"2024-09-14T06:16:31.533701+0000","last_clean":"2024-09-14T06:16:31.533701+0000","last_became_active":"2024-09-14T06:16:20.192649+0000","last_became_peered":"2024-09-14T06:16:20.192649+0000","last_unstale":"2024-09-14T06:16:31.533701+0000","last_undegraded":"2024-09-14T06:16:31.533701+0000","last_fullsized":"2024-09-14T06:16:31.533701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:42.181005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010466430000000001,"stat_sum":{"num_bytes":6380460,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":730,"num_read_kb":5342,"num_write":179,"num_write_kb":2028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"32'23","reported_seq":69,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.533045+0000","last_change":"2024-09-14T06:16:27.310639+0000","last_active":"2024-09-14T06:16:31.533045+0000","last_peered":"2024-09-14T06:16:31.533045+0000","last_clean":"2024-09-14T06:16:31.533045+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:16:31.533045+0000","last_undegraded":"2024-09-14T06:16:31.533045+0000","last_fullsized":"2024-09-14T06:16:31.533045+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:56:49.751336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095849699999999999,"stat_sum":{"num_bytes":2030126,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":4,"num_read":7,"num_read_kb":32,"num_write":20,"num_write_kb":1125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"33'47","reported_seq":423,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.539093+0000","last_change":"2024-09-14T06:16:22.322492+0000","last_active":"2024-09-14T06:16:32.539093+0000","last_peered":"2024-09-14T06:16:32.539093+0000","last_clean":"2024-09-14T06:16:32.539093+0000","last_became_active":"2024-09-14T06:16:20.192807+0000","last_became_peered":"2024-09-14T06:16:20.192807+0000","last_unstale":"2024-09-14T06:16:32.539093+0000","last_undegraded":"2024-09-14T06:16:32.539093+0000","last_fullsized":"2024-09-14T06:16:32.539093+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:13:10.457739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001319401,"stat_sum":{"num_bytes":9422423,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":346,"num_read_kb":5969,"num_write":50,"num_write_kb":1417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"33'97","reported_seq":729,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.606338+0000","last_change":"2024-09-14T06:16:22.201834+0000","last_active":"2024-09-14T06:16:32.606338+0000","last_peered":"2024-09-14T06:16:32.606338+0000","last_clean":"2024-09-14T06:16:32.606338+0000","last_became_active":"2024-09-14T06:16:20.190215+0000","last_became_peered":"2024-09-14T06:16:20.190215+0000","last_unstale":"2024-09-14T06:16:32.606338+0000","last_undegraded":"2024-09-14T06:16:32.606338+0000","last_fullsized":"2024-09-14T06:16:32.606338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:25:38.346496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026065160000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1633,"num_read_kb":1423,"num_write":98,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"32'22","reported_seq":232,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.512518+0000","last_change":"2024-09-14T06:16:22.193116+0000","last_active":"2024-09-14T06:16:31.512518+0000","last_peered":"2024-09-14T06:16:31.512518+0000","last_clean":"2024-09-14T06:16:31.512518+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:16:31.512518+0000","last_undegraded":"2024-09-14T06:16:31.512518+0000","last_fullsized":"2024-09-14T06:16:31.512518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:08.731710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014745120000000001,"stat_sum":{"num_bytes":1909892,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":173,"num_read_kb":1219,"num_write":21,"num_write_kb":181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"32'31","reported_seq":339,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.598493+0000","last_change":"2024-09-14T06:16:22.201461+0000","last_active":"2024-09-14T06:16:32.598493+0000","last_peered":"2024-09-14T06:16:32.598493+0000","last_clean":"2024-09-14T06:16:32.598493+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:16:32.598493+0000","last_undegraded":"2024-09-14T06:16:32.598493+0000","last_fullsized":"2024-09-14T06:16:32.598493+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:13:00.792042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002267542,"stat_sum":{"num_bytes":4597359,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":4,"num_read":280,"num_read_kb":2054,"num_write":35,"num_write_kb":1915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"33'54","reported_seq":167,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.617135+0000","last_change":"2024-09-14T06:16:22.191773+0000","last_active":"2024-09-14T06:16:31.617135+0000","last_peered":"2024-09-14T06:16:31.617135+0000","last_clean":"2024-09-14T06:16:31.617135+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:16:31.617135+0000","last_undegraded":"2024-09-14T06:16:31.617135+0000","last_fullsized":"2024-09-14T06:16:31.617135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:58:19.073959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073833899999999997,"stat_sum":{"num_bytes":5497760,"num_objects":13,"num_object_clones":5,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":117,"num_read_kb":1721,"num_write":64,"num_write_kb":3084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"32'30","reported_seq":69,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.533253+0000","last_change":"2024-09-14T06:16:22.205296+0000","last_active":"2024-09-14T06:16:31.533253+0000","last_peered":"2024-09-14T06:16:31.533253+0000","last_clean":"2024-09-14T06:16:31.533253+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:16:31.533253+0000","last_undegraded":"2024-09-14T06:16:31.533253+0000","last_fullsized":"2024-09-14T06:16:31.533253+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:14:20.958421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012474280000000001,"stat_sum":{"num_bytes":1606999,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":4,"num_read_kb":266,"num_write":27,"num_write_kb":729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"32'44","reported_seq":82,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.533385+0000","last_change":"2024-09-14T06:16:22.204961+0000","last_active":"2024-09-14T06:16:31.533385+0000","last_peered":"2024-09-14T06:16:31.533385+0000","last_clean":"2024-09-14T06:16:31.533385+0000","last_became_active":"2024-09-14T06:16:20.187444+0000","last_became_peered":"2024-09-14T06:16:20.187444+0000","last_unstale":"2024-09-14T06:16:31.533385+0000","last_undegraded":"2024-09-14T06:16:31.533385+0000","last_fullsized":"2024-09-14T06:16:31.533385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:43.084800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001106627,"stat_sum":{"num_bytes":1252822,"num_objects":6,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":3,"num_read":7,"num_read_kb":42,"num_write":43,"num_write_kb":1815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"33'51","reported_seq":149,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-14T06:16:31.813716+0000","last_change":"2024-09-14T06:16:32.541104+0000","last_active":"2024-09-14T06:16:32.541104+0000","last_peered":"2024-09-14T06:16:31.813716+0000","last_clean":"2024-09-14T06:16:31.813716+0000","last_became_active":"2024-09-14T06:16:27.247467+0000","last_became_peered":"2024-09-14T06:16:27.247467+0000","last_unstale":"2024-09-14T06:16:31.813716+0000","last_undegraded":"2024-09-14T06:16:31.813716+0000","last_fullsized":"2024-09-14T06:16:31.813716+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001557389,"stat_sum":{"num_bytes":5327508,"num_objects":11,"num_object_clones":5,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":80,"num_read_kb":1738,"num_write":46,"num_write_kb":1268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"32'29","reported_seq":216,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.544534+0000","last_change":"2024-09-14T06:16:22.194304+0000","last_active":"2024-09-14T06:16:32.544534+0000","last_peered":"2024-09-14T06:16:32.544534+0000","last_clean":"2024-09-14T06:16:32.544534+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:16:32.544534+0000","last_undegraded":"2024-09-14T06:16:32.544534+0000","last_fullsized":"2024-09-14T06:16:32.544534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:22:41.100795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019733379999999998,"stat_sum":{"num_bytes":4108309,"num_objects":6,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":159,"num_read_kb":3757,"num_write":38,"num_write_kb":2507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"33'28","reported_seq":340,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.539594+0000","last_change":"2024-09-14T06:16:27.261533+0000","last_active":"2024-09-14T06:16:32.539594+0000","last_peered":"2024-09-14T06:16:32.539594+0000","last_clean":"2024-09-14T06:16:32.539594+0000","last_became_active":"2024-09-14T06:16:27.248201+0000","last_became_peered":"2024-09-14T06:16:27.248201+0000","last_unstale":"2024-09-14T06:16:32.539594+0000","last_undegraded":"2024-09-14T06:16:32.539594+0000","last_fullsized":"2024-09-14T06:16:32.539594+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:07:31.583519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001465247,"stat_sum":{"num_bytes":2141746,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":212,"num_read_kb":934,"num_write":35,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":14659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.542497+0000","last_change":"2024-09-14T06:16:12.179717+0000","last_active":"2024-09-14T06:16:32.542497+0000","last_peered":"2024-09-14T06:16:32.542497+0000","last_clean":"2024-09-14T06:16:32.542497+0000","last_became_active":"2024-09-14T06:16:10.150426+0000","last_became_peered":"2024-09-14T06:16:10.150426+0000","last_unstale":"2024-09-14T06:16:32.542497+0000","last_undegraded":"2024-09-14T06:16:32.542497+0000","last_fullsized":"2024-09-14T06:16:32.542497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:29:55.403337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010688049999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"31'126","reported_seq":173,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.513016+0000","last_change":"2024-09-14T06:16:22.192972+0000","last_active":"2024-09-14T06:16:31.513016+0000","last_peered":"2024-09-14T06:16:31.513016+0000","last_clean":"2024-09-14T06:16:31.513016+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:16:31.513016+0000","last_undegraded":"2024-09-14T06:16:31.513016+0000","last_fullsized":"2024-09-14T06:16:31.513016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:16.270907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086504599999999996,"stat_sum":{"num_bytes":1998954,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":386,"num_read_kb":298,"num_write":172,"num_write_kb":903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.513188+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:16:31.513188+0000","last_peered":"2024-09-14T06:16:31.513188+0000","last_clean":"2024-09-14T06:16:31.513188+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:16:31.513188+0000","last_undegraded":"2024-09-14T06:16:31.513188+0000","last_fullsized":"2024-09-14T06:16:31.513188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"32'27","reported_seq":714,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.512556+0000","last_change":"2024-09-14T06:16:22.192683+0000","last_active":"2024-09-14T06:16:31.512556+0000","last_peered":"2024-09-14T06:16:31.512556+0000","last_clean":"2024-09-14T06:16:31.512556+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:16:31.512556+0000","last_undegraded":"2024-09-14T06:16:31.512556+0000","last_fullsized":"2024-09-14T06:16:31.512556+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:32:11.335394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001328716,"stat_sum":{"num_bytes":10468179,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":666,"num_read_kb":9501,"num_write":25,"num_write_kb":944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.539888+0000","last_change":"2024-09-14T06:16:12.180261+0000","last_active":"2024-09-14T06:16:32.539888+0000","last_peered":"2024-09-14T06:16:32.539888+0000","last_clean":"2024-09-14T06:16:32.539888+0000","last_became_active":"2024-09-14T06:16:10.147337+0000","last_became_peered":"2024-09-14T06:16:10.147337+0000","last_unstale":"2024-09-14T06:16:32.539888+0000","last_undegraded":"2024-09-14T06:16:32.539888+0000","last_fullsized":"2024-09-14T06:16:32.539888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:54:30.677581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001722187,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"32'35","reported_seq":317,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.516224+0000","last_change":"2024-09-14T06:16:22.213152+0000","last_active":"2024-09-14T06:16:31.516224+0000","last_peered":"2024-09-14T06:16:31.516224+0000","last_clean":"2024-09-14T06:16:31.516224+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:16:31.516224+0000","last_undegraded":"2024-09-14T06:16:31.516224+0000","last_fullsized":"2024-09-14T06:16:31.516224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:56.195141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057590699999999996,"stat_sum":{"num_bytes":7836653,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":255,"num_read_kb":2109,"num_write":40,"num_write_kb":1623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.532899+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:16:31.532899+0000","last_peered":"2024-09-14T06:16:31.532899+0000","last_clean":"2024-09-14T06:16:31.532899+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:16:31.532899+0000","last_undegraded":"2024-09-14T06:16:31.532899+0000","last_fullsized":"2024-09-14T06:16:31.532899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"30'7","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-14T06:16:27.572346+0000","last_change":"2024-09-14T06:16:27.281904+0000","last_active":"2024-09-14T06:16:27.572346+0000","last_peered":"2024-09-14T06:16:27.572346+0000","last_clean":"2024-09-14T06:16:27.572346+0000","last_became_active":"2024-09-14T06:16:27.245439+0000","last_became_peered":"2024-09-14T06:16:27.245439+0000","last_unstale":"2024-09-14T06:16:27.572346+0000","last_undegraded":"2024-09-14T06:16:27.572346+0000","last_fullsized":"2024-09-14T06:16:27.572346+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:09:40.126572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089329700000000004,"stat_sum":{"num_bytes":2742648,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9,"num_write_kb":608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"32'15","reported_seq":68,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.544686+0000","last_change":"2024-09-14T06:16:27.282745+0000","last_active":"2024-09-14T06:16:32.544686+0000","last_peered":"2024-09-14T06:16:32.544686+0000","last_clean":"2024-09-14T06:16:32.544686+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:16:32.544686+0000","last_undegraded":"2024-09-14T06:16:32.544686+0000","last_fullsized":"2024-09-14T06:16:32.544686+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:21:01.776022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002354358,"stat_sum":{"num_bytes":4675684,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":3,"num_read":7,"num_read_kb":114,"num_write":13,"num_write_kb":1184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":46,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.539398+0000","last_change":"2024-09-14T06:16:12.182029+0000","last_active":"2024-09-14T06:16:32.539398+0000","last_peered":"2024-09-14T06:16:32.539398+0000","last_clean":"2024-09-14T06:16:32.539398+0000","last_became_active":"2024-09-14T06:16:10.196390+0000","last_became_peered":"2024-09-14T06:16:10.196390+0000","last_unstale":"2024-09-14T06:16:32.539398+0000","last_undegraded":"2024-09-14T06:16:32.539398+0000","last_fullsized":"2024-09-14T06:16:32.539398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:21.237087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034648000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"33'15","reported_seq":230,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.557521+0000","last_change":"2024-09-14T06:16:22.205051+0000","last_active":"2024-09-14T06:16:31.557521+0000","last_peered":"2024-09-14T06:16:31.557521+0000","last_clean":"2024-09-14T06:16:31.557521+0000","last_became_active":"2024-09-14T06:16:20.196215+0000","last_became_peered":"2024-09-14T06:16:20.196215+0000","last_unstale":"2024-09-14T06:16:31.557521+0000","last_undegraded":"2024-09-14T06:16:31.557521+0000","last_fullsized":"2024-09-14T06:16:31.557521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:49:33.953405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001116559,"stat_sum":{"num_bytes":2583036,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":186,"num_read_kb":1518,"num_write":19,"num_write_kb":711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.538545+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:16:31.538545+0000","last_peered":"2024-09-14T06:16:31.538545+0000","last_clean":"2024-09-14T06:16:31.538545+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:16:31.538545+0000","last_undegraded":"2024-09-14T06:16:31.538545+0000","last_fullsized":"2024-09-14T06:16:31.538545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"32'21","reported_seq":71,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.516149+0000","last_change":"2024-09-14T06:16:27.316499+0000","last_active":"2024-09-14T06:16:31.516149+0000","last_peered":"2024-09-14T06:16:31.516149+0000","last_clean":"2024-09-14T06:16:31.516149+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:16:31.516149+0000","last_undegraded":"2024-09-14T06:16:31.516149+0000","last_fullsized":"2024-09-14T06:16:31.516149+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:41:56.476665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052773999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.538937+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:16:32.538937+0000","last_peered":"2024-09-14T06:16:32.538937+0000","last_clean":"2024-09-14T06:16:32.538937+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:16:32.538937+0000","last_undegraded":"2024-09-14T06:16:32.538937+0000","last_fullsized":"2024-09-14T06:16:32.538937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'55","reported_seq":115,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.512316+0000","last_change":"2024-09-14T06:16:27.302045+0000","last_active":"2024-09-14T06:16:31.512316+0000","last_peered":"2024-09-14T06:16:31.512316+0000","last_clean":"2024-09-14T06:16:31.512316+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:16:31.512316+0000","last_undegraded":"2024-09-14T06:16:31.512316+0000","last_fullsized":"2024-09-14T06:16:31.512316+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:37.164007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060103200000000004,"stat_sum":{"num_bytes":842854,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":16,"num_read_kb":19,"num_write":52,"num_write_kb":254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":65,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.539358+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:16:32.539358+0000","last_peered":"2024-09-14T06:16:32.539358+0000","last_clean":"2024-09-14T06:16:32.539358+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:16:32.539358+0000","last_undegraded":"2024-09-14T06:16:32.539358+0000","last_fullsized":"2024-09-14T06:16:32.539358+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":135,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.550063+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:16:32.550063+0000","last_peered":"2024-09-14T06:16:32.550063+0000","last_clean":"2024-09-14T06:16:32.550063+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:16:32.550063+0000","last_undegraded":"2024-09-14T06:16:32.550063+0000","last_fullsized":"2024-09-14T06:16:32.550063+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"33'46","reported_seq":86,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.606546+0000","last_change":"2024-09-14T06:16:22.191260+0000","last_active":"2024-09-14T06:16:31.606546+0000","last_peered":"2024-09-14T06:16:31.606546+0000","last_clean":"2024-09-14T06:16:31.606546+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:16:31.606546+0000","last_undegraded":"2024-09-14T06:16:31.606546+0000","last_fullsized":"2024-09-14T06:16:31.606546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:20:23.562082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047498199999999999,"stat_sum":{"num_bytes":1968048,"num_objects":7,"num_object_clones":3,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":14,"num_read_kb":25,"num_write":45,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"33'40","reported_seq":81,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.539318+0000","last_change":"2024-09-14T06:16:22.322878+0000","last_active":"2024-09-14T06:16:32.539318+0000","last_peered":"2024-09-14T06:16:32.539318+0000","last_clean":"2024-09-14T06:16:32.539318+0000","last_became_active":"2024-09-14T06:16:20.196837+0000","last_became_peered":"2024-09-14T06:16:20.196837+0000","last_unstale":"2024-09-14T06:16:32.539318+0000","last_undegraded":"2024-09-14T06:16:32.539318+0000","last_fullsized":"2024-09-14T06:16:32.539318+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:32:02.289351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418299,"stat_sum":{"num_bytes":1863471,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":6,"num_read_kb":12,"num_write":38,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"33'34","reported_seq":333,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.539197+0000","last_change":"2024-09-14T06:16:22.323725+0000","last_active":"2024-09-14T06:16:32.539197+0000","last_peered":"2024-09-14T06:16:32.539197+0000","last_clean":"2024-09-14T06:16:32.539197+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:16:32.539197+0000","last_undegraded":"2024-09-14T06:16:32.539197+0000","last_fullsized":"2024-09-14T06:16:32.539197+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:46:29.635227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024221899999999998,"stat_sum":{"num_bytes":3712045,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":264,"num_read_kb":3234,"num_write":34,"num_write_kb":249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"32'19","reported_seq":383,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.538438+0000","last_change":"2024-09-14T06:16:22.219322+0000","last_active":"2024-09-14T06:16:31.538438+0000","last_peered":"2024-09-14T06:16:31.538438+0000","last_clean":"2024-09-14T06:16:31.538438+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:16:31.538438+0000","last_undegraded":"2024-09-14T06:16:31.538438+0000","last_fullsized":"2024-09-14T06:16:31.538438+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:41:21.235024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045017199999999999,"stat_sum":{"num_bytes":6133851,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":339,"num_read_kb":5601,"num_write":25,"num_write_kb":1870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"32'19","reported_seq":373,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.545135+0000","last_change":"2024-09-14T06:16:22.195705+0000","last_active":"2024-09-14T06:16:32.545135+0000","last_peered":"2024-09-14T06:16:32.545135+0000","last_clean":"2024-09-14T06:16:32.545135+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:16:32.545135+0000","last_undegraded":"2024-09-14T06:16:32.545135+0000","last_fullsized":"2024-09-14T06:16:32.545135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:21:53.058155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025578329999999998,"stat_sum":{"num_bytes":1340260,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":315,"num_read_kb":1713,"num_write":21,"num_write_kb":659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"33'413","reported_seq":456,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.540226+0000","last_change":"2024-09-14T06:16:22.323126+0000","last_active":"2024-09-14T06:16:32.540226+0000","last_peered":"2024-09-14T06:16:32.540226+0000","last_clean":"2024-09-14T06:16:32.540226+0000","last_became_active":"2024-09-14T06:16:20.190580+0000","last_became_peered":"2024-09-14T06:16:20.190580+0000","last_unstale":"2024-09-14T06:16:32.540226+0000","last_undegraded":"2024-09-14T06:16:32.540226+0000","last_fullsized":"2024-09-14T06:16:32.540226+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:24:09.750338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001627671,"stat_sum":{"num_bytes":524325,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":2198,"num_read_kb":1791,"num_write":739,"num_write_kb":747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"33'77","reported_seq":129,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-14T06:16:31.583111+0000","last_change":"2024-09-14T06:16:32.545305+0000","last_active":"2024-09-14T06:16:32.545305+0000","last_peered":"2024-09-14T06:16:31.583111+0000","last_clean":"2024-09-14T06:16:31.583111+0000","last_became_active":"2024-09-14T06:16:27.248042+0000","last_became_peered":"2024-09-14T06:16:27.248042+0000","last_unstale":"2024-09-14T06:16:31.583111+0000","last_undegraded":"2024-09-14T06:16:31.583111+0000","last_fullsized":"2024-09-14T06:16:31.583111+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026669160000000001,"stat_sum":{"num_bytes":779493,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":936,"num_read_kb":815,"num_write":71,"num_write_kb":282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"33'26","reported_seq":379,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.544619+0000","last_change":"2024-09-14T06:16:22.195247+0000","last_active":"2024-09-14T06:16:32.544619+0000","last_peered":"2024-09-14T06:16:32.544619+0000","last_clean":"2024-09-14T06:16:32.544619+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:16:32.544619+0000","last_undegraded":"2024-09-14T06:16:32.544619+0000","last_fullsized":"2024-09-14T06:16:32.544619+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:01:50.681933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022870590000000001,"stat_sum":{"num_bytes":1843118,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":321,"num_read_kb":1308,"num_write":22,"num_write_kb":645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"33'43","reported_seq":83,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.550145+0000","last_change":"2024-09-14T06:16:22.200094+0000","last_active":"2024-09-14T06:16:32.550145+0000","last_peered":"2024-09-14T06:16:32.550145+0000","last_clean":"2024-09-14T06:16:32.550145+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:16:32.550145+0000","last_undegraded":"2024-09-14T06:16:32.550145+0000","last_fullsized":"2024-09-14T06:16:32.550145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:35:23.094890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011343239999999999,"stat_sum":{"num_bytes":2435732,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":3,"num_read_kb":0,"num_write":39,"num_write_kb":204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"34'91","reported_seq":586,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.598843+0000","last_change":"2024-09-14T06:16:27.331344+0000","last_active":"2024-09-14T06:16:32.598843+0000","last_peered":"2024-09-14T06:16:32.598843+0000","last_clean":"2024-09-14T06:16:32.598843+0000","last_became_active":"2024-09-14T06:16:27.251848+0000","last_became_peered":"2024-09-14T06:16:27.251848+0000","last_unstale":"2024-09-14T06:16:32.598843+0000","last_undegraded":"2024-09-14T06:16:32.598843+0000","last_fullsized":"2024-09-14T06:16:32.598843+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:31:19.271308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226324,"stat_sum":{"num_bytes":871902,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":963,"num_read_kb":1036,"num_write":109,"num_write_kb":247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":66525,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"32'29","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T06:16:31.516187+0000","last_change":"2024-09-14T06:16:22.213466+0000","last_active":"2024-09-14T06:16:31.516187+0000","last_peered":"2024-09-14T06:16:31.516187+0000","last_clean":"2024-09-14T06:16:31.516187+0000","last_became_active":"2024-09-14T06:16:20.197825+0000","last_became_peered":"2024-09-14T06:16:20.197825+0000","last_unstale":"2024-09-14T06:16:31.516187+0000","last_undegraded":"2024-09-14T06:16:31.516187+0000","last_fullsized":"2024-09-14T06:16:31.516187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:53:12.607419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082016299999999995,"stat_sum":{"num_bytes":2686976,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":3,"num_read":6,"num_read_kb":14,"num_write":33,"num_write_kb":1223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"33'72","reported_seq":393,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-14T06:16:32.592395+0000","last_change":"2024-09-14T06:16:22.322160+0000","last_active":"2024-09-14T06:16:32.592395+0000","last_peered":"2024-09-14T06:16:32.592395+0000","last_clean":"2024-09-14T06:16:32.592395+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:16:32.592395+0000","last_undegraded":"2024-09-14T06:16:32.592395+0000","last_fullsized":"2024-09-14T06:16:32.592395+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:24:16.038287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011502319999999999,"stat_sum":{"num_bytes":3850050,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":681,"num_read_kb":2515,"num_write":69,"num_write_kb":2806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":103432683,"num_objects":184,"num_object_clones":57,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":61,"num_read":11316,"num_read_kb":56120,"num_write":2251,"num_write_kb":32521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":1840849,"num_keys_recovered":9,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6021120,"data_stored":36715454,"data_compressed":898463,"data_compressed_allocated":2453504,"data_compressed_original":33722536,"omap_allocated":0,"internal_metadata":0},"log_size":1788,"ondisk_log_size":1788,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36644,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335196,"statfs":{"total":96636764160,"available":96599240704,"internally_reserved":0,"allocated":1306624,"data_stored":5451708,"data_compressed":218799,"data_compressed_allocated":462848,"data_compressed_original":4900358,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":958464,"data_stored":2183124,"data_compressed":107774,"data_compressed_allocated":212992,"data_compressed_original":1736704,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1003520,"data_stored":4981773,"data_compressed":94236,"data_compressed_allocated":290816,"data_compressed_original":4541994,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":1339392,"data_stored":7753826,"data_compressed":192373,"data_compressed_allocated":524288,"data_compressed_original":7222790,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31836,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":659456,"data_stored":1571652,"data_compressed":14055,"data_compressed_allocated":77824,"data_compressed_original":1232896,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1368064,"data_stored":6973502,"data_compressed":96915,"data_compressed_allocated":356352,"data_compressed_original":6267940,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1081344,"data_stored":4581960,"data_compressed":95929,"data_compressed_allocated":258048,"data_compressed_original":4034090,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1007616,"data_stored":2865214,"data_compressed":47531,"data_compressed_allocated":147456,"data_compressed_original":2286628,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":970752,"data_stored":7605338,"data_compressed":192373,"data_compressed_allocated":524288,"data_compressed_original":7222790,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":4242917,"data_compressed":84046,"data_compressed_allocated":249856,"data_compressed_original":3968554,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":610304,"data_stored":2044920,"data_compressed":107774,"data_compressed_allocated":212992,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1011712,"data_stored":5867895,"data_compressed":222739,"data_compressed_allocated":507904,"data_compressed_original":5449222,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":671744,"data_stored":2729087,"data_compressed":47531,"data_compressed_allocated":147456,"data_compressed_original":2286628,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":839680,"data_stored":5952541,"data_compressed":133030,"data_compressed_allocated":376832,"data_compressed_original":5557802,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1019904,"data_stored":6837257,"data_compressed":96915,"data_compressed_allocated":356352,"data_compressed_original":6267940,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":323584,"data_stored":1435499,"data_compressed":14055,"data_compressed_allocated":77824,"data_compressed_original":1232896,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:16:38.402 INFO:tasks.thrashosds.thrasher:Setting 3.14 to [6, 5, 3, 1] 2024-09-14T06:16:38.402 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.14', '6', '5', '3', '1'] 2024-09-14T06:16:38.402 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.14 6 5 3 1 2024-09-14T06:16:38.407 INFO:teuthology.orchestra.run.smithi039.stdout:1102 read 0xa57df5f thru 0xa589d90 (0xbe32 bytes) 2024-09-14T06:16:38.419 INFO:teuthology.orchestra.run.smithi039.stdout:1104 writesame 0xded1053 thru 0xdedc991 data_size 0x1(0xb93f bytes) 2024-09-14T06:16:38.420 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:38.416+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:38.421 INFO:teuthology.orchestra.run.smithi039.stdout:1105 write 0x8e54ca5 thru 0x8e5eced (0xa049 bytes) 2024-09-14T06:16:38.423 INFO:teuthology.orchestra.run.smithi039.stdout:1106 write 0xaf6c911 thru 0xaf79510 (0xcc00 bytes) 2024-09-14T06:16:38.425 INFO:teuthology.orchestra.run.smithi039.stdout:1107 read 0x39bc20a thru 0x39c9a8b (0xd882 bytes) 2024-09-14T06:16:38.428 INFO:teuthology.orchestra.run.smithi039.stdout:1109 trunc from 0xdedc992 to 0xc4f382d 2024-09-14T06:16:38.437 INFO:teuthology.orchestra.run.smithi039.stdout:1112 punch from 0x49fd22e to 0x4a08974, (0xb746 bytes) 2024-09-14T06:16:38.437 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:38.437 INFO:teuthology.orchestra.run.smithi039.stdout:1114 read 0xa4ecbff thru 0xa4f1645 (0x4a47 bytes) 2024-09-14T06:16:38.437 INFO:teuthology.orchestra.run.smithi039.stdout:1115 write 0x96852ab thru 0x9689cab (0x4a01 bytes) 2024-09-14T06:16:38.439 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:38.439 INFO:teuthology.orchestra.run.smithi039.stdout:1117 read 0x275a006 thru 0x275b231 (0x122c bytes) 2024-09-14T06:16:38.447 INFO:teuthology.orchestra.run.smithi039.stdout:1118 writesame 0xcfa0037 thru 0xcfa0b1e data_size 0x4(0xae8 bytes) 2024-09-14T06:16:38.449 INFO:teuthology.orchestra.run.smithi039.stdout:1119 write 0x5d09f8d thru 0x5d17faa (0xe01e bytes) 2024-09-14T06:16:38.451 INFO:teuthology.orchestra.run.smithi039.stdout:1120 write 0x25a378b thru 0x25b0a8a (0xd300 bytes) 2024-09-14T06:16:38.453 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:38.454 INFO:teuthology.orchestra.run.smithi039.stdout:1122 trunc from 0xcfa0b1f to 0xd728794 2024-09-14T06:16:38.461 INFO:teuthology.orchestra.run.smithi039.stdout:1123 read 0xc3ff58a thru 0xc40e3a6 (0xee1d bytes) 2024-09-14T06:16:38.462 INFO:teuthology.orchestra.run.smithi039.stdout:1125 write 0x5b6e14a thru 0x5b7a7f1 (0xc6a8 bytes) 2024-09-14T06:16:38.464 INFO:teuthology.orchestra.run.smithi039.stdout:1126 write 0x8e59ef6 thru 0x8e647ec (0xa8f7 bytes) 2024-09-14T06:16:38.466 INFO:teuthology.orchestra.run.smithi039.stdout:1127 read 0xa4e32d7 thru 0xa4ef0fd (0xbe27 bytes) 2024-09-14T06:16:38.467 INFO:teuthology.orchestra.run.smithi039.stdout:1128 writesame 0x810ae05 thru 0x8117d5f data_size 0x1(0xcf5b bytes) 2024-09-14T06:16:38.502 INFO:teuthology.orchestra.run.smithi039.stdout:1129 trunc from 0xd728794 to 0x2beb77a 2024-09-14T06:16:38.520 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:38.516+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:38.524 INFO:teuthology.orchestra.run.smithi039.stdout:1131 read 0x194a702 thru 0x1957286 (0xcb85 bytes) 2024-09-14T06:16:38.525 INFO:teuthology.orchestra.run.smithi039.stdout:1132 trunc from 0x2beb77a to 0x2786e57 2024-09-14T06:16:38.537 INFO:teuthology.orchestra.run.smithi039.stdout:1133 read 0x1b501e0 thru 0x1b5e9bd (0xe7de bytes) 2024-09-14T06:16:38.537 INFO:teuthology.orchestra.run.smithi039.stdout:1134 clone 7 order 23 su 2097152 sc 2 2024-09-14T06:16:38.538 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:38.538 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:38.552 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:16:38.621 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:38.620+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:38.674 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:38.674 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:38.688 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:16:38.723 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:38.720+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:38.823 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:38.820+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:38.924 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:38.924+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:38.926 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:38.927 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:38.941 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:16:38.946 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-09-14T06:16:38.947 INFO:teuthology.orchestra.run.smithi039.stderr:set 3.14 pg_upmap_items mapping to [6->5,3->1] 2024-09-14T06:16:39.024 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:39.023+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:39.047 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone6 from 0x2786e57 (overlap 0xccc48b) to 0x147a0d 2024-09-14T06:16:39.074 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:39.074 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:39.090 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:16:39.133 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:39.135+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:39.227 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:39.224+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:39.252 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:39.252 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:39.266 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:16:39.328 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:39.324+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:39.389 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:39.390 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:39.403 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:16:39.429 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:39.424+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:39.521 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:39.521 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:39.530 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:39.531+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:39.536 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:16:39.631 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:39.628+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:39.701 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:39.701 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:39.715 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:16:39.732 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:39.727+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:39.833 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:39.831+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:39.833 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:39.833 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:39.847 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:16:39.933 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:39.927+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:39.992 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:39.992 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:39.998 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-14T06:16:40.012 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:16:40.055 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:40.051+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:40.106 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:16:40.151 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:40.147+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:40.199 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:40.199 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:40.217 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:16:40.257 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:40.258 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:40.258 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:40.251+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:40.277 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:40.277 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:40.294 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:40.295 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:40.313 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:40.313 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:40.334 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:40.338 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:40.339+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:40.356 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:40.356 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:40.378 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:40.378 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:40.381 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:40.381 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:40.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:40.381 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:40.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.381 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4388 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:40.382 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589432+0000", 2024-09-14T06:16:40.382 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.791453741, 2024-09-14T06:16:40.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017698337000000001, 2024-09-14T06:16:40.382 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.382 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.382 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.382 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.382 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.382 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.382 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4388 2024-09-14T06:16:40.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.382 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589432+0000", 2024-09-14T06:16:40.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589432+0000", 2024-09-14T06:16:40.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589433+0000", 2024-09-14T06:16:40.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2300000000000002e-07 2024-09-14T06:16:40.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589435+0000", 2024-09-14T06:16:40.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.902e-06 2024-09-14T06:16:40.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589441+0000", 2024-09-14T06:16:40.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2060000000000004e-06 2024-09-14T06:16:40.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589446+0000", 2024-09-14T06:16:40.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6500000000000004e-06 2024-09-14T06:16:40.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607037+0000", 2024-09-14T06:16:40.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017591496000000002 2024-09-14T06:16:40.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607084+0000", 2024-09-14T06:16:40.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6873999999999999e-05 2024-09-14T06:16:40.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607131+0000", 2024-09-14T06:16:40.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6786000000000002e-05 2024-09-14T06:16:40.386 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.386 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.386 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.386 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4389 3.c 3:37fdb20a:::rbd_data.11528c180b7.0000000000000003:3 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:40.386 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.589453+0000", 2024-09-14T06:16:40.386 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.791433239, 2024-09-14T06:16:40.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017801510999999999, 2024-09-14T06:16:40.386 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.386 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.386 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.386 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.386 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.387 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.387 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4389 2024-09-14T06:16:40.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.387 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.387 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589453+0000", 2024-09-14T06:16:40.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.387 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589453+0000", 2024-09-14T06:16:40.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589453+0000", 2024-09-14T06:16:40.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4499999999999997e-07 2024-09-14T06:16:40.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589455+0000", 2024-09-14T06:16:40.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0370000000000001e-06 2024-09-14T06:16:40.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589462+0000", 2024-09-14T06:16:40.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1360000000000002e-06 2024-09-14T06:16:40.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.589466+0000", 2024-09-14T06:16:40.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.843e-06 2024-09-14T06:16:40.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607149+0000", 2024-09-14T06:16:40.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017682356 2024-09-14T06:16:40.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607209+0000", 2024-09-14T06:16:40.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9902999999999999e-05 2024-09-14T06:16:40.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.607254+0000", 2024-09-14T06:16:40.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5791000000000003e-05 2024-09-14T06:16:40.390 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.390 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.390 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.391 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:10915 3.f 3:f0588a05:::rbd_data.1152f87cbac6.0000000000000004:6 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.391 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.972772+0000", 2024-09-14T06:16:40.391 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4081139340000002, 2024-09-14T06:16:40.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022484922000000001, 2024-09-14T06:16:40.391 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.391 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.391 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.391 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.391 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.391 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.391 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 10915 2024-09-14T06:16:40.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.391 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.391 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.972772+0000", 2024-09-14T06:16:40.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.972772+0000", 2024-09-14T06:16:40.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.972773+0000", 2024-09-14T06:16:40.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7599999999999997e-07 2024-09-14T06:16:40.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.972776+0000", 2024-09-14T06:16:40.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9699999999999999e-06 2024-09-14T06:16:40.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.972783+0000", 2024-09-14T06:16:40.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.525e-06 2024-09-14T06:16:40.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.972789+0000", 2024-09-14T06:16:40.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9780000000000002e-06 2024-09-14T06:16:40.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.983553+0000", 2024-09-14T06:16:40.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.01076331 2024-09-14T06:16:40.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.984173+0000", 2024-09-14T06:16:40.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061989499999999997 2024-09-14T06:16:40.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.995257+0000", 2024-09-14T06:16:40.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011084568 2024-09-14T06:16:40.394 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.394 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.394 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.395 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11183 3.f 3:f0588a05:::rbd_data.1152f87cbac6.0000000000000004:6 [read 573440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.395 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.974509+0000", 2024-09-14T06:16:40.395 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4063773550000001, 2024-09-14T06:16:40.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017691235, 2024-09-14T06:16:40.395 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.395 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.395 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.395 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.395 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.395 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.395 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11183 2024-09-14T06:16:40.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.396 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.396 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.396 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974509+0000", 2024-09-14T06:16:40.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.396 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.396 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974509+0000", 2024-09-14T06:16:40.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.396 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.396 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974509+0000", 2024-09-14T06:16:40.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2e-07 2024-09-14T06:16:40.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974511+0000", 2024-09-14T06:16:40.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0940000000000002e-06 2024-09-14T06:16:40.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974518+0000", 2024-09-14T06:16:40.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6150000000000002e-06 2024-09-14T06:16:40.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974524+0000", 2024-09-14T06:16:40.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2169999999999999e-06 2024-09-14T06:16:40.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992080+0000", 2024-09-14T06:16:40.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017555603999999999 2024-09-14T06:16:40.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992141+0000", 2024-09-14T06:16:40.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1401000000000004e-05 2024-09-14T06:16:40.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992200+0000", 2024-09-14T06:16:40.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8783999999999998e-05 2024-09-14T06:16:40.399 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.399 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.399 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.399 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11184 3.f 3:f0588a05:::rbd_data.1152f87cbac6.0000000000000004:6 [read 696320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.399 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.974532+0000", 2024-09-14T06:16:40.399 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4063540249999997, 2024-09-14T06:16:40.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017810473, 2024-09-14T06:16:40.399 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.399 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.400 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.400 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.400 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.400 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.400 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11184 2024-09-14T06:16:40.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.400 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.400 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.400 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974532+0000", 2024-09-14T06:16:40.400 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.400 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.401 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974532+0000", 2024-09-14T06:16:40.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.401 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.401 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974533+0000", 2024-09-14T06:16:40.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:16:40.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.401 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.401 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974534+0000", 2024-09-14T06:16:40.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.869e-06 2024-09-14T06:16:40.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974540+0000", 2024-09-14T06:16:40.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8289999999999996e-06 2024-09-14T06:16:40.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974545+0000", 2024-09-14T06:16:40.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1189999999999996e-06 2024-09-14T06:16:40.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992222+0000", 2024-09-14T06:16:40.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.01767643 2024-09-14T06:16:40.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992281+0000", 2024-09-14T06:16:40.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9283999999999997e-05 2024-09-14T06:16:40.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992343+0000", 2024-09-14T06:16:40.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1511000000000001e-05 2024-09-14T06:16:40.403 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.403 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.403 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.403 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11185 3.f 3:f0588a05:::rbd_data.1152f87cbac6.0000000000000004:6 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.403 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.974553+0000", 2024-09-14T06:16:40.404 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4063331679999997, 2024-09-14T06:16:40.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017959169000000001, 2024-09-14T06:16:40.404 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.404 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.404 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.404 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.404 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.404 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.404 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11185 2024-09-14T06:16:40.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.404 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974553+0000", 2024-09-14T06:16:40.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974553+0000", 2024-09-14T06:16:40.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974554+0000", 2024-09-14T06:16:40.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6400000000000002e-07 2024-09-14T06:16:40.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974556+0000", 2024-09-14T06:16:40.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.729e-06 2024-09-14T06:16:40.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974563+0000", 2024-09-14T06:16:40.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6209999999999998e-06 2024-09-14T06:16:40.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974568+0000", 2024-09-14T06:16:40.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1229999999999999e-06 2024-09-14T06:16:40.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992367+0000", 2024-09-14T06:16:40.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017799280000000001 2024-09-14T06:16:40.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992434+0000", 2024-09-14T06:16:40.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6667000000000004e-05 2024-09-14T06:16:40.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992512+0000", 2024-09-14T06:16:40.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8084999999999995e-05 2024-09-14T06:16:40.407 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.407 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.408 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.408 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11186 3.f 3:f0588a05:::rbd_data.1152f87cbac6.0000000000000004:6 [read 532480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.408 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.974626+0000", 2024-09-14T06:16:40.408 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.406260165, 2024-09-14T06:16:40.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018043532000000001, 2024-09-14T06:16:40.408 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.408 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.408 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.408 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.408 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.408 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.408 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11186 2024-09-14T06:16:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974626+0000", 2024-09-14T06:16:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974626+0000", 2024-09-14T06:16:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974627+0000", 2024-09-14T06:16:40.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1999999999999999e-07 2024-09-14T06:16:40.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974634+0000", 2024-09-14T06:16:40.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0180000000000004e-06 2024-09-14T06:16:40.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974642+0000", 2024-09-14T06:16:40.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2810000000000008e-06 2024-09-14T06:16:40.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974648+0000", 2024-09-14T06:16:40.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4389999999999997e-06 2024-09-14T06:16:40.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992540+0000", 2024-09-14T06:16:40.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017891289000000001 2024-09-14T06:16:40.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992608+0000", 2024-09-14T06:16:40.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8255000000000003e-05 2024-09-14T06:16:40.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992670+0000", 2024-09-14T06:16:40.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1630000000000005e-05 2024-09-14T06:16:40.412 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.412 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.412 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.412 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11187 3.f 3:f0588a05:::rbd_data.1152f87cbac6.0000000000000004:6 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.412 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.974659+0000", 2024-09-14T06:16:40.412 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4062269389999997, 2024-09-14T06:16:40.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018153424000000001, 2024-09-14T06:16:40.413 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.413 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.413 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.413 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11187 2024-09-14T06:16:40.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.413 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.413 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.413 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974659+0000", 2024-09-14T06:16:40.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974659+0000", 2024-09-14T06:16:40.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974660+0000", 2024-09-14T06:16:40.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8999999999999997e-07 2024-09-14T06:16:40.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974662+0000", 2024-09-14T06:16:40.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.429e-06 2024-09-14T06:16:40.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974669+0000", 2024-09-14T06:16:40.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6579999999999996e-06 2024-09-14T06:16:40.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974674+0000", 2024-09-14T06:16:40.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.198e-06 2024-09-14T06:16:40.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992693+0000", 2024-09-14T06:16:40.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018019352999999998 2024-09-14T06:16:40.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992756+0000", 2024-09-14T06:16:40.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2404999999999996e-05 2024-09-14T06:16:40.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992813+0000", 2024-09-14T06:16:40.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6891000000000001e-05 2024-09-14T06:16:40.417 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.417 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.417 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.417 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11188 3.f 3:f0588a05:::rbd_data.1152f87cbac6.0000000000000004:6 [read 778240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.417 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.974683+0000", 2024-09-14T06:16:40.417 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4062031489999995, 2024-09-14T06:16:40.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018271817999999999, 2024-09-14T06:16:40.417 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.417 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.417 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.417 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.417 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.418 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.418 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11188 2024-09-14T06:16:40.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.418 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974683+0000", 2024-09-14T06:16:40.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974683+0000", 2024-09-14T06:16:40.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974683+0000", 2024-09-14T06:16:40.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5600000000000001e-07 2024-09-14T06:16:40.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974686+0000", 2024-09-14T06:16:40.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:16:40.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974692+0000", 2024-09-14T06:16:40.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.584e-06 2024-09-14T06:16:40.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974697+0000", 2024-09-14T06:16:40.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5959999999999997e-06 2024-09-14T06:16:40.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992835+0000", 2024-09-14T06:16:40.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018138054000000001 2024-09-14T06:16:40.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992896+0000", 2024-09-14T06:16:40.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.101e-05 2024-09-14T06:16:40.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992955+0000", 2024-09-14T06:16:40.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9086999999999999e-05 2024-09-14T06:16:40.421 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.421 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.421 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.421 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11189 3.f 3:f0588a05:::rbd_data.1152f87cbac6.0000000000000004:6 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.422 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.974704+0000", 2024-09-14T06:16:40.422 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4061820000000003, 2024-09-14T06:16:40.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018395495000000001, 2024-09-14T06:16:40.422 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.422 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.422 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.422 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.422 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.422 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.422 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11189 2024-09-14T06:16:40.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.422 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974704+0000", 2024-09-14T06:16:40.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974704+0000", 2024-09-14T06:16:40.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974705+0000", 2024-09-14T06:16:40.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.08e-07 2024-09-14T06:16:40.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974707+0000", 2024-09-14T06:16:40.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.165e-06 2024-09-14T06:16:40.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974713+0000", 2024-09-14T06:16:40.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3459999999999999e-06 2024-09-14T06:16:40.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974718+0000", 2024-09-14T06:16:40.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3229999999999997e-06 2024-09-14T06:16:40.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.425 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.992978+0000", 2024-09-14T06:16:40.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018259099000000001 2024-09-14T06:16:40.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.425 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.993040+0000", 2024-09-14T06:16:40.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2922999999999994e-05 2024-09-14T06:16:40.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.426 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.993100+0000", 2024-09-14T06:16:40.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9231000000000003e-05 2024-09-14T06:16:40.426 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.426 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.426 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.426 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11190 3.f 3:f0588a05:::rbd_data.1152f87cbac6.0000000000000004:6 [read 1024000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.426 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.974726+0000", 2024-09-14T06:16:40.426 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4061606429999998, 2024-09-14T06:16:40.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018521466, 2024-09-14T06:16:40.426 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.426 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.426 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.427 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.427 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.427 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.427 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11190 2024-09-14T06:16:40.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.427 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974726+0000", 2024-09-14T06:16:40.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974726+0000", 2024-09-14T06:16:40.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974726+0000", 2024-09-14T06:16:40.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7599999999999998e-07 2024-09-14T06:16:40.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974728+0000", 2024-09-14T06:16:40.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8649999999999999e-06 2024-09-14T06:16:40.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974734+0000", 2024-09-14T06:16:40.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0870000000000003e-06 2024-09-14T06:16:40.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.974748+0000", 2024-09-14T06:16:40.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3961000000000001e-05 2024-09-14T06:16:40.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.993123+0000", 2024-09-14T06:16:40.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018374970000000001 2024-09-14T06:16:40.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.430 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.430 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.430 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.993186+0000", 2024-09-14T06:16:40.430 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3423000000000006e-05 2024-09-14T06:16:40.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.430 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.430 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.430 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.993247+0000", 2024-09-14T06:16:40.430 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0783999999999999e-05 2024-09-14T06:16:40.430 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.430 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.430 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.430 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.431 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11208 3.f 3:f22406d2:::rbd_data.1152c88ccedd.0000000000000001:5 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.431 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.975139+0000", 2024-09-14T06:16:40.431 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4057475339999996, 2024-09-14T06:16:40.431 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018286386000000002, 2024-09-14T06:16:40.431 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.431 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.431 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.431 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.431 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.431 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.431 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11208 2024-09-14T06:16:40.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.431 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.975139+0000", 2024-09-14T06:16:40.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.975139+0000", 2024-09-14T06:16:40.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.975139+0000", 2024-09-14T06:16:40.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5899999999999995e-07 2024-09-14T06:16:40.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.975143+0000", 2024-09-14T06:16:40.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0199999999999996e-06 2024-09-14T06:16:40.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.975152+0000", 2024-09-14T06:16:40.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0629999999999995e-06 2024-09-14T06:16:40.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.975159+0000", 2024-09-14T06:16:40.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5599999999999999e-06 2024-09-14T06:16:40.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.434 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.434 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.993270+0000", 2024-09-14T06:16:40.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018110503 2024-09-14T06:16:40.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.434 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.434 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.993336+0000", 2024-09-14T06:16:40.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6812999999999999e-05 2024-09-14T06:16:40.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.434 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.434 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.993425+0000", 2024-09-14T06:16:40.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8667999999999999e-05 2024-09-14T06:16:40.434 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.435 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.435 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.435 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.435 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.435 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11209 3.f 3:f22406d2:::rbd_data.1152c88ccedd.0000000000000001:5 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.435 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.975172+0000", 2024-09-14T06:16:40.435 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4057146439999997, 2024-09-14T06:16:40.435 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018421943999999999, 2024-09-14T06:16:40.435 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.435 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.435 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.435 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.435 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.435 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.435 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11209 2024-09-14T06:16:40.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.436 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.436 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.436 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.975172+0000", 2024-09-14T06:16:40.436 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.436 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.436 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.975172+0000", 2024-09-14T06:16:40.436 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.436 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.436 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.975172+0000", 2024-09-14T06:16:40.437 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4600000000000005e-07 2024-09-14T06:16:40.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.437 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.437 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.437 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.975175+0000", 2024-09-14T06:16:40.437 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7269999999999999e-06 2024-09-14T06:16:40.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.437 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.437 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.437 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.975182+0000", 2024-09-14T06:16:40.437 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9299999999999997e-06 2024-09-14T06:16:40.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.437 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.437 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.438 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.975187+0000", 2024-09-14T06:16:40.438 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2179999999999998e-06 2024-09-14T06:16:40.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.438 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.438 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.993449+0000", 2024-09-14T06:16:40.438 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018261771 2024-09-14T06:16:40.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.438 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.438 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.993514+0000", 2024-09-14T06:16:40.438 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5456000000000005e-05 2024-09-14T06:16:40.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.439 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.439 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.993593+0000", 2024-09-14T06:16:40.439 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9295999999999994e-05 2024-09-14T06:16:40.439 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.439 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.439 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.439 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.439 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11237 3.f 3:f0588a05:::rbd_data.1152f87cbac6.0000000000000004:6 [read 221184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.439 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.976117+0000", 2024-09-14T06:16:40.439 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4047691530000002, 2024-09-14T06:16:40.439 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017772992000000001, 2024-09-14T06:16:40.439 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.439 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.440 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.440 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.440 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.440 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.440 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11237 2024-09-14T06:16:40.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.440 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.440 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.440 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.440 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976117+0000", 2024-09-14T06:16:40.440 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.440 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976117+0000", 2024-09-14T06:16:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976118+0000", 2024-09-14T06:16:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8800000000000002e-07 2024-09-14T06:16:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976120+0000", 2024-09-14T06:16:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7800000000000001e-06 2024-09-14T06:16:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.441 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.442 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.442 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976128+0000", 2024-09-14T06:16:40.442 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0779999999999998e-06 2024-09-14T06:16:40.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.442 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.442 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.442 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976133+0000", 2024-09-14T06:16:40.442 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5300000000000004e-06 2024-09-14T06:16:40.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.442 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.442 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.442 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.993763+0000", 2024-09-14T06:16:40.442 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017629623000000001 2024-09-14T06:16:40.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.443 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.443 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.443 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.993829+0000", 2024-09-14T06:16:40.443 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5992000000000001e-05 2024-09-14T06:16:40.443 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.443 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.443 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.443 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.993890+0000", 2024-09-14T06:16:40.443 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1301000000000001e-05 2024-09-14T06:16:40.443 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.443 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.443 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.443 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.443 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.443 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11238 3.f 3:f0588a05:::rbd_data.1152f87cbac6.0000000000000004:6 [read 344064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.444 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.976141+0000", 2024-09-14T06:16:40.444 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4047446800000003, 2024-09-14T06:16:40.444 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017897116000000001, 2024-09-14T06:16:40.444 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.444 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.444 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.444 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.444 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.444 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.444 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11238 2024-09-14T06:16:40.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.444 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.444 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.444 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.445 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976141+0000", 2024-09-14T06:16:40.445 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.445 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.445 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.445 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.447 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976141+0000", 2024-09-14T06:16:40.448 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.448 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.448 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.448 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976142+0000", 2024-09-14T06:16:40.448 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.75e-07 2024-09-14T06:16:40.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.448 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.448 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.448 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976148+0000", 2024-09-14T06:16:40.448 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.553e-06 2024-09-14T06:16:40.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.448 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.448 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.449 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976154+0000", 2024-09-14T06:16:40.449 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6660000000000002e-06 2024-09-14T06:16:40.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.449 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.449 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.449 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976160+0000", 2024-09-14T06:16:40.449 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4269999999999999e-06 2024-09-14T06:16:40.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.449 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.449 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.449 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.993915+0000", 2024-09-14T06:16:40.449 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017755758 2024-09-14T06:16:40.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.450 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.450 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.450 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.993981+0000", 2024-09-14T06:16:40.450 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5454e-05 2024-09-14T06:16:40.450 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.450 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.450 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.450 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.994039+0000", 2024-09-14T06:16:40.450 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7683000000000003e-05 2024-09-14T06:16:40.450 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.450 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.450 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.450 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.450 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.451 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11239 3.f 3:f0588a05:::rbd_data.1152f87cbac6.0000000000000004:6 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.451 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.976167+0000", 2024-09-14T06:16:40.451 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4047187110000001, 2024-09-14T06:16:40.451 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018018933000000001, 2024-09-14T06:16:40.451 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.451 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.451 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.451 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.451 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.451 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.451 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11239 2024-09-14T06:16:40.451 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.451 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.451 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.451 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.452 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976167+0000", 2024-09-14T06:16:40.452 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.452 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.452 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.452 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.452 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976167+0000", 2024-09-14T06:16:40.452 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.452 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.452 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.452 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.452 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976168+0000", 2024-09-14T06:16:40.452 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8999999999999996e-07 2024-09-14T06:16:40.452 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.452 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.453 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.453 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976170+0000", 2024-09-14T06:16:40.453 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.187e-06 2024-09-14T06:16:40.453 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.453 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.453 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.453 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976177+0000", 2024-09-14T06:16:40.453 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3640000000000004e-06 2024-09-14T06:16:40.453 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.454 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.454 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.454 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976181+0000", 2024-09-14T06:16:40.454 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8330000000000001e-06 2024-09-14T06:16:40.454 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.454 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.454 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.454 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.994063+0000", 2024-09-14T06:16:40.454 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.017881261999999998 2024-09-14T06:16:40.454 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.455 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.455 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.455 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.994127+0000", 2024-09-14T06:16:40.455 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4141999999999997e-05 2024-09-14T06:16:40.455 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.455 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.455 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.455 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.994186+0000", 2024-09-14T06:16:40.455 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9555000000000002e-05 2024-09-14T06:16:40.455 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.455 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.455 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.455 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.455 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.456 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11240 3.f 3:f0588a05:::rbd_data.1152f87cbac6.0000000000000004:6 [read 581632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.456 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.976189+0000", 2024-09-14T06:16:40.456 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4046975939999999, 2024-09-14T06:16:40.456 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018141220999999999, 2024-09-14T06:16:40.456 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.456 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.456 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.456 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.456 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.456 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.456 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11240 2024-09-14T06:16:40.456 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.456 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.456 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.457 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.457 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976189+0000", 2024-09-14T06:16:40.457 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.457 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.457 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.457 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976189+0000", 2024-09-14T06:16:40.457 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.457 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.457 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.457 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976189+0000", 2024-09-14T06:16:40.457 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2099999999999996e-07 2024-09-14T06:16:40.458 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.458 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.458 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.458 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976191+0000", 2024-09-14T06:16:40.458 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9479999999999998e-06 2024-09-14T06:16:40.458 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.458 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.458 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.458 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976197+0000", 2024-09-14T06:16:40.458 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1879999999999999e-06 2024-09-14T06:16:40.458 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.458 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.458 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.458 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976203+0000", 2024-09-14T06:16:40.458 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3059999999999997e-06 2024-09-14T06:16:40.459 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.459 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.459 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.459 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.994210+0000", 2024-09-14T06:16:40.459 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018007446 2024-09-14T06:16:40.459 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.459 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.459 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.459 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.994272+0000", 2024-09-14T06:16:40.459 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2189999999999999e-05 2024-09-14T06:16:40.459 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.459 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.459 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.459 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.994330+0000", 2024-09-14T06:16:40.460 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7522000000000003e-05 2024-09-14T06:16:40.460 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.460 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.460 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.460 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.460 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.460 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11241 3.f 3:f0588a05:::rbd_data.1152f87cbac6.0000000000000004:6 [read 704512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.460 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.976210+0000", 2024-09-14T06:16:40.460 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4046764740000004, 2024-09-14T06:16:40.460 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.01826618, 2024-09-14T06:16:40.460 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.460 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.460 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.460 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.461 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.461 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.461 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11241 2024-09-14T06:16:40.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.461 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.461 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.461 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.461 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976210+0000", 2024-09-14T06:16:40.461 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.461 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.461 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.461 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976210+0000", 2024-09-14T06:16:40.462 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.462 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.462 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.462 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.462 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976210+0000", 2024-09-14T06:16:40.462 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.32e-07 2024-09-14T06:16:40.462 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.462 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.462 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.462 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976212+0000", 2024-09-14T06:16:40.462 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.945e-06 2024-09-14T06:16:40.462 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.462 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.462 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.462 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976218+0000", 2024-09-14T06:16:40.463 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0929999999999999e-06 2024-09-14T06:16:40.463 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.463 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.463 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.463 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976223+0000", 2024-09-14T06:16:40.463 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7509999999999999e-06 2024-09-14T06:16:40.463 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.463 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.463 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.463 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.994353+0000", 2024-09-14T06:16:40.463 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018130154999999998 2024-09-14T06:16:40.463 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.463 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.463 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.464 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.994417+0000", 2024-09-14T06:16:40.464 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3767000000000001e-05 2024-09-14T06:16:40.464 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.464 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.464 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.464 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.994476+0000", 2024-09-14T06:16:40.464 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9036999999999997e-05 2024-09-14T06:16:40.464 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.464 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.464 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.464 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.464 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.464 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11242 3.f 3:f0588a05:::rbd_data.1152f87cbac6.0000000000000004:6 [read 827392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.465 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.976230+0000", 2024-09-14T06:16:40.465 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4046559649999999, 2024-09-14T06:16:40.465 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018548490000000001, 2024-09-14T06:16:40.465 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.465 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.465 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.465 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.465 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.465 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.465 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11242 2024-09-14T06:16:40.465 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.465 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.465 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.465 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.465 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976230+0000", 2024-09-14T06:16:40.466 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.466 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.466 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.466 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976230+0000", 2024-09-14T06:16:40.466 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.466 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.466 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.466 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976231+0000", 2024-09-14T06:16:40.466 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:16:40.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.466 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.466 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.466 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976233+0000", 2024-09-14T06:16:40.467 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9850000000000001e-06 2024-09-14T06:16:40.467 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.467 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.467 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.467 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976238+0000", 2024-09-14T06:16:40.467 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7320000000000003e-06 2024-09-14T06:16:40.467 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.467 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.467 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.467 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976245+0000", 2024-09-14T06:16:40.467 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1400000000000002e-06 2024-09-14T06:16:40.467 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.467 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.467 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.468 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.994498+0000", 2024-09-14T06:16:40.468 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018252827999999999 2024-09-14T06:16:40.468 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.468 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.468 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.468 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.994720+0000", 2024-09-14T06:16:40.468 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022148799999999999 2024-09-14T06:16:40.468 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.468 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.468 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.468 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.994779+0000", 2024-09-14T06:16:40.468 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8910999999999999e-05 2024-09-14T06:16:40.468 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.469 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.469 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.469 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.469 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11243 3.f 3:f0588a05:::rbd_data.1152f87cbac6.0000000000000004:6 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.469 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.976254+0000", 2024-09-14T06:16:40.469 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.4046319709999997, 2024-09-14T06:16:40.469 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018681117000000001, 2024-09-14T06:16:40.469 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.469 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.469 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.469 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.469 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.469 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.470 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11243 2024-09-14T06:16:40.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.470 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.470 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.470 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.470 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976254+0000", 2024-09-14T06:16:40.470 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.470 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.470 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.470 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976254+0000", 2024-09-14T06:16:40.470 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.470 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.471 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.471 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976255+0000", 2024-09-14T06:16:40.471 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4900000000000001e-07 2024-09-14T06:16:40.471 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.471 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.471 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.471 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976257+0000", 2024-09-14T06:16:40.471 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4899999999999999e-06 2024-09-14T06:16:40.471 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.471 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.471 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.471 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976264+0000", 2024-09-14T06:16:40.471 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5490000000000003e-06 2024-09-14T06:16:40.471 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.472 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.472 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.472 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.976269+0000", 2024-09-14T06:16:40.472 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9030000000000003e-06 2024-09-14T06:16:40.472 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.472 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.472 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.472 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.994802+0000", 2024-09-14T06:16:40.472 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018533904 2024-09-14T06:16:40.472 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.472 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.472 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.472 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.994870+0000", 2024-09-14T06:16:40.472 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7663000000000006e-05 2024-09-14T06:16:40.473 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.473 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.473 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.473 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.994935+0000", 2024-09-14T06:16:40.473 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5159000000000004e-05 2024-09-14T06:16:40.473 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.473 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.473 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.473 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.473 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.473 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:40.475 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:16:40.475 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:40.476 INFO:teuthology.orchestra.run.smithi039.stdout:1135 read 0xd38dd1 thru 0xd405c9 (0x77f9 bytes) 2024-09-14T06:16:40.477 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:40.477 INFO:teuthology.orchestra.run.smithi039.stdout:1138 read 0xb80c8f thru 0xb8d3b6 (0xc728 bytes) 2024-09-14T06:16:40.477 INFO:teuthology.orchestra.run.smithi039.stdout:1140 punch from 0x21aa7e8 to 0x21b0383, (0x5b9b bytes) 2024-09-14T06:16:40.477 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:40.477 INFO:teuthology.orchestra.run.smithi039.stdout:1143 read 0x11e2e4 thru 0x12dca0 (0xf9bd bytes) 2024-09-14T06:16:40.477 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:40.475+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:40.478 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:40.478 INFO:teuthology.orchestra.run.smithi039.stdout:1145 punch from 0x1209283 to 0x1217bc5, (0xe942 bytes) 2024-09-14T06:16:40.478 INFO:teuthology.orchestra.run.smithi039.stdout:1147 clone 8 order 25 su 524288 sc 8 2024-09-14T06:16:40.575 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:40.571+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:40.601 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:40.601 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:40.601 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:40.601 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:40.616 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:16:40.676 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:40.675+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:40.777 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:40.771+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:40.786 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:40.787 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:40.787 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:40.787 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:40.787 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:40.801 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:16:40.887 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:40.883+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:40.976 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:40.976 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:40.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:40.976 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:40.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.976 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:40.976 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:40.977 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.894685733000003, 2024-09-14T06:16:40.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.56009052699999995, 2024-09-14T06:16:40.977 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.977 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.977 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:40.977 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.977 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:40.977 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:40.977 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 1 2024-09-14T06:16:40.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.977 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.977 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.977 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.977 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:40.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:40.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077065+0000", 2024-09-14T06:16:40.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6429999999999999e-06 2024-09-14T06:16:40.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.979 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.979 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077079+0000", 2024-09-14T06:16:40.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4447000000000001e-05 2024-09-14T06:16:40.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.979 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.979 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077103+0000", 2024-09-14T06:16:40.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3856e-05 2024-09-14T06:16:40.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.979 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.979 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077120+0000", 2024-09-14T06:16:40.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7337e-05 2024-09-14T06:16:40.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077255+0000", 2024-09-14T06:16:40.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000134652 2024-09-14T06:16:40.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:16:40.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077274+0000", 2024-09-14T06:16:40.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9003000000000001e-05 2024-09-14T06:16:40.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634272+0000", 2024-09-14T06:16:40.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.55699813099999995 2024-09-14T06:16:40.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634591+0000", 2024-09-14T06:16:40.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031910800000000002 2024-09-14T06:16:40.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:40.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634892+0000", 2024-09-14T06:16:40.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00030066400000000002 2024-09-14T06:16:40.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:40.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636345+0000", 2024-09-14T06:16:40.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001452896 2024-09-14T06:16:40.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:40.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636974+0000", 2024-09-14T06:16:40.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00062927800000000004 2024-09-14T06:16:40.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:40.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637071+0000", 2024-09-14T06:16:40.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7178999999999999e-05 2024-09-14T06:16:40.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.983 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.983 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637153+0000", 2024-09-14T06:16:40.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1333000000000002e-05 2024-09-14T06:16:40.983 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.983 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.983 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.983 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13136 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.983 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.091799+0000", 2024-09-14T06:16:40.983 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.8799487900000003, 2024-09-14T06:16:40.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022628747000000001, 2024-09-14T06:16:40.984 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.984 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.984 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.984 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.984 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.984 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.984 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13136 2024-09-14T06:16:40.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.984 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.984 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091799+0000", 2024-09-14T06:16:40.984 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091799+0000", 2024-09-14T06:16:40.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091800+0000", 2024-09-14T06:16:40.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.68e-07 2024-09-14T06:16:40.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091802+0000", 2024-09-14T06:16:40.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6699999999999998e-06 2024-09-14T06:16:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091810+0000", 2024-09-14T06:16:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6750000000000002e-06 2024-09-14T06:16:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091816+0000", 2024-09-14T06:16:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1600000000000003e-06 2024-09-14T06:16:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.110345+0000", 2024-09-14T06:16:40.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018528393000000001 2024-09-14T06:16:40.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.111299+0000", 2024-09-14T06:16:40.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00095451100000000001 2024-09-14T06:16:40.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114428+0000", 2024-09-14T06:16:40.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031287699999999999 2024-09-14T06:16:40.987 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.987 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.987 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.987 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13153 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.988 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092161+0000", 2024-09-14T06:16:40.988 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.8795870590000003, 2024-09-14T06:16:40.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022150551000000001, 2024-09-14T06:16:40.988 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.988 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.988 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.988 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.988 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.988 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.988 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13153 2024-09-14T06:16:40.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.988 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092161+0000", 2024-09-14T06:16:40.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092161+0000", 2024-09-14T06:16:40.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092161+0000", 2024-09-14T06:16:40.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.68e-07 2024-09-14T06:16:40.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092164+0000", 2024-09-14T06:16:40.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:16:40.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092172+0000", 2024-09-14T06:16:40.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8029999999999997e-06 2024-09-14T06:16:40.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092178+0000", 2024-09-14T06:16:40.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6210000000000001e-06 2024-09-14T06:16:40.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114130+0000", 2024-09-14T06:16:40.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.021952701000000002 2024-09-14T06:16:40.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114217+0000", 2024-09-14T06:16:40.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6527999999999996e-05 2024-09-14T06:16:40.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114311+0000", 2024-09-14T06:16:40.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4394000000000005e-05 2024-09-14T06:16:40.991 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.991 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.991 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.992 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13154 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.992 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092189+0000", 2024-09-14T06:16:40.992 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.879558941, 2024-09-14T06:16:40.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022359604000000002, 2024-09-14T06:16:40.992 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.992 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.992 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.992 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.992 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.992 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.992 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13154 2024-09-14T06:16:40.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.992 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092189+0000", 2024-09-14T06:16:40.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092189+0000", 2024-09-14T06:16:40.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092189+0000", 2024-09-14T06:16:40.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9700000000000002e-07 2024-09-14T06:16:40.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092192+0000", 2024-09-14T06:16:40.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.317e-06 2024-09-14T06:16:40.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092198+0000", 2024-09-14T06:16:40.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.387e-06 2024-09-14T06:16:40.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092203+0000", 2024-09-14T06:16:40.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.259e-06 2024-09-14T06:16:40.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114338+0000", 2024-09-14T06:16:40.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022135207 2024-09-14T06:16:40.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114451+0000", 2024-09-14T06:16:40.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000112518 2024-09-14T06:16:40.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:40.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114548+0000", 2024-09-14T06:16:40.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7518999999999999e-05 2024-09-14T06:16:40.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.996 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:40.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:40.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.996 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13155 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:40.996 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092211+0000", 2024-09-14T06:16:40.996 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.8795373269999995, 2024-09-14T06:16:40.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.02255184, 2024-09-14T06:16:40.996 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:40.996 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:40.996 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:40.996 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:40.996 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:40.996 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:40.997 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13155 2024-09-14T06:16:40.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.997 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:40.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.997 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:40.997 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092211+0000", 2024-09-14T06:16:40.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.997 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:40.997 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092211+0000", 2024-09-14T06:16:40.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:40.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:40.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092211+0000", 2024-09-14T06:16:40.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8500000000000002e-07 2024-09-14T06:16:40.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:40.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092213+0000", 2024-09-14T06:16:40.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1380000000000001e-06 2024-09-14T06:16:40.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:40.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092220+0000", 2024-09-14T06:16:40.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5250000000000002e-06 2024-09-14T06:16:40.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:40.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092224+0000", 2024-09-14T06:16:40.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.843e-06 2024-09-14T06:16:40.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:40.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114580+0000", 2024-09-14T06:16:40.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022355455999999999 2024-09-14T06:16:40.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:40.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114673+0000", 2024-09-14T06:16:40.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2987999999999998e-05 2024-09-14T06:16:40.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:40.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:40.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114762+0000", 2024-09-14T06:16:41.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9505000000000004e-05 2024-09-14T06:16:41.000 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.000 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.000 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.000 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13156 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 409600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:41.000 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092232+0000", 2024-09-14T06:16:41.000 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.8795162410000001, 2024-09-14T06:16:41.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022738728999999999, 2024-09-14T06:16:41.000 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.001 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.001 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.001 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.001 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.001 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.001 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13156 2024-09-14T06:16:41.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.001 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.001 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.001 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092232+0000", 2024-09-14T06:16:41.001 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092232+0000", 2024-09-14T06:16:41.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092232+0000", 2024-09-14T06:16:41.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8099999999999998e-07 2024-09-14T06:16:41.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092234+0000", 2024-09-14T06:16:41.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0379999999999998e-06 2024-09-14T06:16:41.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092240+0000", 2024-09-14T06:16:41.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2859999999999996e-06 2024-09-14T06:16:41.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092248+0000", 2024-09-14T06:16:41.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5140000000000004e-06 2024-09-14T06:16:41.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114792+0000", 2024-09-14T06:16:41.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022543758000000001 2024-09-14T06:16:41.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114865+0000", 2024-09-14T06:16:41.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3652000000000006e-05 2024-09-14T06:16:41.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114970+0000", 2024-09-14T06:16:41.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010509999999999999 2024-09-14T06:16:41.004 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.004 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.004 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.004 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13157 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 524288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:41.005 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092256+0000", 2024-09-14T06:16:41.005 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.8794923140000002, 2024-09-14T06:16:41.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022964834999999999, 2024-09-14T06:16:41.005 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.005 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.005 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.005 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.005 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.005 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.005 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13157 2024-09-14T06:16:41.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.005 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.006 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.006 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092256+0000", 2024-09-14T06:16:41.006 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.006 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.006 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.006 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092256+0000", 2024-09-14T06:16:41.006 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.006 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.006 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.006 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092256+0000", 2024-09-14T06:16:41.006 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6199999999999998e-07 2024-09-14T06:16:41.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092258+0000", 2024-09-14T06:16:41.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3530000000000002e-06 2024-09-14T06:16:41.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092265+0000", 2024-09-14T06:16:41.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5069999999999997e-06 2024-09-14T06:16:41.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092270+0000", 2024-09-14T06:16:41.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2530000000000004e-06 2024-09-14T06:16:41.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115000+0000", 2024-09-14T06:16:41.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022729787000000001 2024-09-14T06:16:41.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115074+0000", 2024-09-14T06:16:41.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4506999999999997e-05 2024-09-14T06:16:41.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.009 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115220+0000", 2024-09-14T06:16:41.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000145966 2024-09-14T06:16:41.009 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.009 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.009 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.009 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13158 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:41.009 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092282+0000", 2024-09-14T06:16:41.009 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.8794654419999999, 2024-09-14T06:16:41.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.023233365999999998, 2024-09-14T06:16:41.009 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.009 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.010 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.010 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.010 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.010 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.010 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13158 2024-09-14T06:16:41.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.010 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.010 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.010 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092282+0000", 2024-09-14T06:16:41.010 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.010 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.011 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092282+0000", 2024-09-14T06:16:41.011 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.011 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.011 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092283+0000", 2024-09-14T06:16:41.011 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:16:41.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.011 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.011 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092285+0000", 2024-09-14T06:16:41.011 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3460000000000001e-06 2024-09-14T06:16:41.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.012 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092291+0000", 2024-09-14T06:16:41.012 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0549999999999996e-06 2024-09-14T06:16:41.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.012 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092296+0000", 2024-09-14T06:16:41.012 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1800000000000004e-06 2024-09-14T06:16:41.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.012 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115253+0000", 2024-09-14T06:16:41.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022956764000000001 2024-09-14T06:16:41.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.013 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.013 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115347+0000", 2024-09-14T06:16:41.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3762000000000001e-05 2024-09-14T06:16:41.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.013 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.013 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115516+0000", 2024-09-14T06:16:41.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016881200000000001 2024-09-14T06:16:41.013 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.013 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.013 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.014 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.014 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13159 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:41.014 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092307+0000", 2024-09-14T06:16:41.014 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.8794413060000004, 2024-09-14T06:16:41.014 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.023865384999999999, 2024-09-14T06:16:41.014 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.014 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.014 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.014 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.014 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.014 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.014 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13159 2024-09-14T06:16:41.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.015 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.015 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.015 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.015 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092307+0000", 2024-09-14T06:16:41.015 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.015 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.015 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.015 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092307+0000", 2024-09-14T06:16:41.015 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.015 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.015 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.015 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092307+0000", 2024-09-14T06:16:41.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:16:41.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.016 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.016 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.016 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092309+0000", 2024-09-14T06:16:41.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3080000000000002e-06 2024-09-14T06:16:41.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.016 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.016 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.016 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092316+0000", 2024-09-14T06:16:41.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7120000000000003e-06 2024-09-14T06:16:41.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.016 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092321+0000", 2024-09-14T06:16:41.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7729999999999999e-06 2024-09-14T06:16:41.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115546+0000", 2024-09-14T06:16:41.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.023225000999999999 2024-09-14T06:16:41.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115621+0000", 2024-09-14T06:16:41.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4981999999999995e-05 2024-09-14T06:16:41.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.018 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.018 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.018 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116172+0000", 2024-09-14T06:16:41.018 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00055117199999999995 2024-09-14T06:16:41.018 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.018 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.018 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.018 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.018 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13160 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:41.018 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092329+0000", 2024-09-14T06:16:41.018 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.8794191490000003, 2024-09-14T06:16:41.018 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024062849000000001, 2024-09-14T06:16:41.018 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.018 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.019 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.019 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.019 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.019 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.019 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13160 2024-09-14T06:16:41.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.019 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.019 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.019 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.019 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092329+0000", 2024-09-14T06:16:41.019 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.019 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.019 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.019 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092329+0000", 2024-09-14T06:16:41.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.020 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092329+0000", 2024-09-14T06:16:41.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8299999999999997e-07 2024-09-14T06:16:41.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.020 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092331+0000", 2024-09-14T06:16:41.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:16:41.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.021 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092338+0000", 2024-09-14T06:16:41.021 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:16:41.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.021 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.021 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.021 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092351+0000", 2024-09-14T06:16:41.021 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3059e-05 2024-09-14T06:16:41.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.021 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.021 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.021 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116204+0000", 2024-09-14T06:16:41.021 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.023853268 2024-09-14T06:16:41.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.022 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116287+0000", 2024-09-14T06:16:41.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3140999999999996e-05 2024-09-14T06:16:41.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.022 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116392+0000", 2024-09-14T06:16:41.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010451900000000001 2024-09-14T06:16:41.022 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.022 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.022 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.022 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13161 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:41.023 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092359+0000", 2024-09-14T06:16:41.023 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.8793887510000005, 2024-09-14T06:16:41.023 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024248515000000002, 2024-09-14T06:16:41.023 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.023 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.023 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.023 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.023 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.023 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.023 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13161 2024-09-14T06:16:41.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.023 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.023 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.023 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092359+0000", 2024-09-14T06:16:41.024 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.024 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.024 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.024 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092359+0000", 2024-09-14T06:16:41.024 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.024 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.024 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.024 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092360+0000", 2024-09-14T06:16:41.024 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:16:41.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.024 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.024 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.024 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092368+0000", 2024-09-14T06:16:41.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1140000000000006e-06 2024-09-14T06:16:41.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092375+0000", 2024-09-14T06:16:41.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4399999999999999e-06 2024-09-14T06:16:41.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092380+0000", 2024-09-14T06:16:41.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3170000000000001e-06 2024-09-14T06:16:41.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.026 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116426+0000", 2024-09-14T06:16:41.026 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024045492000000002 2024-09-14T06:16:41.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.026 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.026 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.026 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116513+0000", 2024-09-14T06:16:41.026 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6936999999999998e-05 2024-09-14T06:16:41.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.026 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.026 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.026 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116608+0000", 2024-09-14T06:16:41.026 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4765000000000005e-05 2024-09-14T06:16:41.026 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.026 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.026 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.027 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.027 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13162 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:41.027 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092389+0000", 2024-09-14T06:16:41.027 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.8793589390000003, 2024-09-14T06:16:41.027 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024432156, 2024-09-14T06:16:41.027 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.027 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.027 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.027 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.027 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.027 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.027 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13162 2024-09-14T06:16:41.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.027 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.028 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.028 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.028 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092389+0000", 2024-09-14T06:16:41.028 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.028 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.028 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.028 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092389+0000", 2024-09-14T06:16:41.028 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.028 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.028 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.028 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092389+0000", 2024-09-14T06:16:41.028 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:16:41.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092392+0000", 2024-09-14T06:16:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3460000000000001e-06 2024-09-14T06:16:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092398+0000", 2024-09-14T06:16:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6789999999999999e-06 2024-09-14T06:16:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092403+0000", 2024-09-14T06:16:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9389999999999996e-06 2024-09-14T06:16:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116657+0000", 2024-09-14T06:16:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024253896 2024-09-14T06:16:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116732+0000", 2024-09-14T06:16:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4727000000000005e-05 2024-09-14T06:16:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116821+0000", 2024-09-14T06:16:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:16:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.031 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.031 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.031 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13163 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:41.031 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092410+0000", 2024-09-14T06:16:41.031 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.8793374810000003, 2024-09-14T06:16:41.031 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024727810999999999, 2024-09-14T06:16:41.031 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.031 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.031 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.031 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.031 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.031 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.032 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13163 2024-09-14T06:16:41.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.032 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.032 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.032 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.032 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092410+0000", 2024-09-14T06:16:41.032 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.032 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.032 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.032 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092410+0000", 2024-09-14T06:16:41.032 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.032 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.033 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.033 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092411+0000", 2024-09-14T06:16:41.033 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2899999999999999e-07 2024-09-14T06:16:41.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.033 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.033 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.033 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092413+0000", 2024-09-14T06:16:41.033 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3649999999999998e-06 2024-09-14T06:16:41.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.033 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.033 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.033 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092420+0000", 2024-09-14T06:16:41.033 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5729999999999996e-06 2024-09-14T06:16:41.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.034 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.034 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092425+0000", 2024-09-14T06:16:41.034 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0669999999999999e-06 2024-09-14T06:16:41.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.034 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.034 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116850+0000", 2024-09-14T06:16:41.034 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024424833999999999 2024-09-14T06:16:41.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.034 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.034 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117058+0000", 2024-09-14T06:16:41.034 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020800199999999999 2024-09-14T06:16:41.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.035 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.035 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117138+0000", 2024-09-14T06:16:41.035 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0541e-05 2024-09-14T06:16:41.035 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.035 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.035 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.035 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.035 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13164 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:41.035 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092432+0000", 2024-09-14T06:16:41.035 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.8793158050000001, 2024-09-14T06:16:41.035 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024878806999999999, 2024-09-14T06:16:41.035 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.035 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.036 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.036 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.036 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.036 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.036 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13164 2024-09-14T06:16:41.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.036 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.036 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.036 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.036 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092432+0000", 2024-09-14T06:16:41.036 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.036 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.036 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.037 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092432+0000", 2024-09-14T06:16:41.037 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.037 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.037 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.037 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092432+0000", 2024-09-14T06:16:41.037 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:16:41.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.037 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.037 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.037 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092434+0000", 2024-09-14T06:16:41.037 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.968e-06 2024-09-14T06:16:41.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.037 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.038 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.038 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092440+0000", 2024-09-14T06:16:41.038 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.964e-06 2024-09-14T06:16:41.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.038 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.038 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.038 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092445+0000", 2024-09-14T06:16:41.038 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9529999999999998e-06 2024-09-14T06:16:41.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.038 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.038 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.038 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117166+0000", 2024-09-14T06:16:41.038 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024720925000000001 2024-09-14T06:16:41.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.038 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.039 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.039 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117239+0000", 2024-09-14T06:16:41.039 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2650000000000004e-05 2024-09-14T06:16:41.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.039 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.039 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.039 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117311+0000", 2024-09-14T06:16:41.039 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1945000000000003e-05 2024-09-14T06:16:41.039 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.039 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.039 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.039 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.039 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13165 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:41.040 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092453+0000", 2024-09-14T06:16:41.040 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.8792951240000004, 2024-09-14T06:16:41.040 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025052076999999999, 2024-09-14T06:16:41.040 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.040 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.040 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.040 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.040 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.040 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.040 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13165 2024-09-14T06:16:41.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.040 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.040 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.040 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.040 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092453+0000", 2024-09-14T06:16:41.041 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.041 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.041 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.041 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092453+0000", 2024-09-14T06:16:41.041 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.041 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.041 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.041 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092453+0000", 2024-09-14T06:16:41.041 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7300000000000001e-07 2024-09-14T06:16:41.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.041 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.041 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.042 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092455+0000", 2024-09-14T06:16:41.042 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.204e-06 2024-09-14T06:16:41.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.042 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.042 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.042 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092461+0000", 2024-09-14T06:16:41.042 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0299999999999999e-06 2024-09-14T06:16:41.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.042 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.042 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.042 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092466+0000", 2024-09-14T06:16:41.042 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7820000000000001e-06 2024-09-14T06:16:41.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.042 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.043 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.043 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117343+0000", 2024-09-14T06:16:41.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024876334999999999 2024-09-14T06:16:41.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.043 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.043 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117416+0000", 2024-09-14T06:16:41.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3368000000000006e-05 2024-09-14T06:16:41.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.043 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.043 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117505+0000", 2024-09-14T06:16:41.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8885e-05 2024-09-14T06:16:41.043 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.043 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.043 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.044 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.044 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13166 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:41.044 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092490+0000", 2024-09-14T06:16:41.044 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.8792579610000004, 2024-09-14T06:16:41.044 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025236825000000001, 2024-09-14T06:16:41.044 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.044 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.044 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.044 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.044 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.044 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.044 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13166 2024-09-14T06:16:41.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.045 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.045 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.045 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092490+0000", 2024-09-14T06:16:41.045 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.045 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.045 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092490+0000", 2024-09-14T06:16:41.045 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.045 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.045 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092490+0000", 2024-09-14T06:16:41.045 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6199999999999999e-07 2024-09-14T06:16:41.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.046 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.046 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092492+0000", 2024-09-14T06:16:41.046 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7910000000000001e-06 2024-09-14T06:16:41.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.046 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.046 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092497+0000", 2024-09-14T06:16:41.046 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4190000000000001e-06 2024-09-14T06:16:41.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.046 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.046 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092501+0000", 2024-09-14T06:16:41.047 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0230000000000003e-06 2024-09-14T06:16:41.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.047 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.047 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.047 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117533+0000", 2024-09-14T06:16:41.047 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025031623999999999 2024-09-14T06:16:41.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.047 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.047 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.047 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117638+0000", 2024-09-14T06:16:41.047 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010451900000000001 2024-09-14T06:16:41.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.047 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.047 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.047 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117727+0000", 2024-09-14T06:16:41.048 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9086999999999996e-05 2024-09-14T06:16:41.048 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.048 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.048 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.048 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.048 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13167 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:41.048 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092507+0000", 2024-09-14T06:16:41.048 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.8792404730000003, 2024-09-14T06:16:41.048 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025412396, 2024-09-14T06:16:41.048 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.048 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.048 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.048 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.048 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.049 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.049 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13167 2024-09-14T06:16:41.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.049 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.049 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.049 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092507+0000", 2024-09-14T06:16:41.049 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.049 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.049 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092507+0000", 2024-09-14T06:16:41.049 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.050 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.050 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092508+0000", 2024-09-14T06:16:41.050 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.65e-07 2024-09-14T06:16:41.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.050 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.050 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.050 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092509+0000", 2024-09-14T06:16:41.050 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.426e-06 2024-09-14T06:16:41.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.050 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.050 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.050 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092512+0000", 2024-09-14T06:16:41.050 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.416e-06 2024-09-14T06:16:41.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.050 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.051 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.051 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092515+0000", 2024-09-14T06:16:41.051 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.971e-06 2024-09-14T06:16:41.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.051 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.051 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.051 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117755+0000", 2024-09-14T06:16:41.051 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.02523999 2024-09-14T06:16:41.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.051 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.051 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.051 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117830+0000", 2024-09-14T06:16:41.051 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4386000000000002e-05 2024-09-14T06:16:41.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.052 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.052 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.052 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117920+0000", 2024-09-14T06:16:41.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9942e-05 2024-09-14T06:16:41.052 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.052 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.052 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.052 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.052 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13168 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:41.052 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092519+0000", 2024-09-14T06:16:41.052 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.8792283400000001, 2024-09-14T06:16:41.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025607092000000001, 2024-09-14T06:16:41.052 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.053 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.053 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.053 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13168 2024-09-14T06:16:41.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.053 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.053 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.053 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.053 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092519+0000", 2024-09-14T06:16:41.053 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.053 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.053 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.054 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092519+0000", 2024-09-14T06:16:41.054 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.054 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.054 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092520+0000", 2024-09-14T06:16:41.054 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4999999999999999e-07 2024-09-14T06:16:41.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.054 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.054 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092521+0000", 2024-09-14T06:16:41.054 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2440000000000001e-06 2024-09-14T06:16:41.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.055 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.055 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092524+0000", 2024-09-14T06:16:41.055 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3189999999999999e-06 2024-09-14T06:16:41.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.055 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.055 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092527+0000", 2024-09-14T06:16:41.055 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.638e-06 2024-09-14T06:16:41.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.055 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.055 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117948+0000", 2024-09-14T06:16:41.055 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025421293000000001 2024-09-14T06:16:41.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.056 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.056 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.118025+0000", 2024-09-14T06:16:41.056 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6410999999999997e-05 2024-09-14T06:16:41.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.056 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.056 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.056 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.118127+0000", 2024-09-14T06:16:41.056 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000101937 2024-09-14T06:16:41.056 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.057 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.057 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.057 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.057 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13169 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:41.057 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092531+0000", 2024-09-14T06:16:41.057 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.879217143, 2024-09-14T06:16:41.057 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.030222784999999999, 2024-09-14T06:16:41.057 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.057 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.057 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.057 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.057 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.058 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.058 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13169 2024-09-14T06:16:41.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.058 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.058 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.058 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.058 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092531+0000", 2024-09-14T06:16:41.058 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.058 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.058 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.058 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092531+0000", 2024-09-14T06:16:41.058 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.059 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.059 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092531+0000", 2024-09-14T06:16:41.059 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.41e-07 2024-09-14T06:16:41.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.059 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.059 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092545+0000", 2024-09-14T06:16:41.059 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4256999999999999e-05 2024-09-14T06:16:41.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.059 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.060 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092553+0000", 2024-09-14T06:16:41.060 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.374e-06 2024-09-14T06:16:41.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.060 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.060 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.060 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092558+0000", 2024-09-14T06:16:41.060 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2830000000000001e-06 2024-09-14T06:16:41.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.060 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.060 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.060 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.122584+0000", 2024-09-14T06:16:41.060 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.030026091000000001 2024-09-14T06:16:41.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.061 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.061 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.061 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.122672+0000", 2024-09-14T06:16:41.061 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7880999999999994e-05 2024-09-14T06:16:41.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.061 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.061 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.061 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.122753+0000", 2024-09-14T06:16:41.061 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1657999999999996e-05 2024-09-14T06:16:41.061 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.061 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.061 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.061 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.061 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:16984 3.1c 3:3aec2de0:::rbd_data.1152fc09e7e6.000000000000000d:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:41.062 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:37.655478+0000", 2024-09-14T06:16:41.062 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.3162698079999999, 2024-09-14T06:16:41.062 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.034057654999999999, 2024-09-14T06:16:41.062 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.062 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.062 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.062 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.062 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.062 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.062 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 16984 2024-09-14T06:16:41.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.062 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.062 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.062 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.062 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655478+0000", 2024-09-14T06:16:41.062 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.063 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.063 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.063 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655478+0000", 2024-09-14T06:16:41.063 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.063 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.063 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.063 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655480+0000", 2024-09-14T06:16:41.063 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.074e-06 2024-09-14T06:16:41.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.063 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.063 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.063 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655489+0000", 2024-09-14T06:16:41.063 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2799999999999992e-06 2024-09-14T06:16:41.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.064 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.064 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.064 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655509+0000", 2024-09-14T06:16:41.064 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9494e-05 2024-09-14T06:16:41.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.064 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.064 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.064 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655518+0000", 2024-09-14T06:16:41.064 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7730000000000005e-06 2024-09-14T06:16:41.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.064 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.064 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.064 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655599+0000", 2024-09-14T06:16:41.064 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1306999999999999e-05 2024-09-14T06:16:41.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.065 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.065 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.065 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655708+0000", 2024-09-14T06:16:41.065 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010870300000000001 2024-09-14T06:16:41.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.065 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.065 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:41.065 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.661376+0000", 2024-09-14T06:16:41.065 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056679460000000001 2024-09-14T06:16:41.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.065 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.065 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:41.066 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.689383+0000", 2024-09-14T06:16:41.066 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.028007632000000001 2024-09-14T06:16:41.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.066 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.066 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:41.066 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.689407+0000", 2024-09-14T06:16:41.066 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3743e-05 2024-09-14T06:16:41.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.066 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.066 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.066 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.689476+0000", 2024-09-14T06:16:41.066 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9423000000000003e-05 2024-09-14T06:16:41.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.067 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.067 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.067 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.689536+0000", 2024-09-14T06:16:41.067 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9280000000000002e-05 2024-09-14T06:16:41.067 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.067 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.067 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.067 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.067 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.067 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:41.067 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:16:41.069 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:16:41.069 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:41.067+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:41.094 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone7 from 0x2786e57 (overlap 0x2786e57) to 0x0 2024-09-14T06:16:41.170 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:41.171+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:41.222 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-14T06:16:41.241 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:41.241 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:41.241 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:41.241 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:41.245 INFO:teuthology.orchestra.run.smithi039.stdout:1148 trunc from 0x2786e57 to 0x1d6395d 2024-09-14T06:16:41.263 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:16:41.310 INFO:teuthology.orchestra.run.smithi039.stdout:1149 write 0x18ff599 thru 0x190360d (0x4075 bytes) 2024-09-14T06:16:41.310 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:41.303+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:41.312 INFO:teuthology.orchestra.run.smithi039.stdout:1150 punch from 0x98a70c to 0x98ac9f, (0x593 bytes) 2024-09-14T06:16:41.312 INFO:teuthology.orchestra.run.smithi039.stdout:1151 read 0xe58f43 thru 0xe62d0b (0x9dc9 bytes) 2024-09-14T06:16:41.340 INFO:teuthology.orchestra.run.smithi039.stdout:1152 write 0xedd9d2a thru 0xede58a0 (0xbb77 bytes) 2024-09-14T06:16:41.342 INFO:teuthology.orchestra.run.smithi039.stdout:1153 write 0x20a7082 thru 0x20b0963 (0x98e2 bytes) 2024-09-14T06:16:41.345 INFO:teuthology.orchestra.run.smithi039.stdout:1154 write 0x3f6bdac thru 0x3f7a30f (0xe564 bytes) 2024-09-14T06:16:41.349 INFO:teuthology.orchestra.run.smithi039.stdout:1158 writesame 0xeae8963 thru 0xeaf081b data_size 0x1(0x7eb9 bytes) 2024-09-14T06:16:41.372 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:41.367+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:41.391 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:41.391 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:41.391 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:41.391 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:41.391 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:41.393 INFO:teuthology.orchestra.run.smithi039.stdout:1159 write 0xc1b100b thru 0xc1bf7c7 (0xe7bd bytes) 2024-09-14T06:16:41.395 INFO:teuthology.orchestra.run.smithi039.stdout:1162 writesame 0xa7626f4 thru 0xa7647b3 data_size 0x20(0x20c0 bytes) 2024-09-14T06:16:41.399 INFO:teuthology.orchestra.run.smithi039.stdout:1163 read 0xc069e98 thru 0xc072ebc (0x9025 bytes) 2024-09-14T06:16:41.400 INFO:teuthology.orchestra.run.smithi039.stdout:1165 writesame 0xb33eb73 thru 0xb34ae33 data_size 0x1(0xc2c1 bytes) 2024-09-14T06:16:41.406 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:16:41.460 INFO:teuthology.orchestra.run.smithi039.stdout:1166 write 0xe369b3 thru 0xe39f2d (0x357b bytes) 2024-09-14T06:16:41.463 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:41.463 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:41.463 INFO:teuthology.orchestra.run.smithi039.stdout:1170 write 0x9b5f082 thru 0x9b6de75 (0xedf4 bytes) 2024-09-14T06:16:41.467 INFO:teuthology.orchestra.run.smithi039.stdout:1171 write 0xe17abd2 thru 0xe189843 (0xec72 bytes) 2024-09-14T06:16:41.470 INFO:teuthology.orchestra.run.smithi039.stdout:1172 punch from 0x2ea25da to 0x2ea2f4c, (0x972 bytes) 2024-09-14T06:16:41.470 INFO:teuthology.orchestra.run.smithi039.stdout:1173 punch from 0xd27a329 to 0xd286331, (0xc008 bytes) 2024-09-14T06:16:41.470 INFO:teuthology.orchestra.run.smithi039.stdout:1175 read 0xceba831 thru 0xcec911d (0xe8ed bytes) 2024-09-14T06:16:41.470 INFO:teuthology.orchestra.run.smithi039.stdout:1176 write 0xe7e741e thru 0xe7f1c51 (0xa834 bytes) 2024-09-14T06:16:41.473 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:41.471+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:41.494 INFO:teuthology.orchestra.run.smithi039.stdout:1177 trunc from 0xede58a1 to 0x2e7e195 2024-09-14T06:16:41.508 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:41.531 INFO:teuthology.orchestra.run.smithi039.stdout:1181 writesame 0xbaa4f68 thru 0xbaaa80b data_size 0xc(0x58a4 bytes) 2024-09-14T06:16:41.534 INFO:teuthology.orchestra.run.smithi039.stdout:1182 clone 9 order 22 su 262144 sc 7 2024-09-14T06:16:41.574 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:41.575+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:41.591 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:41.591 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:41.591 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:41.591 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:41.591 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.591 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.5 30/29 [PushOp(3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head, version: 28'5, data_included: [0~212992,360448~311296], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head@28'5, size: 1032192, copy_subset: [(0, 1032192)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1032192, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:16:41.591 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:41.591 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.312664024, 2024-09-14T06:16:41.591 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041630249999999999, 2024-09-14T06:16:41.591 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.591 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.591 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.592 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:41.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:41.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275888+0000", 2024-09-14T06:16:41.593 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.367e-06 2024-09-14T06:16:41.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.593 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277098+0000", 2024-09-14T06:16:41.593 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001210661 2024-09-14T06:16:41.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.593 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277122+0000", 2024-09-14T06:16:41.593 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3413000000000001e-05 2024-09-14T06:16:41.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.594 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277141+0000", 2024-09-14T06:16:41.594 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9133000000000001e-05 2024-09-14T06:16:41.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.594 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.594 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277364+0000", 2024-09-14T06:16:41.594 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000223365 2024-09-14T06:16:41.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.594 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.594 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277405+0000", 2024-09-14T06:16:41.594 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1053000000000001e-05 2024-09-14T06:16:41.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.595 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.595 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.280049+0000", 2024-09-14T06:16:41.595 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026440330000000001 2024-09-14T06:16:41.595 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.595 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.595 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.595 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.16 30/29 [PushOp(3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head, version: 28'3, data_included: [0~212992,344064~327680], data_size: 540672, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head@28'3, size: 1032192, copy_subset: [(0, 1032192)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1032192, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:16:41.595 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.287894+0000", 2024-09-14T06:16:41.595 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.300655831, 2024-09-14T06:16:41.595 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032539600000000002, 2024-09-14T06:16:41.595 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.596 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.596 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.596 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.596 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.596 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.596 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.287894+0000", 2024-09-14T06:16:41.596 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.596 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.596 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.596 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.287894+0000", 2024-09-14T06:16:41.596 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.597 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.597 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.287899+0000", 2024-09-14T06:16:41.597 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7280000000000003e-06 2024-09-14T06:16:41.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.597 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.597 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.288805+0000", 2024-09-14T06:16:41.597 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00090622700000000001 2024-09-14T06:16:41.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.597 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.597 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.288855+0000", 2024-09-14T06:16:41.597 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9835e-05 2024-09-14T06:16:41.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.598 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.598 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.288902+0000", 2024-09-14T06:16:41.598 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6854999999999998e-05 2024-09-14T06:16:41.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.598 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.598 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.289120+0000", 2024-09-14T06:16:41.598 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000217495 2024-09-14T06:16:41.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.598 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.598 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.289142+0000", 2024-09-14T06:16:41.599 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2870999999999998e-05 2024-09-14T06:16:41.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.599 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.599 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.599 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.291148+0000", 2024-09-14T06:16:41.599 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020059489999999999 2024-09-14T06:16:41.599 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.599 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.599 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.599 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.599 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3865 3.5 3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head [stat,truncate 1998848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:41.599 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.502395+0000", 2024-09-14T06:16:41.599 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.086155316999999, 2024-09-14T06:16:41.600 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030449240000000001, 2024-09-14T06:16:41.600 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.600 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.600 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.600 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.600 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.600 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.600 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3865 2024-09-14T06:16:41.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.600 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.600 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.600 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.600 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502395+0000", 2024-09-14T06:16:41.600 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.601 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.601 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.601 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502395+0000", 2024-09-14T06:16:41.601 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.601 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.601 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.601 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502397+0000", 2024-09-14T06:16:41.601 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:16:41.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.601 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.601 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.601 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502403+0000", 2024-09-14T06:16:41.602 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3409999999999999e-06 2024-09-14T06:16:41.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.602 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.602 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.602 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502426+0000", 2024-09-14T06:16:41.602 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3181e-05 2024-09-14T06:16:41.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.602 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.602 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.602 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502437+0000", 2024-09-14T06:16:41.602 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0749e-05 2024-09-14T06:16:41.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.602 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.602 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.603 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502510+0000", 2024-09-14T06:16:41.603 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3606e-05 2024-09-14T06:16:41.603 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.603 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.603 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.603 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502595+0000", 2024-09-14T06:16:41.603 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5018999999999994e-05 2024-09-14T06:16:41.603 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.603 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.603 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:41.603 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502745+0000", 2024-09-14T06:16:41.603 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00014936600000000001 2024-09-14T06:16:41.603 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.603 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.604 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:41.604 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.504081+0000", 2024-09-14T06:16:41.604 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0013359310000000001 2024-09-14T06:16:41.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.604 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.604 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:41.604 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505355+0000", 2024-09-14T06:16:41.604 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001273887 2024-09-14T06:16:41.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.604 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.604 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.604 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505402+0000", 2024-09-14T06:16:41.604 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7515000000000002e-05 2024-09-14T06:16:41.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.605 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.605 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.605 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505440+0000", 2024-09-14T06:16:41.605 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7805999999999997e-05 2024-09-14T06:16:41.605 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.605 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.605 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.605 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.605 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3868 3.16 3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head [stat,truncate 1998848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:41.605 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:41.605 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.086068654, 2024-09-14T06:16:41.605 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034445919999999998, 2024-09-14T06:16:41.605 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.606 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.606 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.606 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.606 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.606 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.606 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3868 2024-09-14T06:16:41.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.606 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.606 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.606 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.607 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:41.607 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.607 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.607 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.607 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:41.607 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.607 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.607 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.607 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:41.607 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1900000000000002e-07 2024-09-14T06:16:41.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.607 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.608 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.608 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502501+0000", 2024-09-14T06:16:41.608 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8340999999999999e-05 2024-09-14T06:16:41.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.608 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.608 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.608 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502512+0000", 2024-09-14T06:16:41.608 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1347e-05 2024-09-14T06:16:41.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.608 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.608 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.608 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502518+0000", 2024-09-14T06:16:41.608 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.947e-06 2024-09-14T06:16:41.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.609 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.609 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502570+0000", 2024-09-14T06:16:41.609 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1858999999999999e-05 2024-09-14T06:16:41.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.609 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.609 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502633+0000", 2024-09-14T06:16:41.609 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2974999999999999e-05 2024-09-14T06:16:41.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.609 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:16:41.609 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502795+0000", 2024-09-14T06:16:41.609 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016262899999999999 2024-09-14T06:16:41.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.610 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:41.610 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505179+0000", 2024-09-14T06:16:41.610 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002383844 2024-09-14T06:16:41.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.610 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:41.610 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505676+0000", 2024-09-14T06:16:41.610 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00049634899999999999 2024-09-14T06:16:41.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.610 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.610 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505816+0000", 2024-09-14T06:16:41.611 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00014026300000000001 2024-09-14T06:16:41.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.611 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.611 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.611 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505926+0000", 2024-09-14T06:16:41.611 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000110319 2024-09-14T06:16:41.611 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.611 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.611 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.611 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.611 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3902 3.5 3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:41.611 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.569016+0000", 2024-09-14T06:16:41.611 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.019534782999999, 2024-09-14T06:16:41.612 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003356197, 2024-09-14T06:16:41.612 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.612 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.612 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.612 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.612 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.612 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.612 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3902 2024-09-14T06:16:41.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.612 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.612 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.612 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.612 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569016+0000", 2024-09-14T06:16:41.612 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.613 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.613 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.613 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569016+0000", 2024-09-14T06:16:41.613 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.613 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.613 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.613 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569019+0000", 2024-09-14T06:16:41.613 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2279999999999998e-06 2024-09-14T06:16:41.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.613 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.613 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.613 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569029+0000", 2024-09-14T06:16:41.614 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8079999999999999e-06 2024-09-14T06:16:41.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.614 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.614 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.614 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569059+0000", 2024-09-14T06:16:41.614 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0198000000000001e-05 2024-09-14T06:16:41.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.614 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.614 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.614 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569074+0000", 2024-09-14T06:16:41.614 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4968e-05 2024-09-14T06:16:41.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.614 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.614 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.615 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569165+0000", 2024-09-14T06:16:41.615 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0867999999999998e-05 2024-09-14T06:16:41.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.615 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.615 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569282+0000", 2024-09-14T06:16:41.615 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000117602 2024-09-14T06:16:41.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.615 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:41.615 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569405+0000", 2024-09-14T06:16:41.615 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122471 2024-09-14T06:16:41.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.615 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:41.616 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.570585+0000", 2024-09-14T06:16:41.616 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001180459 2024-09-14T06:16:41.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.616 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:41.616 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.572315+0000", 2024-09-14T06:16:41.616 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0017295329999999999 2024-09-14T06:16:41.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.616 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.616 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.572345+0000", 2024-09-14T06:16:41.616 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0784999999999997e-05 2024-09-14T06:16:41.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.617 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.617 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.572372+0000", 2024-09-14T06:16:41.617 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6276999999999999e-05 2024-09-14T06:16:41.617 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.617 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.617 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.617 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3916 3.16 3:6ab8b1d8:::rbd_data.1152bd92e04f.0000000000000002:head [stat,truncate 1590169] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:41.617 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:41.617 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.009349222999999, 2024-09-14T06:16:41.617 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022958078999999999, 2024-09-14T06:16:41.617 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.618 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.618 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.618 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.618 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.618 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.618 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3916 2024-09-14T06:16:41.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.618 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.618 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.618 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.618 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:41.618 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.618 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.619 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.619 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:41.619 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.619 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.619 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.619 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579203+0000", 2024-09-14T06:16:41.619 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.748e-06 2024-09-14T06:16:41.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.619 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.619 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.619 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579208+0000", 2024-09-14T06:16:41.619 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:16:41.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.620 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.620 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579223+0000", 2024-09-14T06:16:41.620 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.508e-05 2024-09-14T06:16:41.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.620 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.620 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579232+0000", 2024-09-14T06:16:41.620 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9719999999999998e-06 2024-09-14T06:16:41.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.620 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.620 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579307+0000", 2024-09-14T06:16:41.620 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5028000000000001e-05 2024-09-14T06:16:41.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.621 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.621 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579412+0000", 2024-09-14T06:16:41.621 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010525400000000001 2024-09-14T06:16:41.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.621 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:16:41.621 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579535+0000", 2024-09-14T06:16:41.621 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122726 2024-09-14T06:16:41.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.621 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:41.621 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.581415+0000", 2024-09-14T06:16:41.622 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018800360000000001 2024-09-14T06:16:41.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.622 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:41.622 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602017+0000", 2024-09-14T06:16:41.622 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.020602425000000001 2024-09-14T06:16:41.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.622 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.622 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602096+0000", 2024-09-14T06:16:41.622 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8968999999999996e-05 2024-09-14T06:16:41.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.622 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.623 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602159+0000", 2024-09-14T06:16:41.623 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2824999999999996e-05 2024-09-14T06:16:41.623 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.623 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.623 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.623 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.623 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4704 3.5 3:a428c282:::rbd_data.1152bd92e04f.0000000000000047:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:41.623 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.943845+0000", 2024-09-14T06:16:41.623 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.644705318, 2024-09-14T06:16:41.623 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003139476, 2024-09-14T06:16:41.623 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.623 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.623 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.623 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.624 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.624 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.624 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4704 2024-09-14T06:16:41.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.624 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.624 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.624 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.624 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.943845+0000", 2024-09-14T06:16:41.624 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.624 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.624 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.624 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.943845+0000", 2024-09-14T06:16:41.624 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.625 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.625 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.625 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.943846+0000", 2024-09-14T06:16:41.625 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2049999999999999e-06 2024-09-14T06:16:41.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.625 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.625 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.625 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.943851+0000", 2024-09-14T06:16:41.625 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2909999999999998e-06 2024-09-14T06:16:41.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.625 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.625 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.625 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944219+0000", 2024-09-14T06:16:41.625 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036789100000000002 2024-09-14T06:16:41.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.625 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.626 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.626 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944232+0000", 2024-09-14T06:16:41.626 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2564e-05 2024-09-14T06:16:41.626 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.626 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.626 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.626 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944331+0000", 2024-09-14T06:16:41.626 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8628999999999994e-05 2024-09-14T06:16:41.626 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.626 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.626 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.626 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944481+0000", 2024-09-14T06:16:41.626 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015043700000000001 2024-09-14T06:16:41.626 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.626 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.627 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:41.627 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944672+0000", 2024-09-14T06:16:41.627 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000191237 2024-09-14T06:16:41.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.627 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.627 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:41.627 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.945897+0000", 2024-09-14T06:16:41.627 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0012252020000000001 2024-09-14T06:16:41.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.627 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.627 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:41.627 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.946788+0000", 2024-09-14T06:16:41.627 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000890134 2024-09-14T06:16:41.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.627 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.628 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.628 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.946894+0000", 2024-09-14T06:16:41.628 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000106062 2024-09-14T06:16:41.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.628 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.628 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.628 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.946984+0000", 2024-09-14T06:16:41.628 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0823999999999996e-05 2024-09-14T06:16:41.628 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.628 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.628 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.628 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.628 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4873 3.e 3:76656e66:::rbd_data.1152bd92e04f.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=2b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:41.629 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:41.629 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.394902684, 2024-09-14T06:16:41.629 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048796120000000002, 2024-09-14T06:16:41.629 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.629 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.629 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.629 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.629 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.629 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.629 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4873 2024-09-14T06:16:41.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.629 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.629 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.629 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.629 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:41.630 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.630 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.630 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.630 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:41.630 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.630 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.630 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.630 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193651+0000", 2024-09-14T06:16:41.630 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:16:41.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.630 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.630 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.630 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193663+0000", 2024-09-14T06:16:41.631 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2047e-05 2024-09-14T06:16:41.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.631 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.631 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.631 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193692+0000", 2024-09-14T06:16:41.631 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9346e-05 2024-09-14T06:16:41.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.631 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.631 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.631 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193708+0000", 2024-09-14T06:16:41.631 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6123000000000001e-05 2024-09-14T06:16:41.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.631 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.631 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.631 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193798+0000", 2024-09-14T06:16:41.631 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9909999999999998e-05 2024-09-14T06:16:41.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.632 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.632 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193863+0000", 2024-09-14T06:16:41.632 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4583000000000002e-05 2024-09-14T06:16:41.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.632 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:41.632 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.194717+0000", 2024-09-14T06:16:41.632 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00085473299999999999 2024-09-14T06:16:41.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.632 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:41.632 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198400+0000", 2024-09-14T06:16:41.633 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036830890000000001 2024-09-14T06:16:41.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.633 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:41.633 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198461+0000", 2024-09-14T06:16:41.633 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0844000000000002e-05 2024-09-14T06:16:41.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.633 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.633 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198496+0000", 2024-09-14T06:16:41.633 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4820000000000002e-05 2024-09-14T06:16:41.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.634 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.634 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198527+0000", 2024-09-14T06:16:41.634 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1229999999999997e-05 2024-09-14T06:16:41.634 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.634 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.634 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.634 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.634 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:6925 3.5 3:a15e9af1:::rbd_data.1152f87cbac6.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:16:41.634 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:41.634 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.473771146000001, 2024-09-14T06:16:41.634 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0049096490000000003, 2024-09-14T06:16:41.634 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.634 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.634 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.634 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.635 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.635 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.635 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 6925 2024-09-14T06:16:41.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.635 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.635 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.635 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.635 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:41.635 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.635 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.635 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.635 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:41.635 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.636 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.636 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.636 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114780+0000", 2024-09-14T06:16:41.636 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.09e-07 2024-09-14T06:16:41.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.636 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.636 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.636 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114784+0000", 2024-09-14T06:16:41.636 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4669999999999998e-06 2024-09-14T06:16:41.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.636 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.636 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.636 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114799+0000", 2024-09-14T06:16:41.636 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4491000000000001e-05 2024-09-14T06:16:41.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114810+0000", 2024-09-14T06:16:41.637 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1243000000000001e-05 2024-09-14T06:16:41.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115515+0000", 2024-09-14T06:16:41.637 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00070521899999999998 2024-09-14T06:16:41.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115610+0000", 2024-09-14T06:16:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4260999999999998e-05 2024-09-14T06:16:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115742+0000", 2024-09-14T06:16:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000132414 2024-09-14T06:16:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.116650+0000", 2024-09-14T06:16:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00090814900000000002 2024-09-14T06:16:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119568+0000", 2024-09-14T06:16:41.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029178440000000002 2024-09-14T06:16:41.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.639 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.639 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.639 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119636+0000", 2024-09-14T06:16:41.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8365e-05 2024-09-14T06:16:41.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.639 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.639 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.639 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119689+0000", 2024-09-14T06:16:41.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2386999999999998e-05 2024-09-14T06:16:41.639 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.639 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.639 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.640 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:7151 3.e 3:7793b17d:::rbd_data.1152f87cbac6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:16:41.640 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:41.640 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.258953355999999, 2024-09-14T06:16:41.640 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.029556783, 2024-09-14T06:16:41.640 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.640 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.640 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.640 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:41.640 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:41.640 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:41.640 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 7151 2024-09-14T06:16:41.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.640 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.641 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.641 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:41.641 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.641 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.641 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:41.641 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.641 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.641 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329599+0000", 2024-09-14T06:16:41.641 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5270000000000001e-06 2024-09-14T06:16:41.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.642 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.642 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.642 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329610+0000", 2024-09-14T06:16:41.642 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0431000000000001e-05 2024-09-14T06:16:41.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.642 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.642 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.642 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329634+0000", 2024-09-14T06:16:41.642 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3617000000000001e-05 2024-09-14T06:16:41.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.642 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.642 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.642 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329644+0000", 2024-09-14T06:16:41.642 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.095e-05 2024-09-14T06:16:41.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.643 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.643 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329712+0000", 2024-09-14T06:16:41.643 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7067e-05 2024-09-14T06:16:41.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.643 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.643 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329827+0000", 2024-09-14T06:16:41.643 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011498 2024-09-14T06:16:41.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.643 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:41.643 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.334271+0000", 2024-09-14T06:16:41.643 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044445839999999997 2024-09-14T06:16:41.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.644 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.644 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:41.644 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.358967+0000", 2024-09-14T06:16:41.644 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024696356999999999 2024-09-14T06:16:41.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.644 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.644 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:41.644 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359027+0000", 2024-09-14T06:16:41.644 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0006000000000001e-05 2024-09-14T06:16:41.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.644 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.644 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.644 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359099+0000", 2024-09-14T06:16:41.645 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1052999999999999e-05 2024-09-14T06:16:41.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.645 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.645 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.645 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359154+0000", 2024-09-14T06:16:41.645 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5210999999999997e-05 2024-09-14T06:16:41.645 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.645 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.645 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.645 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.645 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:7151 3.e e32/29 ondisk, result = 0)", 2024-09-14T06:16:41.645 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:41.645 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.239008922, 2024-09-14T06:16:41.646 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0096251340000000005, 2024-09-14T06:16:41.646 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.646 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.646 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:41.646 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.646 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.646 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.646 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:41.646 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.646 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.646 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.646 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.646 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:41.647 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.647 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.647 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.647 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349543+0000", 2024-09-14T06:16:41.647 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:16:41.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.647 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.647 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.647 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349552+0000", 2024-09-14T06:16:41.647 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1039999999999996e-06 2024-09-14T06:16:41.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.647 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.647 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.648 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349579+0000", 2024-09-14T06:16:41.648 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6449999999999999e-05 2024-09-14T06:16:41.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.648 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.648 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.648 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349593+0000", 2024-09-14T06:16:41.648 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4099e-05 2024-09-14T06:16:41.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.648 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.648 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.648 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359003+0000", 2024-09-14T06:16:41.648 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0094096119999999995 2024-09-14T06:16:41.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.648 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.648 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.649 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359014+0000", 2024-09-14T06:16:41.649 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0871e-05 2024-09-14T06:16:41.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.649 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.649 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.649 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359167+0000", 2024-09-14T06:16:41.649 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015300700000000001 2024-09-14T06:16:41.649 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.649 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.649 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.649 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.649 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15516 3.1 e36/29 3:860a8b0d:::rbd_data.115223d7013d.00000000000000af:head v 36'26, mlcod=36'24)", 2024-09-14T06:16:41.650 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:41.650 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.3312545719999997, 2024-09-14T06:16:41.650 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035459770000000001, 2024-09-14T06:16:41.650 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.650 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.650 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.650 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.650 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.650 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.650 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:41.650 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.650 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.650 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.650 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:41.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257297+0000", 2024-09-14T06:16:41.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4900000000000004e-07 2024-09-14T06:16:41.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257304+0000", 2024-09-14T06:16:41.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:16:41.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257321+0000", 2024-09-14T06:16:41.652 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6311e-05 2024-09-14T06:16:41.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.652 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.652 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257329+0000", 2024-09-14T06:16:41.652 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0399999999999993e-06 2024-09-14T06:16:41.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.652 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.652 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257414+0000", 2024-09-14T06:16:41.652 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4846999999999996e-05 2024-09-14T06:16:41.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.652 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.652 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257436+0000", 2024-09-14T06:16:41.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2048999999999998e-05 2024-09-14T06:16:41.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.653 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.653 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.653 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.260796+0000", 2024-09-14T06:16:41.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033607559999999999 2024-09-14T06:16:41.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.653 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.653 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.653 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.260842+0000", 2024-09-14T06:16:41.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5226999999999999e-05 2024-09-14T06:16:41.653 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.653 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.653 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.654 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.654 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15558 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'116, mlcod=36'109)", 2024-09-14T06:16:41.654 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.274168+0000", 2024-09-14T06:16:41.654 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.3143822529999998, 2024-09-14T06:16:41.654 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030495710000000001, 2024-09-14T06:16:41.654 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.654 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.654 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.654 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.654 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.654 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.654 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.274168+0000", 2024-09-14T06:16:41.654 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.655 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.274168+0000", 2024-09-14T06:16:41.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.655 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.274168+0000", 2024-09-14T06:16:41.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.65e-07 2024-09-14T06:16:41.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.655 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.274175+0000", 2024-09-14T06:16:41.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8940000000000004e-06 2024-09-14T06:16:41.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.656 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.656 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.274183+0000", 2024-09-14T06:16:41.656 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2949999999999996e-06 2024-09-14T06:16:41.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.656 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.656 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.274188+0000", 2024-09-14T06:16:41.656 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0869999999999997e-06 2024-09-14T06:16:41.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.656 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.656 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.274671+0000", 2024-09-14T06:16:41.657 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000483019 2024-09-14T06:16:41.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.657 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.657 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.274695+0000", 2024-09-14T06:16:41.657 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3969999999999999e-05 2024-09-14T06:16:41.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.657 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.657 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.277169+0000", 2024-09-14T06:16:41.657 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024740420000000001 2024-09-14T06:16:41.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.658 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.658 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.277218+0000", 2024-09-14T06:16:41.658 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8899000000000001e-05 2024-09-14T06:16:41.658 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.658 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.658 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.658 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.658 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15613 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'121, mlcod=36'116)", 2024-09-14T06:16:41.658 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:41.659 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.2956460669999998, 2024-09-14T06:16:41.659 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053619389999999996, 2024-09-14T06:16:41.659 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.659 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.659 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.659 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.659 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.659 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.659 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:41.659 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.659 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.659 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.659 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.659 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:41.660 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.660 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.660 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.660 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292905+0000", 2024-09-14T06:16:41.660 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6800000000000009e-07 2024-09-14T06:16:41.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.660 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.660 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.660 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292915+0000", 2024-09-14T06:16:41.660 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:16:41.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.660 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.661 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.661 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292927+0000", 2024-09-14T06:16:41.661 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-09-14T06:16:41.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.661 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.661 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.661 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292935+0000", 2024-09-14T06:16:41.661 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2459999999999997e-06 2024-09-14T06:16:41.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.661 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.661 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.661 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292992+0000", 2024-09-14T06:16:41.661 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6873999999999998e-05 2024-09-14T06:16:41.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.662 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.662 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.662 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293016+0000", 2024-09-14T06:16:41.662 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4037e-05 2024-09-14T06:16:41.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.662 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.662 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.662 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298217+0000", 2024-09-14T06:16:41.662 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052008710000000001 2024-09-14T06:16:41.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.662 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.662 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.662 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298266+0000", 2024-09-14T06:16:41.662 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9691000000000002e-05 2024-09-14T06:16:41.663 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.663 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.663 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.663 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.663 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.663 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15615 3.11 e36/29 3:8d07d636:::rbd_data.115223d7013d.0000000000000002:head v 36'122, mlcod=36'116)", 2024-09-14T06:16:41.663 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:41.663 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.2947497080000003, 2024-09-14T06:16:41.663 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053069229999999998, 2024-09-14T06:16:41.663 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.663 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.663 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.663 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.663 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.664 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.664 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:41.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.664 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.664 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.664 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:41.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.664 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.664 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.664 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:41.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5799999999999998e-07 2024-09-14T06:16:41.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.665 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.665 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294003+0000", 2024-09-14T06:16:41.665 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020149600000000001 2024-09-14T06:16:41.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.665 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.665 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294016+0000", 2024-09-14T06:16:41.665 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3424999999999999e-05 2024-09-14T06:16:41.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.665 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.665 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294024+0000", 2024-09-14T06:16:41.666 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5240000000000003e-06 2024-09-14T06:16:41.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.666 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.666 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294673+0000", 2024-09-14T06:16:41.666 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064906700000000003 2024-09-14T06:16:41.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.666 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.666 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294698+0000", 2024-09-14T06:16:41.666 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5301999999999999e-05 2024-09-14T06:16:41.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.666 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.667 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299064+0000", 2024-09-14T06:16:41.667 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043652170000000002 2024-09-14T06:16:41.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.667 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.667 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.667 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299108+0000", 2024-09-14T06:16:41.667 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4033999999999999e-05 2024-09-14T06:16:41.667 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.667 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.667 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.667 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.667 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15619 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'124, mlcod=36'119)", 2024-09-14T06:16:41.667 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:41.667 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.2906957600000002, 2024-09-14T06:16:41.668 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061545389999999997, 2024-09-14T06:16:41.668 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.668 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.668 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.668 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.668 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.668 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.668 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:41.668 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.669 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.669 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.669 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:41.669 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.669 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.669 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.669 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:41.669 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3399999999999998e-07 2024-09-14T06:16:41.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.669 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.669 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.669 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297861+0000", 2024-09-14T06:16:41.669 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7640000000000002e-06 2024-09-14T06:16:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297876+0000", 2024-09-14T06:16:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4881e-05 2024-09-14T06:16:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297884+0000", 2024-09-14T06:16:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.588e-06 2024-09-14T06:16:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298294+0000", 2024-09-14T06:16:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00041011200000000002 2024-09-14T06:16:41.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298317+0000", 2024-09-14T06:16:41.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3417999999999998e-05 2024-09-14T06:16:41.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303972+0000", 2024-09-14T06:16:41.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056548680000000004 2024-09-14T06:16:41.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304009+0000", 2024-09-14T06:16:41.672 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7073999999999998e-05 2024-09-14T06:16:41.672 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.672 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.672 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.672 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15621 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'125, mlcod=36'119)", 2024-09-14T06:16:41.672 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:41.672 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.2891847429999999, 2024-09-14T06:16:41.672 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0069496169999999999, 2024-09-14T06:16:41.672 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.672 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.672 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.673 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:41.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:41.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:41.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2200000000000003e-07 2024-09-14T06:16:41.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.674 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.674 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299372+0000", 2024-09-14T06:16:41.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3850000000000001e-06 2024-09-14T06:16:41.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.674 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.674 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299387+0000", 2024-09-14T06:16:41.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4905e-05 2024-09-14T06:16:41.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299393+0000", 2024-09-14T06:16:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9090000000000003e-06 2024-09-14T06:16:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303591+0000", 2024-09-14T06:16:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041976640000000003 2024-09-14T06:16:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303619+0000", 2024-09-14T06:16:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7651e-05 2024-09-14T06:16:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306287+0000", 2024-09-14T06:16:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026686689999999998 2024-09-14T06:16:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306315+0000", 2024-09-14T06:16:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7712000000000002e-05 2024-09-14T06:16:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15623 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'126, mlcod=36'119)", 2024-09-14T06:16:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.2858779299999998, 2024-09-14T06:16:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003658859, 2024-09-14T06:16:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302673+0000", 2024-09-14T06:16:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4199999999999995e-07 2024-09-14T06:16:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302679+0000", 2024-09-14T06:16:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7599999999999999e-06 2024-09-14T06:16:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302689+0000", 2024-09-14T06:16:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0335e-05 2024-09-14T06:16:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302697+0000", 2024-09-14T06:16:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3420000000000002e-06 2024-09-14T06:16:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304047+0000", 2024-09-14T06:16:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0013503530000000001 2024-09-14T06:16:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304059+0000", 2024-09-14T06:16:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2285e-05 2024-09-14T06:16:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306318+0000", 2024-09-14T06:16:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022586580000000002 2024-09-14T06:16:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306331+0000", 2024-09-14T06:16:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3383999999999999e-05 2024-09-14T06:16:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15626 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'127, mlcod=36'120)", 2024-09-14T06:16:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 5.2855135349999998, 2024-09-14T06:16:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038671249999999999, 2024-09-14T06:16:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0100000000000005e-07 2024-09-14T06:16:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303043+0000", 2024-09-14T06:16:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.293e-06 2024-09-14T06:16:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303072+0000", 2024-09-14T06:16:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9135000000000001e-05 2024-09-14T06:16:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303081+0000", 2024-09-14T06:16:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7080000000000003e-06 2024-09-14T06:16:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.305965+0000", 2024-09-14T06:16:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028841769999999999 2024-09-14T06:16:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.305993+0000", 2024-09-14T06:16:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8121000000000001e-05 2024-09-14T06:16:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306861+0000", 2024-09-14T06:16:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00086771099999999996 2024-09-14T06:16:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306904+0000", 2024-09-14T06:16:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3378999999999999e-05 2024-09-14T06:16:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:16521 3.e e36/34 3:77c8e3e5:::rbd_data.1152fc09e7e6.0000000000000003:head v 36'83, mlcod=36'81)", 2024-09-14T06:16:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.6625314429999998, 2024-09-14T06:16:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044948899999999996, 2024-09-14T06:16:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926020+0000", 2024-09-14T06:16:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.545e-06 2024-09-14T06:16:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927192+0000", 2024-09-14T06:16:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011711919999999999 2024-09-14T06:16:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927215+0000", 2024-09-14T06:16:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3677e-05 2024-09-14T06:16:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927226+0000", 2024-09-14T06:16:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0665e-05 2024-09-14T06:16:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927314+0000", 2024-09-14T06:16:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8314999999999997e-05 2024-09-14T06:16:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927344+0000", 2024-09-14T06:16:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9941e-05 2024-09-14T06:16:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.930411+0000", 2024-09-14T06:16:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030670649999999999 2024-09-14T06:16:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.930514+0000", 2024-09-14T06:16:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010249 2024-09-14T06:16:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:41.691 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:41.693 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:16:41.693 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:41.687+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:41.794 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:41.795+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:41.868 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:41.868 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:41.868 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:41.868 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:41.882 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:16:41.931 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:41.927+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:41.990 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-09-14T06:16:41.998 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:41.999+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:42.017 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:42.017 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:42.017 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:42.017 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:42.018 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:42.033 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:16:42.100 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone8 from 0xbaaa80c (overlap 0x1d6395d) to 0xeb1cae 2024-09-14T06:16:42.100 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:42.095+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:42.200 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:42.195+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:42.220 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:42.220 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:42.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:42.221 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:42.221 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.221 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11217 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 925696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.221 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:42.221 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.2287234470000001, 2024-09-14T06:16:42.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049306313999999997, 2024-09-14T06:16:42.221 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.221 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.221 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.221 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.221 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.221 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.221 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11217 2024-09-14T06:16:42.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.222 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.222 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.222 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:42.222 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.222 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.222 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:42.222 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.222 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.222 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:42.223 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.29e-07 2024-09-14T06:16:42.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.223 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.223 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991345+0000", 2024-09-14T06:16:42.223 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1480000000000001e-06 2024-09-14T06:16:42.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.223 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.223 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991349+0000", 2024-09-14T06:16:42.223 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9840000000000003e-06 2024-09-14T06:16:42.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.224 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.224 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991359+0000", 2024-09-14T06:16:42.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4050000000000006e-06 2024-09-14T06:16:42.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.224 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.224 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.030826+0000", 2024-09-14T06:16:42.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.039467214 2024-09-14T06:16:42.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.224 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.224 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.030894+0000", 2024-09-14T06:16:42.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8400999999999998e-05 2024-09-14T06:16:42.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.225 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.225 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040650+0000", 2024-09-14T06:16:42.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0097559329999999996 2024-09-14T06:16:42.225 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.225 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.225 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.225 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.225 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11262 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 81920~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.225 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:42.225 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.2285374109999996, 2024-09-14T06:16:42.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049588879000000002, 2024-09-14T06:16:42.225 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.225 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.225 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.225 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.226 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.226 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.226 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11262 2024-09-14T06:16:42.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.226 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.226 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.226 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.226 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:42.226 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.226 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.226 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.226 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:42.226 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.227 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:42.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9599999999999999e-07 2024-09-14T06:16:42.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.227 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991533+0000", 2024-09-14T06:16:42.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.362e-06 2024-09-14T06:16:42.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.227 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991539+0000", 2024-09-14T06:16:42.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:16:42.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.228 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.228 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991544+0000", 2024-09-14T06:16:42.228 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:16:42.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.228 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.228 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040755+0000", 2024-09-14T06:16:42.228 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049210938000000003 2024-09-14T06:16:42.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.228 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.228 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040833+0000", 2024-09-14T06:16:42.228 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7812999999999995e-05 2024-09-14T06:16:42.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.228 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.229 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041119+0000", 2024-09-14T06:16:42.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0002855 2024-09-14T06:16:42.229 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.229 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.229 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.229 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.229 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11263 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.229 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:42.229 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.2285146329999996, 2024-09-14T06:16:42.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049944574999999998, 2024-09-14T06:16:42.229 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.229 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.229 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.229 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.230 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.230 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.230 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11263 2024-09-14T06:16:42.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.230 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.230 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.230 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:42.230 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.230 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.230 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:42.230 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.51e-07 2024-09-14T06:16:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991560+0000", 2024-09-14T06:16:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6719999999999998e-06 2024-09-14T06:16:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991567+0000", 2024-09-14T06:16:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1470000000000004e-06 2024-09-14T06:16:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991572+0000", 2024-09-14T06:16:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.237e-06 2024-09-14T06:16:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041149+0000", 2024-09-14T06:16:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049576988000000002 2024-09-14T06:16:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041277+0000", 2024-09-14T06:16:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000127687 2024-09-14T06:16:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041497+0000", 2024-09-14T06:16:42.233 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022039299999999999 2024-09-14T06:16:42.233 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.233 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.233 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.233 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.233 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11264 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 327680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.233 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:42.233 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.2284882919999998, 2024-09-14T06:16:42.233 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050248567000000001, 2024-09-14T06:16:42.233 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.233 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.233 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.233 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.234 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.234 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.234 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11264 2024-09-14T06:16:42.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.234 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.234 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.234 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.234 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:42.234 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.234 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.234 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.234 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:42.234 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.235 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.235 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:42.235 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.41e-07 2024-09-14T06:16:42.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.235 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.235 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991581+0000", 2024-09-14T06:16:42.235 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3740000000000001e-06 2024-09-14T06:16:42.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.235 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.235 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991585+0000", 2024-09-14T06:16:42.235 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9400000000000004e-06 2024-09-14T06:16:42.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.236 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.236 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991587+0000", 2024-09-14T06:16:42.236 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.694e-06 2024-09-14T06:16:42.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.236 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.236 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.236 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041527+0000", 2024-09-14T06:16:42.236 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049939349000000001 2024-09-14T06:16:42.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.236 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.236 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.236 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041598+0000", 2024-09-14T06:16:42.236 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1633999999999999e-05 2024-09-14T06:16:42.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.236 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.237 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.237 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041827+0000", 2024-09-14T06:16:42.237 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000229235 2024-09-14T06:16:42.237 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.237 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.237 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.237 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.237 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:42.237 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.228474952, 2024-09-14T06:16:42.237 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:16:42.237 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.237 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.237 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.238 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.238 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.238 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.238 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:16:42.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.238 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.238 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:42.238 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.238 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:42.238 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.239 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.239 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:42.239 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:16:42.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.239 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.239 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:16:42.239 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:16:42.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.239 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.239 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:16:42.239 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:16:42.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.240 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.240 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.240 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:16:42.240 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:16:42.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.240 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.240 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.240 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:16:42.240 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:16:42.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.240 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.240 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.240 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:16:42.240 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:16:42.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.241 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.241 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.241 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:16:42.241 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:16:42.241 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.241 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.241 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.241 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.241 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.241 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:42.241 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.2284618800000002, 2024-09-14T06:16:42.241 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:16:42.241 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.241 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.241 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.242 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.242 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.242 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.242 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:16:42.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.242 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.242 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.242 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:42.242 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.242 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.242 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:42.242 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:16:42.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:16:42.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:16:42.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:16:42.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:16:42.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:16:42.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.244 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.244 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.244 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:16:42.244 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:16:42.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.244 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.244 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.244 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:16:42.244 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:16:42.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.244 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.244 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.244 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:16:42.244 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:16:42.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.245 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.245 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.245 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:16:42.245 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:16:42.245 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.245 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.245 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.245 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.245 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.245 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:42.245 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.2284444480000003, 2024-09-14T06:16:42.245 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:16:42.245 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.245 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.246 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.246 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.246 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.246 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.246 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:16:42.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.246 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.246 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.246 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.246 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:42.246 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.246 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.247 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.247 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:42.247 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.247 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.247 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.247 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:42.247 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:16:42.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.247 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.247 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.247 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:16:42.247 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:16:42.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.248 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.248 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:16:42.248 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:16:42.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.248 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.248 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:16:42.248 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:16:42.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.248 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.248 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:16:42.248 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:16:42.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.249 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.249 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:16:42.249 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:16:42.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.249 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.249 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.249 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:16:42.249 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:16:42.249 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.249 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.249 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.249 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.249 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.250 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:42.250 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.2284233430000002, 2024-09-14T06:16:42.250 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:16:42.250 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.250 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.250 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.250 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.250 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.250 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.250 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:16:42.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.250 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.250 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.250 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.250 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:42.250 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.251 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.251 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.251 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:42.251 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.251 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.251 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.251 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:42.251 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:16:42.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.251 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.251 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.251 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:16:42.251 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:16:42.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.252 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.252 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.252 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:16:42.252 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:16:42.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.252 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.252 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.252 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:16:42.252 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:16:42.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.252 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.252 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.252 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:16:42.252 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:16:42.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.253 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.253 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:16:42.253 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:16:42.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.253 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.253 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:16:42.253 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:16:42.253 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.253 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.253 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.253 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.253 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:42.254 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.2284022349999999, 2024-09-14T06:16:42.254 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:16:42.254 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.254 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.254 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.254 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.254 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.254 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.254 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:16:42.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.254 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.254 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.254 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.254 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:42.254 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.256 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.256 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:16:42.256 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:16:42.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.256 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.256 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.256 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:16:42.256 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:16:42.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.256 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.256 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.256 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:16:42.256 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:16:42.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.256 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.257 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.257 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:16:42.257 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:16:42.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.257 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.257 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.257 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:16:42.257 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:16:42.257 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.257 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.257 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.257 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.257 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13590 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 491520~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.258 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:42.258 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.0824273529999999, 2024-09-14T06:16:42.258 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042553466999999998, 2024-09-14T06:16:42.258 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.258 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.258 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.258 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.258 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.258 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.258 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13590 2024-09-14T06:16:42.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.258 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.258 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.258 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.259 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:42.259 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.259 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.259 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.259 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:42.259 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.259 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.259 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.259 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:42.259 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.34e-07 2024-09-14T06:16:42.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.259 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.260 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.260 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137642+0000", 2024-09-14T06:16:42.260 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0150000000000002e-06 2024-09-14T06:16:42.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.260 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.260 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.260 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137649+0000", 2024-09-14T06:16:42.260 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4010000000000002e-06 2024-09-14T06:16:42.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.261 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.261 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137653+0000", 2024-09-14T06:16:42.261 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:16:42.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.261 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.261 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.179985+0000", 2024-09-14T06:16:42.261 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042331580000000001 2024-09-14T06:16:42.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.261 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.261 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180076+0000", 2024-09-14T06:16:42.261 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0796000000000003e-05 2024-09-14T06:16:42.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.262 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.262 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.262 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180193+0000", 2024-09-14T06:16:42.262 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000117471 2024-09-14T06:16:42.262 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.262 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.262 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.262 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.262 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13592 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 606208~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.262 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:42.262 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.0824027850000002, 2024-09-14T06:16:42.262 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042709440000000001, 2024-09-14T06:16:42.262 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.262 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.263 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.263 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.263 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.263 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.263 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13592 2024-09-14T06:16:42.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.263 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.263 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.263 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.263 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:42.263 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.263 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.263 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.264 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:42.264 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.264 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.264 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.264 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137665+0000", 2024-09-14T06:16:42.264 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4299999999999998e-07 2024-09-14T06:16:42.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.264 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.264 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.264 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137667+0000", 2024-09-14T06:16:42.264 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5780000000000001e-06 2024-09-14T06:16:42.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.264 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.265 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.265 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137675+0000", 2024-09-14T06:16:42.265 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0870000000000001e-06 2024-09-14T06:16:42.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.265 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.265 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.265 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137680+0000", 2024-09-14T06:16:42.265 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4999999999999999e-06 2024-09-14T06:16:42.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.265 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.265 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.265 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180223+0000", 2024-09-14T06:16:42.265 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042542808000000001 2024-09-14T06:16:42.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.266 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.266 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180294+0000", 2024-09-14T06:16:42.266 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1497999999999999e-05 2024-09-14T06:16:42.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.266 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.266 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180374+0000", 2024-09-14T06:16:42.266 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9525999999999997e-05 2024-09-14T06:16:42.266 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.266 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.266 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.266 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13594 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 729088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.266 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:42.267 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.0823792210000001, 2024-09-14T06:16:42.267 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042830604000000001, 2024-09-14T06:16:42.267 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.267 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.267 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.267 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.267 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.267 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.267 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13594 2024-09-14T06:16:42.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.267 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.267 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.267 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.267 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:42.267 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.268 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.268 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.268 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:42.268 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.268 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.268 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.268 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:42.268 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0699999999999998e-07 2024-09-14T06:16:42.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.268 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.268 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.268 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137690+0000", 2024-09-14T06:16:42.269 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.012e-06 2024-09-14T06:16:42.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.269 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.269 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.269 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137698+0000", 2024-09-14T06:16:42.269 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4379999999999998e-06 2024-09-14T06:16:42.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.269 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.269 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.269 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137703+0000", 2024-09-14T06:16:42.270 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1250000000000001e-06 2024-09-14T06:16:42.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.270 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.270 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.270 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180404+0000", 2024-09-14T06:16:42.270 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042701465000000001 2024-09-14T06:16:42.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.270 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.270 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.270 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180456+0000", 2024-09-14T06:16:42.271 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1227999999999998e-05 2024-09-14T06:16:42.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.271 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.271 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180519+0000", 2024-09-14T06:16:42.271 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-09-14T06:16:42.271 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.271 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.271 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.271 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13595 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 851968~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.271 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:42.271 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.0823567159999996, 2024-09-14T06:16:42.271 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043012433000000003, 2024-09-14T06:16:42.271 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.272 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.272 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.272 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.272 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.272 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.272 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13595 2024-09-14T06:16:42.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.272 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.272 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.272 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:42.272 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.272 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.273 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:42.273 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.273 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.273 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137711+0000", 2024-09-14T06:16:42.273 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:16:42.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.273 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.273 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137713+0000", 2024-09-14T06:16:42.273 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5170000000000002e-06 2024-09-14T06:16:42.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.274 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.274 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137720+0000", 2024-09-14T06:16:42.274 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-09-14T06:16:42.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.274 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.274 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137726+0000", 2024-09-14T06:16:42.274 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9580000000000004e-06 2024-09-14T06:16:42.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.274 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.274 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180551+0000", 2024-09-14T06:16:42.274 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042825611999999999 2024-09-14T06:16:42.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.275 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180623+0000", 2024-09-14T06:16:42.275 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1673000000000003e-05 2024-09-14T06:16:42.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.275 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180723+0000", 2024-09-14T06:16:42.275 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.9928999999999998e-05 2024-09-14T06:16:42.275 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.275 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.275 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.275 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13597 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 974848~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.276 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:42.276 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.0823338309999997, 2024-09-14T06:16:42.276 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043180442999999999, 2024-09-14T06:16:42.276 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.276 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.276 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.276 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.276 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.276 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.276 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13597 2024-09-14T06:16:42.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.276 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.276 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.276 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.277 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:42.277 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.277 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.277 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.277 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:42.277 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.277 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.278 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.278 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137734+0000", 2024-09-14T06:16:42.278 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4499999999999997e-07 2024-09-14T06:16:42.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.278 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.278 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.278 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137736+0000", 2024-09-14T06:16:42.278 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0470000000000001e-06 2024-09-14T06:16:42.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.278 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.278 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.278 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137742+0000", 2024-09-14T06:16:42.278 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2269999999999998e-06 2024-09-14T06:16:42.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.279 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.279 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137747+0000", 2024-09-14T06:16:42.279 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6670000000000004e-06 2024-09-14T06:16:42.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.279 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.279 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180753+0000", 2024-09-14T06:16:42.279 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043006132000000002 2024-09-14T06:16:42.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.279 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.280 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180820+0000", 2024-09-14T06:16:42.280 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7516999999999997e-05 2024-09-14T06:16:42.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.280 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.280 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.280 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180914+0000", 2024-09-14T06:16:42.280 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3407999999999997e-05 2024-09-14T06:16:42.280 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.280 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.280 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.280 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.280 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13684 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 999424~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.280 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:42.281 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.0801012879999998, 2024-09-14T06:16:42.281 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042643702999999998, 2024-09-14T06:16:42.281 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.281 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.281 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.281 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.281 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.281 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.281 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13684 2024-09-14T06:16:42.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.281 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.281 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.281 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.282 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:42.282 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.282 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.282 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.282 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:42.282 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.282 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.282 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.282 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139967+0000", 2024-09-14T06:16:42.282 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1200000000000003e-07 2024-09-14T06:16:42.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.282 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.282 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.282 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141080+0000", 2024-09-14T06:16:42.283 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011133029999999999 2024-09-14T06:16:42.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.283 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.283 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.283 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141084+0000", 2024-09-14T06:16:42.283 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.544e-06 2024-09-14T06:16:42.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.283 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.283 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.283 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141088+0000", 2024-09-14T06:16:42.283 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0280000000000002e-06 2024-09-14T06:16:42.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.283 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.283 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.284 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182457+0000", 2024-09-14T06:16:42.284 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041368690999999999 2024-09-14T06:16:42.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.284 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.284 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.284 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182528+0000", 2024-09-14T06:16:42.284 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0881000000000001e-05 2024-09-14T06:16:42.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.284 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.284 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.284 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182610+0000", 2024-09-14T06:16:42.284 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1644000000000006e-05 2024-09-14T06:16:42.284 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.285 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.285 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.285 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.285 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13721 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 532480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.285 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:42.285 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.0788005539999999, 2024-09-14T06:16:42.285 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042577526999999997, 2024-09-14T06:16:42.285 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.285 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.285 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.285 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.285 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13721 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.286 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:42.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2999999999999999e-07 2024-09-14T06:16:42.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.287 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.287 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141268+0000", 2024-09-14T06:16:42.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0130000000000001e-06 2024-09-14T06:16:42.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.287 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.287 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141271+0000", 2024-09-14T06:16:42.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0970000000000002e-06 2024-09-14T06:16:42.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.287 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.288 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141274+0000", 2024-09-14T06:16:42.288 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5809999999999999e-06 2024-09-14T06:16:42.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.288 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.288 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.288 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183522+0000", 2024-09-14T06:16:42.288 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042248591000000002 2024-09-14T06:16:42.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.288 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.288 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.288 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183691+0000", 2024-09-14T06:16:42.288 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016885399999999999 2024-09-14T06:16:42.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183844+0000", 2024-09-14T06:16:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015316099999999999 2024-09-14T06:16:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13722 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 655360~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.0787902520000001, 2024-09-14T06:16:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042741610999999999, 2024-09-14T06:16:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13722 2024-09-14T06:16:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:42.291 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.291 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.291 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.291 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:42.291 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:16:42.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.291 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.291 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.291 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141278+0000", 2024-09-14T06:16:42.291 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.066e-06 2024-09-14T06:16:42.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.291 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.291 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.291 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141282+0000", 2024-09-14T06:16:42.291 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7100000000000001e-06 2024-09-14T06:16:42.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.292 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.292 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.292 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141285+0000", 2024-09-14T06:16:42.292 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.554e-06 2024-09-14T06:16:42.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.292 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.292 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.292 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183824+0000", 2024-09-14T06:16:42.292 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042539043999999998 2024-09-14T06:16:42.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.292 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.292 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.292 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183906+0000", 2024-09-14T06:16:42.293 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2298000000000004e-05 2024-09-14T06:16:42.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.293 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.293 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.293 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184019+0000", 2024-09-14T06:16:42.293 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000112724 2024-09-14T06:16:42.293 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.293 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.293 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.293 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.293 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13723 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 778240~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.293 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:42.294 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.0787792830000003, 2024-09-14T06:16:42.294 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042995901000000003, 2024-09-14T06:16:42.294 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.294 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.294 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.294 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.294 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.294 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.294 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13723 2024-09-14T06:16:42.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.294 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.294 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.294 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.294 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:42.294 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.295 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.295 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.295 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:42.295 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.295 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.295 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.295 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:42.295 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9799999999999999e-07 2024-09-14T06:16:42.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.295 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.295 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.295 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141289+0000", 2024-09-14T06:16:42.295 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0589999999999999e-06 2024-09-14T06:16:42.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.296 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.296 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.296 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141292+0000", 2024-09-14T06:16:42.296 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.208e-06 2024-09-14T06:16:42.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.296 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.296 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.296 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141295+0000", 2024-09-14T06:16:42.296 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5160000000000001e-06 2024-09-14T06:16:42.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.296 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.296 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.296 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184069+0000", 2024-09-14T06:16:42.296 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042774176999999997 2024-09-14T06:16:42.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.297 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.297 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.297 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184169+0000", 2024-09-14T06:16:42.297 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000100073 2024-09-14T06:16:42.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.297 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.297 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.297 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184284+0000", 2024-09-14T06:16:42.297 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011457 2024-09-14T06:16:42.297 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.297 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.297 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.298 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.298 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13724 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 901120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.298 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:16:42.298 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.078768889, 2024-09-14T06:16:42.298 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043197179000000002, 2024-09-14T06:16:42.298 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.298 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.298 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.298 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.298 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.298 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.298 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13724 2024-09-14T06:16:42.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.298 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.298 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.298 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:16:42.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.299 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.299 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:16:42.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.299 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.299 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141299+0000", 2024-09-14T06:16:42.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.03e-07 2024-09-14T06:16:42.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.299 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.299 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141300+0000", 2024-09-14T06:16:42.300 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0389999999999999e-06 2024-09-14T06:16:42.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.300 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.300 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.300 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141303+0000", 2024-09-14T06:16:42.300 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0479999999999999e-06 2024-09-14T06:16:42.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.300 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.300 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.300 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141305+0000", 2024-09-14T06:16:42.300 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5739999999999998e-06 2024-09-14T06:16:42.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.300 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.300 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.301 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184317+0000", 2024-09-14T06:16:42.301 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043011701999999999 2024-09-14T06:16:42.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.301 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.301 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.301 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184394+0000", 2024-09-14T06:16:42.301 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7324999999999996e-05 2024-09-14T06:16:42.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.301 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.301 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.301 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184496+0000", 2024-09-14T06:16:42.301 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010118799999999999 2024-09-14T06:16:42.301 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.301 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.302 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.302 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.302 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13726 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 1024000~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:42.302 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:42.302 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 6.078758616, 2024-09-14T06:16:42.302 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043397185999999997, 2024-09-14T06:16:42.302 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:42.302 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:42.302 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:42.302 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:42.302 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:42.302 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:42.302 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13726 2024-09-14T06:16:42.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.302 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:42.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.303 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:42.303 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:42.303 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.303 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:42.303 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:42.303 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:42.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.303 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:42.303 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:42.303 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:16:42.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.304 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.304 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:42.304 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141310+0000", 2024-09-14T06:16:42.304 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:16:42.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.304 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.304 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:42.304 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141313+0000", 2024-09-14T06:16:42.304 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0309999999999999e-06 2024-09-14T06:16:42.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.304 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.304 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:42.304 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141315+0000", 2024-09-14T06:16:42.304 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.52e-06 2024-09-14T06:16:42.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.305 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:42.305 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184527+0000", 2024-09-14T06:16:42.305 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043211855 2024-09-14T06:16:42.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.305 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:42.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184606+0000", 2024-09-14T06:16:42.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8287000000000004e-05 2024-09-14T06:16:42.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:42.308 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:42.308 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:42.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184706+0000", 2024-09-14T06:16:42.309 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010021700000000001 2024-09-14T06:16:42.309 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.309 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.309 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.309 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:42.309 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:42.309 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:42.311 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:16:42.311 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-14T06:16:42.311 INFO:teuthology.orchestra.run.smithi039.stdout:1183 read 0x152fc30 thru 0x1538c47 (0x9018 bytes) 2024-09-14T06:16:42.311 INFO:teuthology.orchestra.run.smithi039.stdout:1185 trunc from 0xbaaa80c to 0x68d0bc3 2024-09-14T06:16:42.312 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:42.311+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:42.325 INFO:teuthology.orchestra.run.smithi039.stdout:1188 writesame 0x5ff12e4 thru 0x5ff9e37 data_size 0x4a(0x8b54 bytes) 2024-09-14T06:16:42.328 INFO:teuthology.orchestra.run.smithi039.stdout:1189 punch from 0x4279e9f to 0x427c6e1, (0x2842 bytes) 2024-09-14T06:16:42.328 INFO:teuthology.orchestra.run.smithi039.stdout:1190 read 0x20e77bb thru 0x20f21d1 (0xaa17 bytes) 2024-09-14T06:16:42.330 INFO:teuthology.orchestra.run.smithi039.stdout:1194 write 0x57994f7 thru 0x57a75b8 (0xe0c2 bytes) 2024-09-14T06:16:42.333 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:42.334 INFO:teuthology.orchestra.run.smithi039.stdout:1197 writesame 0x66cb850 thru 0x66db205 data_size 0x1(0xf9b6 bytes) 2024-09-14T06:16:42.395 INFO:teuthology.orchestra.run.smithi039.stdout:1198 read 0x5238c6f thru 0x5242f61 (0xa2f3 bytes) 2024-09-14T06:16:42.397 INFO:teuthology.orchestra.run.smithi039.stdout:1200 read 0x3ddf64d thru 0x3deedd4 (0xf788 bytes) 2024-09-14T06:16:42.411 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:42.407+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:42.424 INFO:teuthology.orchestra.run.smithi039.stdout:1201 write 0x6b80d22 thru 0x6b89a6e (0x8d4d bytes) 2024-09-14T06:16:42.442 INFO:teuthology.orchestra.run.smithi039.stdout:1204 writesame 0x728ea4f thru 0x7296bc9 data_size 0x57(0x817b bytes) 2024-09-14T06:16:42.447 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:42.448 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:42.448 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:16:42.448 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:42.454 INFO:teuthology.orchestra.run.smithi039.stdout:1205 write 0x731fd9d thru 0x73259f2 (0x5c56 bytes) 2024-09-14T06:16:42.465 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:16:42.476 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:16:42.487 INFO:teuthology.orchestra.run.smithi039.stdout:1206 write 0xbb7c8c1 thru 0xbb821de (0x591e bytes) 2024-09-14T06:16:42.494 INFO:teuthology.orchestra.run.smithi039.stdout:1207 writesame 0x3c05ad5 thru 0x3c0ea5a data_size 0x1(0x8f86 bytes) 2024-09-14T06:16:42.511 INFO:teuthology.orchestra.run.smithi039.stdout:1209 write 0x509138b thru 0x509149f (0x115 bytes) 2024-09-14T06:16:42.512 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:42.507+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:42.514 INFO:teuthology.orchestra.run.smithi039.stdout:1210 read 0x6da49f4 thru 0x6da6736 (0x1d43 bytes) 2024-09-14T06:16:42.515 INFO:teuthology.orchestra.run.smithi039.stdout:1211 write 0x4a6dde7 thru 0x4a7db94 (0xfdae bytes) 2024-09-14T06:16:42.522 INFO:teuthology.orchestra.run.smithi039.stdout:1213 read 0x87c555a thru 0x87ca134 (0x4bdb bytes) 2024-09-14T06:16:42.539 INFO:teuthology.orchestra.run.smithi039.stdout:1214 trunc from 0xbb821df to 0x4adb3de 2024-09-14T06:16:42.569 INFO:teuthology.orchestra.run.smithi039.stdout:1215 punch from 0x1d8dc7d to 0x1d905b0, (0x2933 bytes) 2024-09-14T06:16:42.569 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:42.588 INFO:teuthology.orchestra.run.smithi039.stdout:1219 write 0xc7a3757 thru 0xc7abeb2 (0x875c bytes) 2024-09-14T06:16:42.593 INFO:teuthology.orchestra.run.smithi039.stdout:1220 writesame 0xb4e938e thru 0xb4f05b8 data_size 0x1(0x722b bytes) 2024-09-14T06:16:42.595 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:42.595 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:42.596 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:16:42.596 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:16:42.596 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:42.610 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:16:42.612 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:42.607+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:42.620 INFO:teuthology.orchestra.run.smithi039.stdout:1221 punch from 0x92c9d37 to 0x92d709b, (0xd364 bytes) 2024-09-14T06:16:42.620 INFO:teuthology.orchestra.run.smithi039.stdout:1222 trunc from 0xc7abeb3 to 0xc57eda6 2024-09-14T06:16:42.630 INFO:teuthology.orchestra.run.smithi039.stdout:1223 write 0x387e3a1 thru 0x388e092 (0xfcf2 bytes) 2024-09-14T06:16:42.632 INFO:teuthology.orchestra.run.smithi039.stdout:1224 read 0x7549f6c thru 0x754c22e (0x22c3 bytes) 2024-09-14T06:16:42.632 INFO:teuthology.orchestra.run.smithi039.stdout:1226 write 0x56a497 thru 0x56a924 (0x48e bytes) 2024-09-14T06:16:42.633 INFO:teuthology.orchestra.run.smithi039.stdout:1229 punch from 0x3a30ba6 to 0x3a3740b, (0x6865 bytes) 2024-09-14T06:16:42.634 INFO:teuthology.orchestra.run.smithi039.stdout:1230 read 0x17c546f thru 0x17ca711 (0x52a3 bytes) 2024-09-14T06:16:42.634 INFO:teuthology.orchestra.run.smithi039.stdout:1232 write 0xbe1fc56 thru 0xbe26421 (0x67cc bytes) 2024-09-14T06:16:42.638 INFO:teuthology.orchestra.run.smithi039.stdout:1234 read 0x2578e78 thru 0x2581fec (0x9175 bytes) 2024-09-14T06:16:42.639 INFO:teuthology.orchestra.run.smithi039.stdout:1235 write 0x435822 thru 0x436ac8 (0x12a7 bytes) 2024-09-14T06:16:42.650 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:42.650 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:42.663 INFO:teuthology.orchestra.run.smithi039.stdout:1236 trunc from 0xc57eda6 to 0xaa7f050 2024-09-14T06:16:42.673 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:42.673 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:42.675 INFO:teuthology.orchestra.run.smithi039.stdout:1237 write 0x7217b34 thru 0x722711d (0xf5ea bytes) 2024-09-14T06:16:42.690 INFO:teuthology.orchestra.run.smithi039.stdout:1240 write 0xe588592 thru 0xe58f3ba (0x6e29 bytes) 2024-09-14T06:16:42.691 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:42.691 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:42.694 INFO:teuthology.orchestra.run.smithi039.stdout:1242 trunc from 0xe58f3bb to 0xe7ab661 2024-09-14T06:16:42.700 INFO:teuthology.orchestra.run.smithi039.stdout:1247 trunc from 0xe7ab661 to 0xe558c1f 2024-09-14T06:16:42.707 INFO:teuthology.orchestra.run.smithi039.stdout:1248 write 0x660f5be thru 0x6611b33 (0x2576 bytes) 2024-09-14T06:16:42.711 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:42.711 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:42.711 INFO:teuthology.orchestra.run.smithi039.stdout:1249 punch from 0x9a807d4 to 0x9a8e367, (0xdb93 bytes) 2024-09-14T06:16:42.712 INFO:teuthology.orchestra.run.smithi039.stdout:1250 read 0xa24d6e7 thru 0xa251094 (0x39ae bytes) 2024-09-14T06:16:42.712 INFO:teuthology.orchestra.run.smithi039.stdout:1251 write 0x6f1a3e7 thru 0x6f1d09f (0x2cb9 bytes) 2024-09-14T06:16:42.713 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:42.711+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:42.721 INFO:teuthology.orchestra.run.smithi039.stdout:1252 write 0xe6845b1 thru 0xe68f2c0 (0xad10 bytes) 2024-09-14T06:16:42.725 INFO:teuthology.orchestra.run.smithi039.stdout:1253 read 0x11cc72 thru 0x1259d2 (0x8d61 bytes) 2024-09-14T06:16:42.726 INFO:teuthology.orchestra.run.smithi039.stdout:1254 punch from 0x77f9aa7 to 0x7806959, (0xceb2 bytes) 2024-09-14T06:16:42.726 INFO:teuthology.orchestra.run.smithi039.stdout:1256 read 0xa124254 thru 0xa12f5f0 (0xb39d bytes) 2024-09-14T06:16:42.726 INFO:teuthology.orchestra.run.smithi039.stdout:1257 read 0x1803b3b thru 0x180abf6 (0x70bc bytes) 2024-09-14T06:16:42.728 INFO:teuthology.orchestra.run.smithi039.stdout:1258 trunc from 0xe68f2c1 to 0xd397428 2024-09-14T06:16:42.731 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:42.731 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:42.737 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:42.737 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:42.750 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:42.750 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:42.750 INFO:teuthology.orchestra.run.smithi039.stdout:1260 writesame 0xaaab04d thru 0xaaab7b6 data_size 0x2(0x76a bytes) 2024-09-14T06:16:42.753 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:16:42.754 INFO:teuthology.orchestra.run.smithi039.stdout:1264 trunc from 0xd397428 to 0xd44863c 2024-09-14T06:16:42.759 INFO:teuthology.orchestra.run.smithi039.stdout:1267 read 0x4061c62 thru 0x40632cb (0x166a bytes) 2024-09-14T06:16:42.759 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:42.759 INFO:teuthology.orchestra.run.smithi039.stdout:1269 read 0x9172235 thru 0x91820d9 (0xfea5 bytes) 2024-09-14T06:16:42.760 INFO:teuthology.orchestra.run.smithi039.stdout:1270 write 0xa0fbce8 thru 0xa1035a9 (0x78c2 bytes) 2024-09-14T06:16:42.763 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:42.763 INFO:teuthology.orchestra.run.smithi039.stdout:1272 read 0x41a81f6 thru 0x41b58a0 (0xd6ab bytes) 2024-09-14T06:16:42.764 INFO:teuthology.orchestra.run.smithi039.stdout:1273 read 0xacfad59 thru 0xad04023 (0x92cb bytes) 2024-09-14T06:16:42.764 INFO:teuthology.orchestra.run.smithi039.stdout:1274 read 0x73774ce thru 0x7380fba (0x9aed bytes) 2024-09-14T06:16:42.764 INFO:teuthology.orchestra.run.smithi039.stdout:1276 read 0x3dd090d thru 0x3dd7454 (0x6b48 bytes) 2024-09-14T06:16:42.764 INFO:teuthology.orchestra.run.smithi039.stdout:1277 read 0x9310a2b thru 0x93128c8 (0x1e9e bytes) 2024-09-14T06:16:42.765 INFO:teuthology.orchestra.run.smithi039.stdout:1278 write 0x891dc54 thru 0x8921183 (0x3530 bytes) 2024-09-14T06:16:42.774 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:42.774 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:42.776 INFO:teuthology.orchestra.run.smithi039.stdout:1279 write 0xee575bd thru 0xee5ed52 (0x7796 bytes) 2024-09-14T06:16:42.780 INFO:teuthology.orchestra.run.smithi039.stdout:1280 punch from 0x31c32e5 to 0x31d2c93, (0xf9ae bytes) 2024-09-14T06:16:42.781 INFO:teuthology.orchestra.run.smithi039.stdout:1282 trunc from 0xee5ed53 to 0xdd36729 2024-09-14T06:16:42.792 INFO:teuthology.orchestra.run.smithi039.stdout:1283 read 0x4890a56 thru 0x48963ca (0x5975 bytes) 2024-09-14T06:16:42.793 INFO:teuthology.orchestra.run.smithi039.stdout:1286 write 0x1785fac thru 0x1793d15 (0xdd6a bytes) 2024-09-14T06:16:42.795 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:42.795 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:42.806 INFO:teuthology.orchestra.run.smithi039.stdout:1287 punch from 0x78f4bbf to 0x78f4eac, (0x2ed bytes) 2024-09-14T06:16:42.806 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:42.814 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:42.807+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:42.814 INFO:teuthology.orchestra.run.smithi039.stdout:1289 trunc from 0xdd36729 to 0x8b056fe 2024-09-14T06:16:42.826 INFO:teuthology.orchestra.run.smithi039.stdout:1290 punch from 0x1144bbe to 0x1150794, (0xbbd6 bytes) 2024-09-14T06:16:42.826 INFO:teuthology.orchestra.run.smithi039.stdout:1293 read 0x53c88c thru 0x541822 (0x4f97 bytes) 2024-09-14T06:16:42.845 INFO:teuthology.orchestra.run.smithi039.stdout:1294 write 0xbb739a9 thru 0xbb7c84e (0x8ea6 bytes) 2024-09-14T06:16:42.849 INFO:teuthology.orchestra.run.smithi039.stdout:1297 read 0xb9a9323 thru 0xb9b6304 (0xcfe2 bytes) 2024-09-14T06:16:42.850 INFO:teuthology.orchestra.run.smithi039.stdout:1299 write 0x82a01e4 thru 0x82a036e (0x18b bytes) 2024-09-14T06:16:42.853 INFO:teuthology.orchestra.run.smithi039.stdout:1300 read 0x36d42c1 thru 0x36d980b (0x554b bytes) 2024-09-14T06:16:42.859 INFO:teuthology.orchestra.run.smithi039.stdout:1301 write 0xbfcff91 thru 0xbfd2ce3 (0x2d53 bytes) 2024-09-14T06:16:42.863 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:42.863 INFO:teuthology.orchestra.run.smithi039.stdout:1303 write 0x5ad205c thru 0x5add6c5 (0xb66a bytes) 2024-09-14T06:16:42.866 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:42.866 INFO:teuthology.orchestra.run.smithi039.stdout:1305 read 0x1fdd7f3 thru 0x1fdf9ae (0x21bc bytes) 2024-09-14T06:16:42.867 INFO:teuthology.orchestra.run.smithi039.stdout:1306 punch from 0xb69c182 to 0xb6a3e91, (0x7d0f bytes) 2024-09-14T06:16:42.867 INFO:teuthology.orchestra.run.smithi039.stdout:1307 read 0x4efcd3f thru 0x4efe4d1 (0x1793 bytes) 2024-09-14T06:16:42.885 INFO:teuthology.orchestra.run.smithi039.stdout:1308 trunc from 0xbfd2ce4 to 0x6ca288 2024-09-14T06:16:42.915 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:42.911+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:42.924 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:42.924 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:42.925 INFO:teuthology.orchestra.run.smithi039.stdout:1309 trunc from 0x6ca288 to 0x97ad4e6 2024-09-14T06:16:42.935 INFO:teuthology.orchestra.run.smithi039.stdout:1311 read 0x5ad88f thru 0x5af955 (0x20c7 bytes) 2024-09-14T06:16:42.936 INFO:teuthology.orchestra.run.smithi039.stdout:1312 punch from 0x33f5f93 to 0x3401f0f, (0xbf7c bytes) 2024-09-14T06:16:42.938 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:16:42.946 INFO:teuthology.orchestra.run.smithi039.stdout:1313 write 0xa62d993 thru 0xa63645b (0x8ac9 bytes) 2024-09-14T06:16:42.950 INFO:teuthology.orchestra.run.smithi039.stdout:1314 punch from 0xa3c3039 to 0xa3cc484, (0x944b bytes) 2024-09-14T06:16:42.950 INFO:teuthology.orchestra.run.smithi039.stdout:1315 writesame 0x26fa99a thru 0x26fdcee data_size 0x1(0x3355 bytes) 2024-09-14T06:16:42.960 INFO:teuthology.orchestra.run.smithi039.stdout:1316 write 0x5778b8c thru 0x577b1f9 (0x266e bytes) 2024-09-14T06:16:42.966 INFO:teuthology.orchestra.run.smithi039.stdout:1317 punch from 0x372d925 to 0x37338e9, (0x5fc4 bytes) 2024-09-14T06:16:42.966 INFO:teuthology.orchestra.run.smithi039.stdout:1318 read 0x877128b thru 0x877c21a (0xaf90 bytes) 2024-09-14T06:16:42.966 INFO:teuthology.orchestra.run.smithi039.stdout:1320 write 0x2cbee4a thru 0x2cc2fc3 (0x417a bytes) 2024-09-14T06:16:42.985 INFO:teuthology.orchestra.run.smithi039.stdout:1321 write 0xc407e0d thru 0xc414c28 (0xce1c bytes) 2024-09-14T06:16:42.992 INFO:teuthology.orchestra.run.smithi039.stdout:1323 writesame 0x42892b3 thru 0x4292265 data_size 0x1(0x8fb3 bytes) 2024-09-14T06:16:43.016 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:43.015+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:43.044 INFO:teuthology.orchestra.run.smithi039.stdout:1324 punch from 0x37b3a06 to 0x37b9e1e, (0x6418 bytes) 2024-09-14T06:16:43.044 INFO:teuthology.orchestra.run.smithi039.stdout:1325 read 0x7987d47 thru 0x7990ad4 (0x8d8e bytes) 2024-09-14T06:16:43.044 INFO:teuthology.orchestra.run.smithi039.stdout:1326 writesame 0xa71c78e thru 0xa72b205 data_size 0xa4(0xea78 bytes) 2024-09-14T06:16:43.052 INFO:teuthology.orchestra.run.smithi039.stdout:1328 read 0x4799e92 thru 0x47a254d (0x86bc bytes) 2024-09-14T06:16:43.063 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:43.063 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:43.077 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:16:43.084 INFO:teuthology.orchestra.run.smithi039.stdout:1330 trunc from 0xc414c29 to 0x3f0de2 2024-09-14T06:16:43.113 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:43.117 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:43.111+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:43.130 INFO:teuthology.orchestra.run.smithi039.stdout:1332 trunc from 0x3f0de2 to 0x6af8d93 2024-09-14T06:16:43.142 INFO:teuthology.orchestra.run.smithi039.stdout:1333 read 0x168f21b thru 0x169b68d (0xc473 bytes) 2024-09-14T06:16:43.159 INFO:teuthology.orchestra.run.smithi039.stdout:1334 write 0x81f0450 thru 0x81fdfaf (0xdb60 bytes) 2024-09-14T06:16:43.165 INFO:teuthology.orchestra.run.smithi039.stdout:1338 punch from 0x52022d7 to 0x520c080, (0x9da9 bytes) 2024-09-14T06:16:43.174 INFO:teuthology.orchestra.run.smithi039.stdout:1340 trunc from 0x81fdfb0 to 0x4cee93d 2024-09-14T06:16:43.194 INFO:teuthology.orchestra.run.smithi039.stdout:1341 punch from 0x47fd56 to 0x48c6c3, (0xc96d bytes) 2024-09-14T06:16:43.194 INFO:teuthology.orchestra.run.smithi039.stdout:1342 punch from 0x46394be to 0x463dfe2, (0x4b24 bytes) 2024-09-14T06:16:43.211 INFO:teuthology.orchestra.run.smithi039.stdout:1343 write 0x6898422 thru 0x689cac8 (0x46a7 bytes) 2024-09-14T06:16:43.217 INFO:teuthology.orchestra.run.smithi039.stdout:1344 punch from 0x495d33 to 0x49fef6, (0xa1c3 bytes) 2024-09-14T06:16:43.218 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:43.215+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:43.247 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:43.247 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:43.252 INFO:teuthology.orchestra.run.smithi039.stdout:1345 write 0xea9fba1 thru 0xeaabe90 (0xc2f0 bytes) 2024-09-14T06:16:43.260 INFO:teuthology.orchestra.run.smithi039.stdout:1346 write 0x4f9385a thru 0x4f95671 (0x1e18 bytes) 2024-09-14T06:16:43.262 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:16:43.265 INFO:teuthology.orchestra.run.smithi039.stdout:1348 read 0x9a0b26b thru 0x9a1543d (0xa1d3 bytes) 2024-09-14T06:16:43.265 INFO:teuthology.orchestra.run.smithi039.stdout:1349 punch from 0xc361548 to 0xc36d012, (0xbaca bytes) 2024-09-14T06:16:43.266 INFO:teuthology.orchestra.run.smithi039.stdout:1352 read 0x86bc626 thru 0x86c8aed (0xc4c8 bytes) 2024-09-14T06:16:43.289 INFO:teuthology.orchestra.run.smithi039.stdout:1353 trunc from 0xeaabe91 to 0x648047e 2024-09-14T06:16:43.311 INFO:teuthology.orchestra.run.smithi039.stdout:1355 write 0xa710a1 thru 0xa7780d (0x676d bytes) 2024-09-14T06:16:43.319 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:43.319+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:43.320 INFO:teuthology.orchestra.run.smithi039.stdout:1357 trunc from 0x648047e to 0x89c9b61 2024-09-14T06:16:43.333 INFO:teuthology.orchestra.run.smithi039.stdout:1360 read 0x143748f thru 0x14425dd (0xb14f bytes) 2024-09-14T06:16:43.334 INFO:teuthology.orchestra.run.smithi039.stdout:1361 read 0x2e06f6d thru 0x2e0c107 (0x519b bytes) 2024-09-14T06:16:43.350 INFO:teuthology.orchestra.run.smithi039.stdout:1362 writesame 0x9d66fa6 thru 0x9d74a00 data_size 0x3(0xda5b bytes) 2024-09-14T06:16:43.350 INFO:teuthology.orchestra.run.smithi039.stdout:1363 read 0x949cc3d thru 0x94a14a4 (0x4868 bytes) 2024-09-14T06:16:43.368 INFO:teuthology.orchestra.run.smithi039.stdout:1364 write 0xbe9fe6e thru 0xbea6f7c (0x710f bytes) 2024-09-14T06:16:43.375 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:43.375 INFO:teuthology.orchestra.run.smithi039.stdout:1366 write 0x1e1dd6e thru 0x1e216ae (0x3941 bytes) 2024-09-14T06:16:43.383 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:43.383 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:43.397 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:16:43.403 INFO:teuthology.orchestra.run.smithi039.stdout:1368 writesame 0xed3cbce thru 0xed4ca0e data_size 0x1(0xfe41 bytes) 2024-09-14T06:16:43.403 INFO:teuthology.orchestra.run.smithi039.stdout:1369 write 0xb59399e thru 0xb5a184b (0xdeae bytes) 2024-09-14T06:16:43.436 INFO:teuthology.orchestra.run.smithi039.stdout:1370 trunc from 0xed4ca0f to 0x55bfb76 2024-09-14T06:16:43.446 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:43.447+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:43.460 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:43.460 INFO:teuthology.orchestra.run.smithi039.stdout:1372 read 0x4d39fac thru 0x4d40da2 (0x6df7 bytes) 2024-09-14T06:16:43.460 INFO:teuthology.orchestra.run.smithi039.stdout:1373 punch from 0x3c22b65 to 0x3c3087b, (0xdd16 bytes) 2024-09-14T06:16:43.475 INFO:teuthology.orchestra.run.smithi039.stdout:1375 trunc from 0x55bfb76 to 0xab859e8 2024-09-14T06:16:43.487 INFO:teuthology.orchestra.run.smithi039.stdout:1377 read 0x4ba466d thru 0x4bab1a7 (0x6b3b bytes) 2024-09-14T06:16:43.487 INFO:teuthology.orchestra.run.smithi039.stdout:1378 write 0xa63dd6d thru 0xa647b43 (0x9dd7 bytes) 2024-09-14T06:16:43.502 INFO:teuthology.orchestra.run.smithi039.stdout:1381 trunc from 0xab859e8 to 0x84c97e2 2024-09-14T06:16:43.520 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:43.519+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:43.521 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:43.522 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:43.524 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:43.524 INFO:teuthology.orchestra.run.smithi039.stdout:1383 writesame 0x455799d thru 0x455c727 data_size 0xd(0x4d8b bytes) 2024-09-14T06:16:43.537 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:16:43.626 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:43.627+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:43.705 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:43.705 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:43.719 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:16:43.723 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:43.723+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:43.751 INFO:teuthology.orchestra.run.smithi039.stdout:1384 write 0x8ec695f thru 0x8ed007b (0x971d bytes) 2024-09-14T06:16:43.757 INFO:teuthology.orchestra.run.smithi039.stdout:1386 write 0x827285b thru 0x8281e42 (0xf5e8 bytes) 2024-09-14T06:16:43.765 INFO:teuthology.orchestra.run.smithi039.stdout:1390 read 0x6a620a2 thru 0x6a66f9f (0x4efe bytes) 2024-09-14T06:16:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:1392 writesame 0xd3cf27a thru 0xd3dc877 data_size 0x31(0xd5fe bytes) 2024-09-14T06:16:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:1393 read 0xc3730f2 thru 0xc373bdc (0xaeb bytes) 2024-09-14T06:16:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:1395 write 0x3521deb thru 0x352c1c1 (0xa3d7 bytes) 2024-09-14T06:16:43.812 INFO:teuthology.orchestra.run.smithi039.stdout:1396 trunc from 0xd3dc878 to 0x7bcc7a3 2024-09-14T06:16:43.823 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:43.823+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:43.837 INFO:teuthology.orchestra.run.smithi039.stdout:1397 writesame 0x25f5ab6 thru 0x2602649 data_size 0x81(0xcb94 bytes) 2024-09-14T06:16:43.847 INFO:teuthology.orchestra.run.smithi039.stdout:1398 punch from 0xf4c06f to 0xf573a3, (0xb334 bytes) 2024-09-14T06:16:43.847 INFO:teuthology.orchestra.run.smithi039.stdout:1399 writesame 0x33cb686 thru 0x33cf2de data_size 0x1(0x3c59 bytes) 2024-09-14T06:16:43.864 INFO:teuthology.orchestra.run.smithi039.stdout:1400 flatten 2024-09-14T06:16:43.891 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:43.891 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:43.892 INFO:teuthology.orchestra.run.smithi039.stdout:1401 write 0x7c841b2 thru 0x7c88171 (0x3fc0 bytes) 2024-09-14T06:16:43.900 INFO:teuthology.orchestra.run.smithi039.stdout:1402 read 0x53ff6c7 thru 0x5408c0b (0x9545 bytes) 2024-09-14T06:16:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:1403 punch from 0x661eb92 to 0x6621f20, (0x338e bytes) 2024-09-14T06:16:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:1405 read 0x40d88fb thru 0x40daccf (0x23d5 bytes) 2024-09-14T06:16:43.906 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:16:43.916 INFO:teuthology.orchestra.run.smithi039.stdout:1409 write 0x87d51c4 thru 0x87e2692 (0xd4cf bytes) 2024-09-14T06:16:43.918 INFO:teuthology.orchestra.run.smithi039.stdout:1412 punch from 0x430faaa to 0x431c7e4, (0xcd3a bytes) 2024-09-14T06:16:43.919 INFO:teuthology.orchestra.run.smithi039.stdout:1413 writesame 0x408df3f thru 0x4099203 data_size 0x2d(0xb2c5 bytes) 2024-09-14T06:16:43.924 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:43.923+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:43.944 INFO:teuthology.orchestra.run.smithi039.stdout:1414 writesame 0xa04d260 thru 0xa0535ad data_size 0x39(0x634e bytes) 2024-09-14T06:16:43.944 INFO:teuthology.orchestra.run.smithi039.stdout:1415 write 0xfd6d3a thru 0xfd9dae (0x3075 bytes) 2024-09-14T06:16:43.945 INFO:teuthology.orchestra.run.smithi039.stdout:1416 write 0x257a131 thru 0x2586f2f (0xcdff bytes) 2024-09-14T06:16:43.951 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:43.951 INFO:teuthology.orchestra.run.smithi039.stdout:1418 writesame 0x6dce1 thru 0x6ecfe data_size 0x1(0x101e bytes) 2024-09-14T06:16:43.957 INFO:teuthology.orchestra.run.smithi039.stdout:1419 punch from 0x5e9ccec to 0x5ea418f, (0x74a3 bytes) 2024-09-14T06:16:43.971 INFO:teuthology.orchestra.run.smithi039.stdout:1421 trunc from 0xa0535ae to 0x5056b2a 2024-09-14T06:16:43.978 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:16:43.978 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:16:43.978 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-14T06:16:43.979 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-14T06:16:43.994 INFO:teuthology.orchestra.run.smithi039.stdout:1422 write 0x41e336d thru 0x41e67b7 (0x344b bytes) 2024-09-14T06:16:43.996 INFO:teuthology.orchestra.run.smithi039.stdout:1423 read 0x16c8703 thru 0x16cc803 (0x4101 bytes) 2024-09-14T06:16:44.013 INFO:teuthology.orchestra.run.smithi039.stdout:1424 writesame 0x7a9d56b thru 0x7aa8202 data_size 0x54(0xac98 bytes) 2024-09-14T06:16:44.013 INFO:teuthology.orchestra.run.smithi039.stdout:1425 writesame 0x850577 thru 0x852fc5 data_size 0x1(0x2a4f bytes) 2024-09-14T06:16:44.016 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:16:44.031 INFO:teuthology.orchestra.run.smithi039.stdout:1426 trunc from 0x7aa8203 to 0x223474e 2024-09-14T06:16:44.031 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:44.031+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:44.058 INFO:teuthology.orchestra.run.smithi039.stdout:1429 trunc from 0x223474e to 0x54e1b52 2024-09-14T06:16:44.080 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:44.080 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:44.081 INFO:teuthology.orchestra.run.smithi039.stdout:1430 writesame 0x939210d thru 0x93974ec data_size 0x3d(0x53e0 bytes) 2024-09-14T06:16:44.094 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:16:44.097 INFO:teuthology.orchestra.run.smithi039.stdout:1431 writesame 0xc81201f thru 0xc81f6a2 data_size 0x2(0xd684 bytes) 2024-09-14T06:16:44.097 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.097 INFO:teuthology.orchestra.run.smithi039.stdout:1434 read 0x5a19350 thru 0x5a1b790 (0x2441 bytes) 2024-09-14T06:16:44.109 INFO:teuthology.orchestra.run.smithi039.stdout:1435 trunc from 0xc81f6a3 to 0x47754ca 2024-09-14T06:16:44.117 INFO:teuthology.orchestra.run.smithi039.stdout:1436 read 0x37c66b2 thru 0x37d2f40 (0xc88f bytes) 2024-09-14T06:16:44.117 INFO:teuthology.orchestra.run.smithi039.stdout:1438 read 0x31c020d thru 0x31c81b4 (0x7fa8 bytes) 2024-09-14T06:16:44.117 INFO:teuthology.orchestra.run.smithi039.stdout:1439 write 0xc996d8 thru 0xc9f261 (0x5b8a bytes) 2024-09-14T06:16:44.119 INFO:teuthology.orchestra.run.smithi039.stdout:1441 write 0x47532b thru 0x4797b1 (0x4487 bytes) 2024-09-14T06:16:44.138 INFO:teuthology.orchestra.run.smithi039.stdout:1442 write 0xb41a397 thru 0xb41d727 (0x3391 bytes) 2024-09-14T06:16:44.142 INFO:teuthology.orchestra.run.smithi039.stdout:1443 write 0x611a926 thru 0x611f27b (0x4956 bytes) 2024-09-14T06:16:44.146 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:44.147+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:44.146 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.146 INFO:teuthology.orchestra.run.smithi039.stdout:1445 writesame 0x6adf2f5 thru 0x6ae0667 data_size 0x1(0x1373 bytes) 2024-09-14T06:16:44.152 INFO:teuthology.orchestra.run.smithi039.stdout:1446 punch from 0x241509e to 0x241a12c, (0x508e bytes) 2024-09-14T06:16:44.153 INFO:teuthology.orchestra.run.smithi039.stdout:1450 trunc from 0xb41d728 to 0xda82a86 2024-09-14T06:16:44.160 INFO:teuthology.orchestra.run.smithi039.stdout:1452 write 0xb44f5d thru 0xb52249 (0xd2ed bytes) 2024-09-14T06:16:44.163 INFO:teuthology.orchestra.run.smithi039.stdout:1453 punch from 0x1a02e0d to 0x1a0d7a1, (0xa994 bytes) 2024-09-14T06:16:44.163 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.163 INFO:teuthology.orchestra.run.smithi039.stdout:1455 writesame 0x1e61f98 thru 0x1e712eb data_size 0x57(0xf354 bytes) 2024-09-14T06:16:44.165 INFO:teuthology.orchestra.run.smithi039.stdout:1456 read 0x53c383b thru 0x53cb4f7 (0x7cbd bytes) 2024-09-14T06:16:44.168 INFO:teuthology.orchestra.run.smithi039.stdout:1457 trunc from 0xda82a86 to 0xb407882 2024-09-14T06:16:44.170 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-14T06:16:44.170 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-14T06:16:44.170 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-14T06:16:44.170 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-14T06:16:44.170 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-14T06:16:44.170 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-14T06:16:44.170 INFO:teuthology.orchestra.run.smithi039.stderr:bench [] [] [] [] 2024-09-14T06:16:44.170 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:16:44.170 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-14T06:16:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-14T06:16:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-14T06:16:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2024-09-14T06:16:44.173 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:16:44.173 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:16:44.174 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-14T06:16:44.177 INFO:teuthology.orchestra.run.smithi039.stdout:1458 read 0x586d3e7 thru 0x587aa37 (0xd651 bytes) 2024-09-14T06:16:44.177 INFO:teuthology.orchestra.run.smithi039.stdout:1459 punch from 0x346bf7e to 0x347b12e, (0xf1b0 bytes) 2024-09-14T06:16:44.178 INFO:teuthology.orchestra.run.smithi039.stdout:1460 write 0x5f15917 thru 0x5f1f580 (0x9c6a bytes) 2024-09-14T06:16:44.182 INFO:teuthology.orchestra.run.smithi039.stdout:1462 punch from 0x4737014 to 0x473d494, (0x6480 bytes) 2024-09-14T06:16:44.182 INFO:teuthology.orchestra.run.smithi039.stdout:1463 read 0x68332d6 thru 0x68333d3 (0xfe bytes) 2024-09-14T06:16:44.183 INFO:teuthology.orchestra.run.smithi039.stdout:1464 punch from 0x651d8a to 0x65683a, (0x4ab0 bytes) 2024-09-14T06:16:44.191 INFO:teuthology.orchestra.run.smithi039.stdout:1465 trunc from 0xb407882 to 0x5e54025 2024-09-14T06:16:44.224 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:44.225 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:44.226 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:44.223+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:44.228 INFO:teuthology.orchestra.run.smithi039.stdout:1466 write 0xd8eb8e7 thru 0xd8f2e74 (0x758e bytes) 2024-09-14T06:16:44.233 INFO:teuthology.orchestra.run.smithi039.stdout:1467 write 0x50aa2a8 thru 0x50b6862 (0xc5bb bytes) 2024-09-14T06:16:44.237 INFO:teuthology.orchestra.run.smithi039.stdout:1472 writesame 0x2f114b6 thru 0x2f1a912 data_size 0x1(0x945d bytes) 2024-09-14T06:16:44.240 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:16:44.262 INFO:teuthology.orchestra.run.smithi039.stdout:1473 punch from 0x1d2754a to 0x1d3494f, (0xd405 bytes) 2024-09-14T06:16:44.262 INFO:teuthology.orchestra.run.smithi039.stdout:1474 write 0x7503816 thru 0x7506fe6 (0x37d1 bytes) 2024-09-14T06:16:44.267 INFO:teuthology.orchestra.run.smithi039.stdout:1475 read 0xb9770dd thru 0xb97c48b (0x53af bytes) 2024-09-14T06:16:44.267 INFO:teuthology.orchestra.run.smithi039.stdout:1479 punch from 0xccd13b5 to 0xccd53ca, (0x4015 bytes) 2024-09-14T06:16:44.267 INFO:teuthology.orchestra.run.smithi039.stdout:1480 read 0x124ded4 thru 0x1253b0f (0x5c3c bytes) 2024-09-14T06:16:44.268 INFO:teuthology.orchestra.run.smithi039.stdout:1481 read 0x92aa3b7 thru 0x92b6c59 (0xc8a3 bytes) 2024-09-14T06:16:44.268 INFO:teuthology.orchestra.run.smithi039.stdout:1482 read 0x9de41f8 thru 0x9deb62a (0x7433 bytes) 2024-09-14T06:16:44.268 INFO:teuthology.orchestra.run.smithi039.stdout:1484 write 0x6cca85c thru 0x6cd7011 (0xc7b6 bytes) 2024-09-14T06:16:44.273 INFO:teuthology.orchestra.run.smithi039.stdout:1485 punch from 0x36efc30 to 0x36f4b0c, (0x4edc bytes) 2024-09-14T06:16:44.281 INFO:teuthology.orchestra.run.smithi039.stdout:1488 trunc from 0xd8f2e75 to 0x88cd58e 2024-09-14T06:16:44.297 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.306 INFO:teuthology.orchestra.run.smithi039.stdout:1490 writesame 0x9b428e3 thru 0x9b4b143 data_size 0x1(0x8861 bytes) 2024-09-14T06:16:44.307 INFO:teuthology.orchestra.run.smithi039.stdout:1491 read 0x6cb1db7 thru 0x6cc1cc5 (0xff0f bytes) 2024-09-14T06:16:44.307 INFO:teuthology.orchestra.run.smithi039.stdout:1493 read 0x26bbf74 thru 0x26bd487 (0x1514 bytes) 2024-09-14T06:16:44.308 INFO:teuthology.orchestra.run.smithi039.stdout:1495 punch from 0x32cc966 to 0x32d1ed0, (0x556a bytes) 2024-09-14T06:16:44.308 INFO:teuthology.orchestra.run.smithi039.stdout:1496 read 0x3d7fa5a thru 0x3d8663b (0x6be2 bytes) 2024-09-14T06:16:44.309 INFO:teuthology.orchestra.run.smithi039.stdout:1498 trunc from 0x9b4b144 to 0xa5c621d 2024-09-14T06:16:44.317 INFO:teuthology.orchestra.run.smithi039.stdout:1499 read 0x30a4688 thru 0x30ae05a (0x99d3 bytes) 2024-09-14T06:16:44.318 INFO:teuthology.orchestra.run.smithi039.stdout:1500 read 0x7ef8de thru 0x7f1346 (0x1a69 bytes) 2024-09-14T06:16:44.318 INFO:teuthology.orchestra.run.smithi039.stdout:1501 write 0x6fdbe4f thru 0x6fe93dd (0xd58f bytes) 2024-09-14T06:16:44.323 INFO:teuthology.orchestra.run.smithi039.stdout:1502 read 0x126de7 thru 0x1363a2 (0xf5bc bytes) 2024-09-14T06:16:44.324 INFO:teuthology.orchestra.run.smithi039.stdout:1503 read 0x3b06f7f thru 0x3b0931e (0x23a0 bytes) 2024-09-14T06:16:44.324 INFO:teuthology.orchestra.run.smithi039.stdout:1504 read 0x7913aa thru 0x79daa6 (0xc6fd bytes) 2024-09-14T06:16:44.327 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:44.327+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:44.335 INFO:teuthology.orchestra.run.smithi039.stdout:1506 trunc from 0xa5c621d to 0x34da555 2024-09-14T06:16:44.348 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-14T06:16:44.348 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-14T06:16:44.348 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-14T06:16:44.348 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-14T06:16:44.348 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-14T06:16:44.349 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-14T06:16:44.349 INFO:teuthology.orchestra.run.smithi039.stderr:bench [] [] [] [] 2024-09-14T06:16:44.349 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:16:44.349 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-14T06:16:44.349 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-14T06:16:44.349 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-14T06:16:44.349 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2024-09-14T06:16:44.352 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:16:44.352 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:16:44.352 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-14T06:16:44.368 INFO:teuthology.orchestra.run.smithi039.stdout:1507 writesame 0x6186b36 thru 0x618b8bc data_size 0x1(0x4d87 bytes) 2024-09-14T06:16:44.368 INFO:teuthology.orchestra.run.smithi039.stdout:1509 write 0x572a8a1 thru 0x57380d1 (0xd831 bytes) 2024-09-14T06:16:44.373 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.373 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.373 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.378 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:44.378 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:44.381 INFO:teuthology.orchestra.run.smithi039.stdout:1513 write 0x65aa03d thru 0x65b233b (0x82ff bytes) 2024-09-14T06:16:44.390 INFO:teuthology.orchestra.run.smithi039.stdout:1514 trunc from 0x65b233c to 0x978f6d0 2024-09-14T06:16:44.397 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.397 INFO:teuthology.orchestra.run.smithi039.stdout:1516 write 0x1b26b5d thru 0x1b2886a (0x1d0e bytes) 2024-09-14T06:16:44.400 INFO:teuthology.orchestra.run.smithi039.stdout:1518 read 0x112f8b1 thru 0x1139050 (0x97a0 bytes) 2024-09-14T06:16:44.400 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.401 INFO:teuthology.orchestra.run.smithi039.stdout:1521 punch from 0x607e7ce to 0x6089e59, (0xb68b bytes) 2024-09-14T06:16:44.401 INFO:teuthology.orchestra.run.smithi039.stdout:1522 writesame 0x4fd6b12 thru 0x4fd7f66 data_size 0x5(0x1455 bytes) 2024-09-14T06:16:44.405 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:16:44.416 INFO:teuthology.orchestra.run.smithi039.stdout:1524 write 0xaea0bf1 thru 0xaeae129 (0xd539 bytes) 2024-09-14T06:16:44.421 INFO:teuthology.orchestra.run.smithi039.stdout:1525 read 0x19a22aa thru 0x19a5964 (0x36bb bytes) 2024-09-14T06:16:44.421 INFO:teuthology.orchestra.run.smithi039.stdout:1526 writesame 0x3d6c998 thru 0x3d7265e data_size 0x5b(0x5cc7 bytes) 2024-09-14T06:16:44.426 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.426 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.426 INFO:teuthology.orchestra.run.smithi039.stdout:1529 write 0x1b11d88 thru 0x1b203c7 (0xe640 bytes) 2024-09-14T06:16:44.427 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:44.427+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:44.429 INFO:teuthology.orchestra.run.smithi039.stdout:1531 write 0x5721c3c thru 0x5721e6e (0x233 bytes) 2024-09-14T06:16:44.431 INFO:teuthology.orchestra.run.smithi039.stdout:1533 read 0x4dc4227 thru 0x4dcf495 (0xb26f bytes) 2024-09-14T06:16:44.431 INFO:teuthology.orchestra.run.smithi039.stdout:1534 read 0x2a88485 thru 0x2a8858d (0x109 bytes) 2024-09-14T06:16:44.432 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.432 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.445 INFO:teuthology.orchestra.run.smithi039.stdout:1540 writesame 0xea9801b thru 0xea9816e data_size 0x1(0x154 bytes) 2024-09-14T06:16:44.446 INFO:teuthology.orchestra.run.smithi039.stdout:1541 write 0xc75e95 thru 0xc7875f (0x28cb bytes) 2024-09-14T06:16:44.449 INFO:teuthology.orchestra.run.smithi039.stdout:1542 write 0xc60731e thru 0xc60a59a (0x327d bytes) 2024-09-14T06:16:44.456 INFO:teuthology.orchestra.run.smithi039.stdout:1543 read 0x68af11 thru 0x68b622 (0x712 bytes) 2024-09-14T06:16:44.459 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-09-14T06:16:44.464 INFO:teuthology.orchestra.run.smithi039.stdout:1544 read 0x76aa9bb thru 0x76b79b1 (0xcff7 bytes) 2024-09-14T06:16:44.464 INFO:teuthology.orchestra.run.smithi039.stdout:1545 punch from 0xbbf3a38 to 0xbc01a7d, (0xe045 bytes) 2024-09-14T06:16:44.465 INFO:teuthology.orchestra.run.smithi039.stdout:1546 writesame 0x59bf06f thru 0x59c660c data_size 0x5(0x759e bytes) 2024-09-14T06:16:44.481 INFO:teuthology.orchestra.run.smithi039.stdout:1547 write 0x6effa85 thru 0x6f0bfbb (0xc537 bytes) 2024-09-14T06:16:44.488 INFO:teuthology.orchestra.run.smithi039.stdout:1550 write 0xf0ee48 thru 0xf1b655 (0xc80e bytes) 2024-09-14T06:16:44.490 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.491 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.502 INFO:teuthology.orchestra.run.smithi039.stdout:1553 trunc from 0xea9816f to 0x7d31c2c 2024-09-14T06:16:44.528 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:44.527+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:44.534 INFO:teuthology.orchestra.run.smithi039.stdout:1554 write 0xea8573d thru 0xea85e51 (0x715 bytes) 2024-09-14T06:16:44.540 INFO:teuthology.orchestra.run.smithi039.stdout:1555 punch from 0xc1a0405 to 0xc1a5950, (0x554b bytes) 2024-09-14T06:16:44.540 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.540 INFO:teuthology.orchestra.run.smithi039.stdout:1559 writesame 0x20b9044 thru 0x20bd80d data_size 0x9(0x47ca bytes) 2024-09-14T06:16:44.544 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-14T06:16:44.544 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-14T06:16:44.544 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-14T06:16:44.544 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-14T06:16:44.544 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-14T06:16:44.544 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-14T06:16:44.544 INFO:teuthology.orchestra.run.smithi039.stderr:bench [] [] [] [] 2024-09-14T06:16:44.544 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:16:44.544 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-14T06:16:44.544 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-14T06:16:44.545 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-14T06:16:44.545 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2024-09-14T06:16:44.547 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:16:44.548 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:16:44.548 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-14T06:16:44.559 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:44.559 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:44.559 INFO:teuthology.orchestra.run.smithi039.stdout:1560 trunc from 0xea85e52 to 0x32a5d63 2024-09-14T06:16:44.577 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:16:44.584 INFO:teuthology.orchestra.run.smithi039.stdout:1562 read 0x29eed10 thru 0x29f3c68 (0x4f59 bytes) 2024-09-14T06:16:44.585 INFO:teuthology.orchestra.run.smithi039.stdout:1563 punch from 0x8d56d8 to 0x8e4c33, (0xf55b bytes) 2024-09-14T06:16:44.585 INFO:teuthology.orchestra.run.smithi039.stdout:1564 read 0x283bebf thru 0x2843c0e (0x7d50 bytes) 2024-09-14T06:16:44.586 INFO:teuthology.orchestra.run.smithi039.stdout:1565 punch from 0xd39b7a to 0xd49267, (0xf6ed bytes) 2024-09-14T06:16:44.586 INFO:teuthology.orchestra.run.smithi039.stdout:1567 read 0x2a5f189 thru 0x2a61775 (0x25ed bytes) 2024-09-14T06:16:44.596 INFO:teuthology.orchestra.run.smithi039.stdout:1569 write 0x41f48e3 thru 0x41fb43e (0x6b5c bytes) 2024-09-14T06:16:44.603 INFO:teuthology.orchestra.run.smithi039.stdout:1570 read 0x1235383 thru 0x123ebd1 (0x984f bytes) 2024-09-14T06:16:44.603 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.603 INFO:teuthology.orchestra.run.smithi039.stdout:1572 read 0x39eb4ce thru 0x39f9e7e (0xe9b1 bytes) 2024-09-14T06:16:44.603 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.612 INFO:teuthology.orchestra.run.smithi039.stdout:1575 write 0x516a2a4 thru 0x51795b4 (0xf311 bytes) 2024-09-14T06:16:44.619 INFO:teuthology.orchestra.run.smithi039.stdout:1576 write 0x10ec09 thru 0x112b4c (0x3f44 bytes) 2024-09-14T06:16:44.629 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:44.627+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:44.630 INFO:teuthology.orchestra.run.smithi039.stdout:1577 write 0x644de7b thru 0x645be4a (0xdfd0 bytes) 2024-09-14T06:16:44.649 INFO:teuthology.orchestra.run.smithi039.stdout:1578 write 0xa535d13 thru 0xa53a4f6 (0x47e4 bytes) 2024-09-14T06:16:44.664 INFO:teuthology.orchestra.run.smithi039.stdout:1579 write 0xb475423 thru 0xb475b7f (0x75d bytes) 2024-09-14T06:16:44.670 INFO:teuthology.orchestra.run.smithi039.stdout:1581 writesame 0x2a37191 thru 0x2a3dfd1 data_size 0x5(0x6e41 bytes) 2024-09-14T06:16:44.676 INFO:teuthology.orchestra.run.smithi039.stdout:1582 read 0x55d5502 thru 0x55dada9 (0x58a8 bytes) 2024-09-14T06:16:44.676 INFO:teuthology.orchestra.run.smithi039.stdout:1584 read 0x27018b1 thru 0x270229a (0x9ea bytes) 2024-09-14T06:16:44.677 INFO:teuthology.orchestra.run.smithi039.stdout:1585 write 0x85ba24e thru 0x85c646b (0xc21e bytes) 2024-09-14T06:16:44.684 INFO:teuthology.orchestra.run.smithi039.stdout:1586 punch from 0x66f9006 to 0x66f9a6a, (0xa64 bytes) 2024-09-14T06:16:44.684 INFO:teuthology.orchestra.run.smithi039.stdout:1587 read 0x5a74f0e thru 0x5a754c4 (0x5b7 bytes) 2024-09-14T06:16:44.684 INFO:teuthology.orchestra.run.smithi039.stdout:1589 write 0x2f65506 thru 0x2f6a716 (0x5211 bytes) 2024-09-14T06:16:44.685 INFO:teuthology.orchestra.run.smithi039.stdout:1590 read 0x5b82f8 thru 0x5c138e (0x9097 bytes) 2024-09-14T06:16:44.689 INFO:teuthology.orchestra.run.smithi039.stdout:1592 trunc from 0xb475b80 to 0xde84291 2024-09-14T06:16:44.696 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.696 INFO:teuthology.orchestra.run.smithi039.stdout:1594 writesame 0xc11331b thru 0xc1219e4 data_size 0x81(0xe6ca bytes) 2024-09-14T06:16:44.702 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.702 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.702 INFO:teuthology.orchestra.run.smithi039.stdout:1597 write 0x2a42628 thru 0x2a49aef (0x74c8 bytes) 2024-09-14T06:16:44.705 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.705 INFO:teuthology.orchestra.run.smithi039.stdout:1599 read 0xc7bd3ff thru 0xc7c57b3 (0x83b5 bytes) 2024-09-14T06:16:44.705 INFO:teuthology.orchestra.run.smithi039.stdout:1600 read 0xd1219b1 thru 0xd1318a7 (0xfef7 bytes) 2024-09-14T06:16:44.705 INFO:teuthology.orchestra.run.smithi039.stdout:1602 read 0x69860c4 thru 0x69958c1 (0xf7fe bytes) 2024-09-14T06:16:44.705 INFO:teuthology.orchestra.run.smithi039.stdout:1603 read 0xc0915df thru 0xc09e592 (0xcfb4 bytes) 2024-09-14T06:16:44.706 INFO:teuthology.orchestra.run.smithi039.stdout:1604 writesame 0x88dcbd1 thru 0x88e51e5 data_size 0x5(0x8615 bytes) 2024-09-14T06:16:44.716 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:44.717 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:44.720 INFO:teuthology.orchestra.run.smithi039.stdout:1605 trunc from 0xde84291 to 0x844c5f0 2024-09-14T06:16:44.729 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-14T06:16:44.729 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-14T06:16:44.729 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-14T06:16:44.729 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-14T06:16:44.729 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-14T06:16:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-14T06:16:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:bench [] [] [] [] 2024-09-14T06:16:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:16:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-14T06:16:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-14T06:16:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-14T06:16:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2024-09-14T06:16:44.730 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:44.727+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:44.732 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:16:44.733 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:16:44.733 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-14T06:16:44.734 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:16:44.738 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.747 INFO:teuthology.orchestra.run.smithi039.stdout:1607 writesame 0xa1b0364 thru 0xa1bd648 data_size 0x1(0xd2e5 bytes) 2024-09-14T06:16:44.754 INFO:teuthology.orchestra.run.smithi039.stdout:1608 writesame 0xa2f6074 thru 0xa2f86d8 data_size 0x1(0x2665 bytes) 2024-09-14T06:16:44.754 INFO:teuthology.orchestra.run.smithi039.stdout:1609 read 0x74edc59 thru 0x74f3fa7 (0x634f bytes) 2024-09-14T06:16:44.763 INFO:teuthology.orchestra.run.smithi039.stdout:1610 write 0xc274061 thru 0xc279fd4 (0x5f74 bytes) 2024-09-14T06:16:44.769 INFO:teuthology.orchestra.run.smithi039.stdout:1612 read 0x929e7a7 thru 0x92a4ad1 (0x632b bytes) 2024-09-14T06:16:44.769 INFO:teuthology.orchestra.run.smithi039.stdout:1613 writesame 0x6c81384 thru 0x6c851f7 data_size 0xe(0x3e74 bytes) 2024-09-14T06:16:44.773 INFO:teuthology.orchestra.run.smithi039.stdout:1614 punch from 0x6e1a8a6 to 0x6e2281d, (0x7f77 bytes) 2024-09-14T06:16:44.773 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:44.777 INFO:teuthology.orchestra.run.smithi039.stdout:1618 trunc from 0xc279fd5 to 0x947a313 2024-09-14T06:16:44.803 INFO:teuthology.orchestra.run.smithi039.stdout:1619 write 0xd6346e4 thru 0xd635656 (0xf73 bytes) 2024-09-14T06:16:44.817 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:16:44.819 INFO:teuthology.orchestra.run.smithi039.stdout:1620 trunc from 0xd635657 to 0x36e2ab9 2024-09-14T06:16:44.830 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:44.831+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:44.865 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:44.865 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:44.867 INFO:teuthology.orchestra.run.smithi039.stdout:1622 writesame 0xee4d9c6 thru 0xee4ee41 data_size 0x2e(0x147c bytes) 2024-09-14T06:16:44.876 INFO:teuthology.orchestra.run.smithi039.stdout:1623 trunc from 0xee4ee42 to 0x81b4de5 2024-09-14T06:16:44.883 INFO:teuthology.orchestra.run.smithi039.stdout:1624 write 0xd5038b thru 0xd53993 (0x3609 bytes) 2024-09-14T06:16:44.886 INFO:teuthology.orchestra.run.smithi039.stdout:1625 read 0x4761d4c thru 0x4770426 (0xe6db bytes) 2024-09-14T06:16:44.888 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:16:44.893 INFO:teuthology.orchestra.run.smithi039.stdout:1626 trunc from 0x81b4de5 to 0x2832d22 2024-09-14T06:16:44.911 INFO:teuthology.orchestra.run.smithi039.stdout:1628 trunc from 0x2832d22 to 0x683c4c3 2024-09-14T06:16:44.918 INFO:teuthology.orchestra.run.smithi039.stdout:1630 write 0x44a71a0 thru 0x44a8bae (0x1a0f bytes) 2024-09-14T06:16:44.931 INFO:teuthology.orchestra.run.smithi039.stdout:1633 trunc from 0x683c4c3 to 0xc434242 2024-09-14T06:16:44.931 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:44.931+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:44.937 INFO:teuthology.orchestra.run.smithi039.stdout:1634 read 0x8d0a13e thru 0x8d19669 (0xf52c bytes) 2024-09-14T06:16:44.943 INFO:teuthology.orchestra.run.smithi039.stdout:1637 trunc from 0xc434242 to 0x849579e 2024-09-14T06:16:44.951 INFO:teuthology.orchestra.run.smithi039.stdout:1638 write 0x7845a84 thru 0x7845df5 (0x372 bytes) 2024-09-14T06:16:44.957 INFO:teuthology.orchestra.run.smithi039.stdout:1639 write 0x18c9557 thru 0x18d4053 (0xaafd bytes) 2024-09-14T06:16:44.967 INFO:teuthology.orchestra.run.smithi039.stdout:1640 trunc from 0x849579e to 0x3483800 2024-09-14T06:16:44.992 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:44.993 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:44.995 INFO:teuthology.orchestra.run.smithi039.stdout:1641 write 0x3f008ef thru 0x3f08392 (0x7aa4 bytes) 2024-09-14T06:16:44.999 INFO:teuthology.orchestra.run.smithi039.stdout:1643 read 0x11b5b93 thru 0x11c504a (0xf4b8 bytes) 2024-09-14T06:16:45.000 INFO:teuthology.orchestra.run.smithi039.stdout:1644 writesame 0x1c18a11 thru 0x1c25ea7 data_size 0x3(0xd497 bytes) 2024-09-14T06:16:45.010 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:45.010 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:45.015 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:45.015 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:45.015 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:45.015 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:45.015 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:45.021 INFO:teuthology.orchestra.run.smithi039.stdout:1645 writesame 0xa45f957 thru 0xa460778 data_size 0x9(0xe22 bytes) 2024-09-14T06:16:45.029 INFO:teuthology.orchestra.run.smithi039.stdout:1646 trunc from 0xa460779 to 0x3b4f44b 2024-09-14T06:16:45.030 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:45.030 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:45.032 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:45.031+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:45.034 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:16:45.048 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:45.048 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:45.049 INFO:teuthology.orchestra.run.smithi039.stdout:1647 writesame 0x7151192 thru 0x71558ca data_size 0x1(0x4739 bytes) 2024-09-14T06:16:45.049 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:45.049 INFO:teuthology.orchestra.run.smithi039.stdout:1649 writesame 0x112f21d thru 0x112f25c data_size 0x1(0x40 bytes) 2024-09-14T06:16:45.052 INFO:teuthology.orchestra.run.smithi039.stdout:1651 punch from 0x877f8d to 0x88790e, (0xf981 bytes) 2024-09-14T06:16:45.052 INFO:teuthology.orchestra.run.smithi039.stdout:1652 read 0x506bec9 thru 0x5072c08 (0x6d40 bytes) 2024-09-14T06:16:45.052 INFO:teuthology.orchestra.run.smithi039.stdout:1654 punch from 0x13f2dd7 to 0x1402c4f, (0xfe78 bytes) 2024-09-14T06:16:45.052 INFO:teuthology.orchestra.run.smithi039.stdout:1657 read 0x62ed4b4 thru 0x62f11fd (0x3d4a bytes) 2024-09-14T06:16:45.052 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:45.060 INFO:teuthology.orchestra.run.smithi039.stdout:1659 trunc from 0x71558cb to 0xe3d2cc4 2024-09-14T06:16:45.067 INFO:teuthology.orchestra.run.smithi039.stdout:1660 read 0xd0dacd thru 0xd0e69e (0xbd2 bytes) 2024-09-14T06:16:45.068 INFO:teuthology.orchestra.run.smithi039.stdout:1662 write 0x6b2f34c thru 0x6b38ac3 (0x9778 bytes) 2024-09-14T06:16:45.069 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:45.069 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:45.074 INFO:teuthology.orchestra.run.smithi039.stdout:1663 write 0x1bdb628 thru 0x1bdc3c3 (0xd9c bytes) 2024-09-14T06:16:45.078 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:45.078 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:45.078 INFO:teuthology.orchestra.run.smithi039.stdout:1667 read 0x67c6185 thru 0x67cc71d (0x6599 bytes) 2024-09-14T06:16:45.078 INFO:teuthology.orchestra.run.smithi039.stdout:1668 read 0x58af16f thru 0x58bbeb4 (0xcd46 bytes) 2024-09-14T06:16:45.078 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:45.078 INFO:teuthology.orchestra.run.smithi039.stdout:1670 punch from 0xd3b4faa to 0xd3bc66b, (0x76c1 bytes) 2024-09-14T06:16:45.078 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:45.078 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:45.078 INFO:teuthology.orchestra.run.smithi039.stdout:1676 write 0x4e04cca thru 0x4e13097 (0xe3ce bytes) 2024-09-14T06:16:45.083 INFO:teuthology.orchestra.run.smithi039.stdout:1677 write 0x50b3f87 thru 0x50ba229 (0x62a3 bytes) 2024-09-14T06:16:45.085 INFO:teuthology.orchestra.run.smithi039.stdout:1678 write 0xc17c5ef thru 0xc185758 (0x916a bytes) 2024-09-14T06:16:45.091 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:45.092 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:45.092 INFO:teuthology.orchestra.run.smithi039.stdout:1679 punch from 0x1d5bdd8 to 0x1d60adc, (0x4d04 bytes) 2024-09-14T06:16:45.092 INFO:teuthology.orchestra.run.smithi039.stdout:1680 read 0xdeb3e04 thru 0xdeb7a29 (0x3c26 bytes) 2024-09-14T06:16:45.093 INFO:teuthology.orchestra.run.smithi039.stdout:1682 read 0x64ed656 thru 0x64f8801 (0xb1ac bytes) 2024-09-14T06:16:45.093 INFO:teuthology.orchestra.run.smithi039.stdout:1684 read 0xdabfa00 thru 0xdac8153 (0x8754 bytes) 2024-09-14T06:16:45.093 INFO:teuthology.orchestra.run.smithi039.stdout:1685 read 0x3ae8f51 thru 0x3aeb396 (0x2446 bytes) 2024-09-14T06:16:45.093 INFO:teuthology.orchestra.run.smithi039.stdout:1687 punch from 0x4fa3739 to 0x4faf327, (0xbbee bytes) 2024-09-14T06:16:45.093 INFO:teuthology.orchestra.run.smithi039.stdout:1690 writesame 0x7cea5de thru 0x7cf4779 data_size 0x2(0xa19c bytes) 2024-09-14T06:16:45.105 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-14T06:16:45.105 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-14T06:16:45.105 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-14T06:16:45.105 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-14T06:16:45.105 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-14T06:16:45.105 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-14T06:16:45.105 INFO:teuthology.orchestra.run.smithi039.stderr:bench [] [] [] [] 2024-09-14T06:16:45.105 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:16:45.105 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-14T06:16:45.106 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-14T06:16:45.106 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-14T06:16:45.106 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2024-09-14T06:16:45.107 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:16:45.108 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:16:45.108 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-14T06:16:45.112 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:45.113 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:45.125 INFO:teuthology.orchestra.run.smithi039.stdout:1691 clone 10 order 23 su 1048576 sc 10 2024-09-14T06:16:45.133 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:45.131+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:45.134 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:45.134 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:45.166 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.087748230999999, 2024-09-14T06:16:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.56009052699999995, 2024-09-14T06:16:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 1 2024-09-14T06:16:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077065+0000", 2024-09-14T06:16:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6429999999999999e-06 2024-09-14T06:16:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077079+0000", 2024-09-14T06:16:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4447000000000001e-05 2024-09-14T06:16:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077103+0000", 2024-09-14T06:16:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3856e-05 2024-09-14T06:16:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077120+0000", 2024-09-14T06:16:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7337e-05 2024-09-14T06:16:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077255+0000", 2024-09-14T06:16:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000134652 2024-09-14T06:16:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:16:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077274+0000", 2024-09-14T06:16:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9003000000000001e-05 2024-09-14T06:16:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634272+0000", 2024-09-14T06:16:45.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.55699813099999995 2024-09-14T06:16:45.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634591+0000", 2024-09-14T06:16:45.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031910800000000002 2024-09-14T06:16:45.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:45.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634892+0000", 2024-09-14T06:16:45.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00030066400000000002 2024-09-14T06:16:45.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:45.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636345+0000", 2024-09-14T06:16:45.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001452896 2024-09-14T06:16:45.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:45.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636974+0000", 2024-09-14T06:16:45.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00062927800000000004 2024-09-14T06:16:45.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637071+0000", 2024-09-14T06:16:45.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7178999999999999e-05 2024-09-14T06:16:45.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.173 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.173 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637153+0000", 2024-09-14T06:16:45.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1333000000000002e-05 2024-09-14T06:16:45.173 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.173 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.173 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.173 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13136 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.173 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.091799+0000", 2024-09-14T06:16:45.173 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.073011288, 2024-09-14T06:16:45.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022628747000000001, 2024-09-14T06:16:45.173 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.173 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.174 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.174 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.174 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.174 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.174 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13136 2024-09-14T06:16:45.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.174 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.174 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091799+0000", 2024-09-14T06:16:45.174 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.175 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091799+0000", 2024-09-14T06:16:45.175 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.175 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.175 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091800+0000", 2024-09-14T06:16:45.175 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.68e-07 2024-09-14T06:16:45.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.175 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.175 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091802+0000", 2024-09-14T06:16:45.175 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6699999999999998e-06 2024-09-14T06:16:45.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.175 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091810+0000", 2024-09-14T06:16:45.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6750000000000002e-06 2024-09-14T06:16:45.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091816+0000", 2024-09-14T06:16:45.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1600000000000003e-06 2024-09-14T06:16:45.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.110345+0000", 2024-09-14T06:16:45.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018528393000000001 2024-09-14T06:16:45.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.177 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.177 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.111299+0000", 2024-09-14T06:16:45.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00095451100000000001 2024-09-14T06:16:45.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.177 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.177 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.177 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114428+0000", 2024-09-14T06:16:45.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031287699999999999 2024-09-14T06:16:45.177 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.177 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.177 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.177 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.177 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13154 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.177 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092189+0000", 2024-09-14T06:16:45.177 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0726214390000006, 2024-09-14T06:16:45.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022359604000000002, 2024-09-14T06:16:45.178 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.178 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.178 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.178 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.178 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.178 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.178 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13154 2024-09-14T06:16:45.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.178 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.178 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.178 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092189+0000", 2024-09-14T06:16:45.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.179 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092189+0000", 2024-09-14T06:16:45.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.179 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092189+0000", 2024-09-14T06:16:45.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9700000000000002e-07 2024-09-14T06:16:45.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.179 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092192+0000", 2024-09-14T06:16:45.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.317e-06 2024-09-14T06:16:45.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092198+0000", 2024-09-14T06:16:45.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.387e-06 2024-09-14T06:16:45.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092203+0000", 2024-09-14T06:16:45.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.259e-06 2024-09-14T06:16:45.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114338+0000", 2024-09-14T06:16:45.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022135207 2024-09-14T06:16:45.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.181 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.181 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.181 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114451+0000", 2024-09-14T06:16:45.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000112518 2024-09-14T06:16:45.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.181 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.181 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.181 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114548+0000", 2024-09-14T06:16:45.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7518999999999999e-05 2024-09-14T06:16:45.181 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.181 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.181 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.182 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.182 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13155 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.182 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092211+0000", 2024-09-14T06:16:45.182 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0725998249999993, 2024-09-14T06:16:45.182 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.02255184, 2024-09-14T06:16:45.182 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.182 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.182 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.182 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.183 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.183 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.183 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13155 2024-09-14T06:16:45.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.183 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.183 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.183 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.183 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092211+0000", 2024-09-14T06:16:45.183 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.183 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.183 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.183 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092211+0000", 2024-09-14T06:16:45.183 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.184 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.184 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092211+0000", 2024-09-14T06:16:45.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8500000000000002e-07 2024-09-14T06:16:45.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.184 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.184 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092213+0000", 2024-09-14T06:16:45.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1380000000000001e-06 2024-09-14T06:16:45.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.184 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.184 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092220+0000", 2024-09-14T06:16:45.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5250000000000002e-06 2024-09-14T06:16:45.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.185 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092224+0000", 2024-09-14T06:16:45.185 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.843e-06 2024-09-14T06:16:45.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.185 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114580+0000", 2024-09-14T06:16:45.185 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022355455999999999 2024-09-14T06:16:45.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.185 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114673+0000", 2024-09-14T06:16:45.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2987999999999998e-05 2024-09-14T06:16:45.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.186 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114762+0000", 2024-09-14T06:16:45.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9505000000000004e-05 2024-09-14T06:16:45.186 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.186 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.186 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.186 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13156 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 409600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.186 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092232+0000", 2024-09-14T06:16:45.187 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0725787390000008, 2024-09-14T06:16:45.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022738728999999999, 2024-09-14T06:16:45.187 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.187 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.187 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.187 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.187 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.187 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.187 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13156 2024-09-14T06:16:45.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.187 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092232+0000", 2024-09-14T06:16:45.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.188 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092232+0000", 2024-09-14T06:16:45.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.188 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092232+0000", 2024-09-14T06:16:45.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8099999999999998e-07 2024-09-14T06:16:45.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.188 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092234+0000", 2024-09-14T06:16:45.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0379999999999998e-06 2024-09-14T06:16:45.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.189 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.189 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092240+0000", 2024-09-14T06:16:45.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2859999999999996e-06 2024-09-14T06:16:45.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.189 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.189 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092248+0000", 2024-09-14T06:16:45.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5140000000000004e-06 2024-09-14T06:16:45.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.189 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.189 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114792+0000", 2024-09-14T06:16:45.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022543758000000001 2024-09-14T06:16:45.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.190 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.190 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114865+0000", 2024-09-14T06:16:45.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3652000000000006e-05 2024-09-14T06:16:45.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.190 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.190 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114970+0000", 2024-09-14T06:16:45.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010509999999999999 2024-09-14T06:16:45.190 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.190 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.190 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.191 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13157 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 524288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.191 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092256+0000", 2024-09-14T06:16:45.191 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0725548119999999, 2024-09-14T06:16:45.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022964834999999999, 2024-09-14T06:16:45.191 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.191 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.191 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.191 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.191 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.191 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.191 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13157 2024-09-14T06:16:45.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.192 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092256+0000", 2024-09-14T06:16:45.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092256+0000", 2024-09-14T06:16:45.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092256+0000", 2024-09-14T06:16:45.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6199999999999998e-07 2024-09-14T06:16:45.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092258+0000", 2024-09-14T06:16:45.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3530000000000002e-06 2024-09-14T06:16:45.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092265+0000", 2024-09-14T06:16:45.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5069999999999997e-06 2024-09-14T06:16:45.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092270+0000", 2024-09-14T06:16:45.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2530000000000004e-06 2024-09-14T06:16:45.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.194 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115000+0000", 2024-09-14T06:16:45.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022729787000000001 2024-09-14T06:16:45.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.194 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115074+0000", 2024-09-14T06:16:45.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4506999999999997e-05 2024-09-14T06:16:45.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.194 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115220+0000", 2024-09-14T06:16:45.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000145966 2024-09-14T06:16:45.195 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.195 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.195 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.195 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13158 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.195 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092282+0000", 2024-09-14T06:16:45.195 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0725279400000005, 2024-09-14T06:16:45.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.023233365999999998, 2024-09-14T06:16:45.195 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.195 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.195 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.195 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.195 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.196 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.196 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13158 2024-09-14T06:16:45.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.196 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092282+0000", 2024-09-14T06:16:45.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092282+0000", 2024-09-14T06:16:45.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092283+0000", 2024-09-14T06:16:45.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:16:45.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092285+0000", 2024-09-14T06:16:45.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3460000000000001e-06 2024-09-14T06:16:45.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092291+0000", 2024-09-14T06:16:45.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0549999999999996e-06 2024-09-14T06:16:45.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092296+0000", 2024-09-14T06:16:45.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1800000000000004e-06 2024-09-14T06:16:45.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115253+0000", 2024-09-14T06:16:45.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022956764000000001 2024-09-14T06:16:45.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115347+0000", 2024-09-14T06:16:45.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3762000000000001e-05 2024-09-14T06:16:45.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.199 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115516+0000", 2024-09-14T06:16:45.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016881200000000001 2024-09-14T06:16:45.199 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.199 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.199 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.199 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13159 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.199 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092307+0000", 2024-09-14T06:16:45.199 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0725038040000001, 2024-09-14T06:16:45.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.023865384999999999, 2024-09-14T06:16:45.200 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.200 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.200 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.200 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.200 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.200 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.200 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13159 2024-09-14T06:16:45.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.200 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092307+0000", 2024-09-14T06:16:45.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092307+0000", 2024-09-14T06:16:45.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092307+0000", 2024-09-14T06:16:45.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:16:45.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092309+0000", 2024-09-14T06:16:45.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3080000000000002e-06 2024-09-14T06:16:45.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092316+0000", 2024-09-14T06:16:45.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7120000000000003e-06 2024-09-14T06:16:45.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092321+0000", 2024-09-14T06:16:45.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7729999999999999e-06 2024-09-14T06:16:45.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115546+0000", 2024-09-14T06:16:45.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.023225000999999999 2024-09-14T06:16:45.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115621+0000", 2024-09-14T06:16:45.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4981999999999995e-05 2024-09-14T06:16:45.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116172+0000", 2024-09-14T06:16:45.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00055117199999999995 2024-09-14T06:16:45.203 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.203 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.203 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.203 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13160 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.203 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092329+0000", 2024-09-14T06:16:45.204 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.072481647, 2024-09-14T06:16:45.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024062849000000001, 2024-09-14T06:16:45.204 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.204 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.204 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.204 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.204 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.204 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.204 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13160 2024-09-14T06:16:45.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.204 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092329+0000", 2024-09-14T06:16:45.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.205 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.205 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092329+0000", 2024-09-14T06:16:45.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.205 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.205 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092329+0000", 2024-09-14T06:16:45.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8299999999999997e-07 2024-09-14T06:16:45.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.205 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.205 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092331+0000", 2024-09-14T06:16:45.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:16:45.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092338+0000", 2024-09-14T06:16:45.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:16:45.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092351+0000", 2024-09-14T06:16:45.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3059e-05 2024-09-14T06:16:45.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116204+0000", 2024-09-14T06:16:45.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.023853268 2024-09-14T06:16:45.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116287+0000", 2024-09-14T06:16:45.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3140999999999996e-05 2024-09-14T06:16:45.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116392+0000", 2024-09-14T06:16:45.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010451900000000001 2024-09-14T06:16:45.207 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.207 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.207 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.208 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.208 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13161 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.208 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092359+0000", 2024-09-14T06:16:45.208 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0724512490000002, 2024-09-14T06:16:45.208 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024248515000000002, 2024-09-14T06:16:45.208 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.208 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.208 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.208 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.208 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.208 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.208 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13161 2024-09-14T06:16:45.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.209 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.209 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092359+0000", 2024-09-14T06:16:45.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.209 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092359+0000", 2024-09-14T06:16:45.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.210 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092360+0000", 2024-09-14T06:16:45.210 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:16:45.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.210 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.210 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.210 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092368+0000", 2024-09-14T06:16:45.210 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1140000000000006e-06 2024-09-14T06:16:45.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.210 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.210 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.210 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092375+0000", 2024-09-14T06:16:45.210 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4399999999999999e-06 2024-09-14T06:16:45.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.211 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.211 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.211 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092380+0000", 2024-09-14T06:16:45.211 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3170000000000001e-06 2024-09-14T06:16:45.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.211 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.211 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.211 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116426+0000", 2024-09-14T06:16:45.211 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024045492000000002 2024-09-14T06:16:45.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.211 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.211 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.212 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116513+0000", 2024-09-14T06:16:45.212 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6936999999999998e-05 2024-09-14T06:16:45.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.212 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.212 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.212 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116608+0000", 2024-09-14T06:16:45.212 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4765000000000005e-05 2024-09-14T06:16:45.212 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.212 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.212 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.212 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.212 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13162 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.213 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092389+0000", 2024-09-14T06:16:45.213 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0724214369999991, 2024-09-14T06:16:45.213 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024432156, 2024-09-14T06:16:45.213 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.213 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.213 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.213 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.213 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.213 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.213 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13162 2024-09-14T06:16:45.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.213 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.213 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.213 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.214 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092389+0000", 2024-09-14T06:16:45.214 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.214 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.214 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.214 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092389+0000", 2024-09-14T06:16:45.214 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.214 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.214 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.214 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092389+0000", 2024-09-14T06:16:45.214 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:16:45.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.215 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.215 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092392+0000", 2024-09-14T06:16:45.215 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3460000000000001e-06 2024-09-14T06:16:45.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.215 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.215 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092398+0000", 2024-09-14T06:16:45.215 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6789999999999999e-06 2024-09-14T06:16:45.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.215 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.215 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092403+0000", 2024-09-14T06:16:45.215 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9389999999999996e-06 2024-09-14T06:16:45.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.216 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.216 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116657+0000", 2024-09-14T06:16:45.216 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024253896 2024-09-14T06:16:45.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.216 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.216 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.216 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116732+0000", 2024-09-14T06:16:45.216 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4727000000000005e-05 2024-09-14T06:16:45.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.216 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.216 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.216 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116821+0000", 2024-09-14T06:16:45.216 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:16:45.216 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.217 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.217 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.217 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.217 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13163 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.217 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092410+0000", 2024-09-14T06:16:45.217 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0723999790000001, 2024-09-14T06:16:45.217 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024727810999999999, 2024-09-14T06:16:45.217 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.217 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.217 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.217 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.217 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.217 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.218 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13163 2024-09-14T06:16:45.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.218 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.218 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.218 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.218 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092410+0000", 2024-09-14T06:16:45.218 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.218 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.218 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.218 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092410+0000", 2024-09-14T06:16:45.218 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.218 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.219 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.219 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092411+0000", 2024-09-14T06:16:45.219 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2899999999999999e-07 2024-09-14T06:16:45.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.219 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.219 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.219 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092413+0000", 2024-09-14T06:16:45.219 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3649999999999998e-06 2024-09-14T06:16:45.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.219 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.219 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.219 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092420+0000", 2024-09-14T06:16:45.219 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5729999999999996e-06 2024-09-14T06:16:45.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.219 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.220 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.220 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092425+0000", 2024-09-14T06:16:45.220 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0669999999999999e-06 2024-09-14T06:16:45.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.220 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.220 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.220 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116850+0000", 2024-09-14T06:16:45.220 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024424833999999999 2024-09-14T06:16:45.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.220 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.220 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.220 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117058+0000", 2024-09-14T06:16:45.220 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020800199999999999 2024-09-14T06:16:45.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.221 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.221 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.221 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117138+0000", 2024-09-14T06:16:45.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0541e-05 2024-09-14T06:16:45.221 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.221 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.221 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.221 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.221 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13164 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.221 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092432+0000", 2024-09-14T06:16:45.221 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0723783030000007, 2024-09-14T06:16:45.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024878806999999999, 2024-09-14T06:16:45.222 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.222 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.222 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.222 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.222 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.222 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.222 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13164 2024-09-14T06:16:45.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.222 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.222 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.222 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092432+0000", 2024-09-14T06:16:45.222 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.223 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.223 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092432+0000", 2024-09-14T06:16:45.223 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.223 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.223 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092432+0000", 2024-09-14T06:16:45.223 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:16:45.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.223 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.223 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092434+0000", 2024-09-14T06:16:45.223 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.968e-06 2024-09-14T06:16:45.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.224 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.224 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092440+0000", 2024-09-14T06:16:45.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.964e-06 2024-09-14T06:16:45.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.224 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.224 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092445+0000", 2024-09-14T06:16:45.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9529999999999998e-06 2024-09-14T06:16:45.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.224 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.224 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117166+0000", 2024-09-14T06:16:45.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024720925000000001 2024-09-14T06:16:45.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.225 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.225 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.225 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117239+0000", 2024-09-14T06:16:45.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2650000000000004e-05 2024-09-14T06:16:45.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.225 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.225 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.225 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117311+0000", 2024-09-14T06:16:45.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1945000000000003e-05 2024-09-14T06:16:45.225 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.225 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.225 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.226 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.226 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13165 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.226 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092453+0000", 2024-09-14T06:16:45.226 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0723576220000002, 2024-09-14T06:16:45.226 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025052076999999999, 2024-09-14T06:16:45.226 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.226 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.226 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.226 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.226 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.226 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.226 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13165 2024-09-14T06:16:45.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.227 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.227 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092453+0000", 2024-09-14T06:16:45.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.227 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092453+0000", 2024-09-14T06:16:45.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.228 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092453+0000", 2024-09-14T06:16:45.228 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7300000000000001e-07 2024-09-14T06:16:45.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.228 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.228 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092455+0000", 2024-09-14T06:16:45.228 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.204e-06 2024-09-14T06:16:45.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.228 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.228 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092461+0000", 2024-09-14T06:16:45.228 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0299999999999999e-06 2024-09-14T06:16:45.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.229 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.229 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092466+0000", 2024-09-14T06:16:45.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7820000000000001e-06 2024-09-14T06:16:45.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.229 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.229 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.229 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117343+0000", 2024-09-14T06:16:45.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024876334999999999 2024-09-14T06:16:45.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.229 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.229 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.229 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117416+0000", 2024-09-14T06:16:45.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3368000000000006e-05 2024-09-14T06:16:45.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.230 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.230 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117505+0000", 2024-09-14T06:16:45.230 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8885e-05 2024-09-14T06:16:45.230 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.230 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.230 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.230 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13166 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.230 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092490+0000", 2024-09-14T06:16:45.230 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0723204590000002, 2024-09-14T06:16:45.230 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025236825000000001, 2024-09-14T06:16:45.230 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.230 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.231 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.231 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.231 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.231 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.231 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13166 2024-09-14T06:16:45.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.231 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.231 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092490+0000", 2024-09-14T06:16:45.231 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.232 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092490+0000", 2024-09-14T06:16:45.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.232 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.232 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092490+0000", 2024-09-14T06:16:45.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6199999999999999e-07 2024-09-14T06:16:45.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.232 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.232 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092492+0000", 2024-09-14T06:16:45.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7910000000000001e-06 2024-09-14T06:16:45.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.233 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.233 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092497+0000", 2024-09-14T06:16:45.233 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4190000000000001e-06 2024-09-14T06:16:45.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.233 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.233 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.233 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092501+0000", 2024-09-14T06:16:45.233 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0230000000000003e-06 2024-09-14T06:16:45.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.233 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.233 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.233 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117533+0000", 2024-09-14T06:16:45.233 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025031623999999999 2024-09-14T06:16:45.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.233 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.234 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.234 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117638+0000", 2024-09-14T06:16:45.234 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010451900000000001 2024-09-14T06:16:45.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.234 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.234 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.234 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117727+0000", 2024-09-14T06:16:45.234 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9086999999999996e-05 2024-09-14T06:16:45.234 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.234 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.234 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.234 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.234 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13167 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.235 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092507+0000", 2024-09-14T06:16:45.235 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0723029709999992, 2024-09-14T06:16:45.235 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025412396, 2024-09-14T06:16:45.235 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.235 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.235 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.235 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.235 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.235 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.235 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13167 2024-09-14T06:16:45.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.235 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.235 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.236 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092507+0000", 2024-09-14T06:16:45.236 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.236 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.236 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.236 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092507+0000", 2024-09-14T06:16:45.236 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.236 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.236 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.236 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092508+0000", 2024-09-14T06:16:45.236 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.65e-07 2024-09-14T06:16:45.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.237 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.237 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092509+0000", 2024-09-14T06:16:45.237 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.426e-06 2024-09-14T06:16:45.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.237 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.237 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092512+0000", 2024-09-14T06:16:45.237 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.416e-06 2024-09-14T06:16:45.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.237 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.237 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092515+0000", 2024-09-14T06:16:45.237 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.971e-06 2024-09-14T06:16:45.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.238 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117755+0000", 2024-09-14T06:16:45.238 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.02523999 2024-09-14T06:16:45.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.238 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117830+0000", 2024-09-14T06:16:45.238 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4386000000000002e-05 2024-09-14T06:16:45.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.238 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117920+0000", 2024-09-14T06:16:45.239 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9942e-05 2024-09-14T06:16:45.239 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.239 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.239 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.239 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13168 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.239 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092519+0000", 2024-09-14T06:16:45.239 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0722908380000007, 2024-09-14T06:16:45.239 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025607092000000001, 2024-09-14T06:16:45.239 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.239 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.239 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.240 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.240 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.240 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.240 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13168 2024-09-14T06:16:45.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.240 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.240 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.240 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.240 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092519+0000", 2024-09-14T06:16:45.240 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.240 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.240 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.241 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092519+0000", 2024-09-14T06:16:45.241 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.241 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.241 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.241 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092520+0000", 2024-09-14T06:16:45.241 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4999999999999999e-07 2024-09-14T06:16:45.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.241 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.241 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.241 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092521+0000", 2024-09-14T06:16:45.241 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2440000000000001e-06 2024-09-14T06:16:45.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.241 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.242 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.242 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092524+0000", 2024-09-14T06:16:45.242 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3189999999999999e-06 2024-09-14T06:16:45.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.242 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.242 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092527+0000", 2024-09-14T06:16:45.242 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.638e-06 2024-09-14T06:16:45.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.242 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.242 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117948+0000", 2024-09-14T06:16:45.242 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025421293000000001 2024-09-14T06:16:45.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.118025+0000", 2024-09-14T06:16:45.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6410999999999997e-05 2024-09-14T06:16:45.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.118127+0000", 2024-09-14T06:16:45.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000101937 2024-09-14T06:16:45.244 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.244 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.244 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.244 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.244 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13169 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.244 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092531+0000", 2024-09-14T06:16:45.245 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0722796409999997, 2024-09-14T06:16:45.245 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.030222784999999999, 2024-09-14T06:16:45.245 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.245 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.245 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.245 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.245 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.245 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.245 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13169 2024-09-14T06:16:45.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.245 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.245 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.245 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.246 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092531+0000", 2024-09-14T06:16:45.246 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.246 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.246 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.246 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092531+0000", 2024-09-14T06:16:45.246 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.246 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.246 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.246 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092531+0000", 2024-09-14T06:16:45.246 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.41e-07 2024-09-14T06:16:45.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.247 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.247 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.247 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092545+0000", 2024-09-14T06:16:45.247 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4256999999999999e-05 2024-09-14T06:16:45.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.247 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.247 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.247 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092553+0000", 2024-09-14T06:16:45.247 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.374e-06 2024-09-14T06:16:45.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.247 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.247 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.247 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092558+0000", 2024-09-14T06:16:45.248 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2830000000000001e-06 2024-09-14T06:16:45.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.248 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.248 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.122584+0000", 2024-09-14T06:16:45.248 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.030026091000000001 2024-09-14T06:16:45.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.248 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.248 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.122672+0000", 2024-09-14T06:16:45.248 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7880999999999994e-05 2024-09-14T06:16:45.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.249 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.249 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.122753+0000", 2024-09-14T06:16:45.249 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1657999999999996e-05 2024-09-14T06:16:45.249 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.249 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.249 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.249 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.249 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:16984 3.1c 3:3aec2de0:::rbd_data.1152fc09e7e6.000000000000000d:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:45.249 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:37.655478+0000", 2024-09-14T06:16:45.249 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 7.5093323060000001, 2024-09-14T06:16:45.249 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.034057654999999999, 2024-09-14T06:16:45.249 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.249 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.250 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.250 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.250 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.250 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.250 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 16984 2024-09-14T06:16:45.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.250 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.250 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.250 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.250 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655478+0000", 2024-09-14T06:16:45.250 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.250 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.251 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.251 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655478+0000", 2024-09-14T06:16:45.251 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.251 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.251 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.251 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655480+0000", 2024-09-14T06:16:45.251 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.074e-06 2024-09-14T06:16:45.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.251 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.251 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.251 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655489+0000", 2024-09-14T06:16:45.251 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2799999999999992e-06 2024-09-14T06:16:45.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.252 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.252 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.252 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655509+0000", 2024-09-14T06:16:45.252 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9494e-05 2024-09-14T06:16:45.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.252 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.252 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.252 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655518+0000", 2024-09-14T06:16:45.252 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7730000000000005e-06 2024-09-14T06:16:45.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.252 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.252 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.252 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655599+0000", 2024-09-14T06:16:45.252 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1306999999999999e-05 2024-09-14T06:16:45.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.253 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.253 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655708+0000", 2024-09-14T06:16:45.253 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010870300000000001 2024-09-14T06:16:45.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.253 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:45.253 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.661376+0000", 2024-09-14T06:16:45.253 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056679460000000001 2024-09-14T06:16:45.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.253 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:45.253 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.689383+0000", 2024-09-14T06:16:45.254 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.028007632000000001 2024-09-14T06:16:45.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.254 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.254 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:45.254 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.689407+0000", 2024-09-14T06:16:45.254 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3743e-05 2024-09-14T06:16:45.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.254 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.254 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.254 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.689476+0000", 2024-09-14T06:16:45.254 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9423000000000003e-05 2024-09-14T06:16:45.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.254 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.255 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.255 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.689536+0000", 2024-09-14T06:16:45.255 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9280000000000002e-05 2024-09-14T06:16:45.255 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.255 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.255 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.255 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:18695 3.10 3:0c23de83:::rbd_data.1152227be94f.000000000000001a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:16:45.255 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:42.335500+0000", 2024-09-14T06:16:45.255 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 2.8293108170000001, 2024-09-14T06:16:45.255 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.060751287000000001, 2024-09-14T06:16:45.255 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.255 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.256 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.256 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.256 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.256 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.256 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 18695 2024-09-14T06:16:45.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.256 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.256 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.256 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.256 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.335500+0000", 2024-09-14T06:16:45.256 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.256 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.256 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.257 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.335500+0000", 2024-09-14T06:16:45.257 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.257 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.257 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.257 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.335502+0000", 2024-09-14T06:16:45.257 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9870000000000002e-06 2024-09-14T06:16:45.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.257 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.257 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.257 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.335512+0000", 2024-09-14T06:16:45.257 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0251999999999999e-05 2024-09-14T06:16:45.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.257 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.257 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.258 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.335540+0000", 2024-09-14T06:16:45.258 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8411999999999998e-05 2024-09-14T06:16:45.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.258 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.258 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.258 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.335557+0000", 2024-09-14T06:16:45.258 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7030000000000001e-05 2024-09-14T06:16:45.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.258 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.258 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.258 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.335654+0000", 2024-09-14T06:16:45.258 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7113000000000003e-05 2024-09-14T06:16:45.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.258 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.259 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.259 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.335737+0000", 2024-09-14T06:16:45.259 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2662000000000002e-05 2024-09-14T06:16:45.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.259 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.259 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:45.259 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.343481+0000", 2024-09-14T06:16:45.259 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0077441410000000004 2024-09-14T06:16:45.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.259 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.259 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:45.259 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.395863+0000", 2024-09-14T06:16:45.259 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052381843999999997 2024-09-14T06:16:45.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.260 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.260 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:45.260 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.395917+0000", 2024-09-14T06:16:45.260 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.397e-05 2024-09-14T06:16:45.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.260 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.260 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.260 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.396119+0000", 2024-09-14T06:16:45.260 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020236900000000001 2024-09-14T06:16:45.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.260 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.260 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.261 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.396251+0000", 2024-09-14T06:16:45.261 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000131507 2024-09-14T06:16:45.261 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.261 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.261 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.261 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.261 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.261 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:45.262 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:16:45.263 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:16:45.263 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-14T06:16:45.263 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-14T06:16:45.264 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-14T06:16:45.264 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-14T06:16:45.264 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-14T06:16:45.264 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-14T06:16:45.264 INFO:teuthology.orchestra.run.smithi039.stderr:bench [] [] [] [] 2024-09-14T06:16:45.264 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:16:45.264 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-14T06:16:45.264 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-14T06:16:45.264 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-14T06:16:45.264 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2024-09-14T06:16:45.264 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:16:45.264 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-14T06:16:45.265 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:45.263+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:45.363 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:45.363+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:45.437 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:45.437 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:45.437 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:45.437 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:45.452 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:16:45.465 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:45.463+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-14T06:16:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-14T06:16:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-14T06:16:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-14T06:16:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-14T06:16:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-14T06:16:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:bench [] [] [] [] 2024-09-14T06:16:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:16:45.471 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-14T06:16:45.471 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-14T06:16:45.471 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-14T06:16:45.471 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2024-09-14T06:16:45.473 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:16:45.473 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:16:45.473 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-14T06:16:45.566 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:45.567+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:45.589 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:45.589 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:45.590 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:45.590 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:45.590 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:45.607 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:16:45.667 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:45.667+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:45.673 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-14T06:16:45.673 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-14T06:16:45.673 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-14T06:16:45.673 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-14T06:16:45.673 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-14T06:16:45.673 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-14T06:16:45.673 INFO:teuthology.orchestra.run.smithi039.stderr:bench [] [] [] [] 2024-09-14T06:16:45.673 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:16:45.673 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-14T06:16:45.673 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-14T06:16:45.673 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-14T06:16:45.674 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2024-09-14T06:16:45.676 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:16:45.676 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:16:45.732 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:45.732 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:45.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:45.732 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:45.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.732 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.5 30/29 [PushOp(3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head, version: 28'5, data_included: [0~212992,360448~311296], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head@28'5, size: 1032192, copy_subset: [(0, 1032192)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1032192, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:16:45.733 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:45.733 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 18.455741301, 2024-09-14T06:16:45.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041630249999999999, 2024-09-14T06:16:45.733 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.733 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.733 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.733 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:45.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:45.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275888+0000", 2024-09-14T06:16:45.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.367e-06 2024-09-14T06:16:45.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277098+0000", 2024-09-14T06:16:45.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001210661 2024-09-14T06:16:45.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277122+0000", 2024-09-14T06:16:45.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3413000000000001e-05 2024-09-14T06:16:45.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277141+0000", 2024-09-14T06:16:45.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9133000000000001e-05 2024-09-14T06:16:45.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277364+0000", 2024-09-14T06:16:45.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000223365 2024-09-14T06:16:45.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277405+0000", 2024-09-14T06:16:45.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1053000000000001e-05 2024-09-14T06:16:45.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.280049+0000", 2024-09-14T06:16:45.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026440330000000001 2024-09-14T06:16:45.736 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.736 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.736 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.737 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.16 30/29 [PushOp(3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head, version: 28'3, data_included: [0~212992,344064~327680], data_size: 540672, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head@28'3, size: 1032192, copy_subset: [(0, 1032192)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1032192, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:16:45.737 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.287894+0000", 2024-09-14T06:16:45.737 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 18.443733108, 2024-09-14T06:16:45.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032539600000000002, 2024-09-14T06:16:45.737 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.737 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.737 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.737 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.287894+0000", 2024-09-14T06:16:45.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.287894+0000", 2024-09-14T06:16:45.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.287899+0000", 2024-09-14T06:16:45.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7280000000000003e-06 2024-09-14T06:16:45.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.288805+0000", 2024-09-14T06:16:45.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00090622700000000001 2024-09-14T06:16:45.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.288855+0000", 2024-09-14T06:16:45.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9835e-05 2024-09-14T06:16:45.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.288902+0000", 2024-09-14T06:16:45.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6854999999999998e-05 2024-09-14T06:16:45.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.289120+0000", 2024-09-14T06:16:45.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000217495 2024-09-14T06:16:45.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.289142+0000", 2024-09-14T06:16:45.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2870999999999998e-05 2024-09-14T06:16:45.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.291148+0000", 2024-09-14T06:16:45.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020059489999999999 2024-09-14T06:16:45.740 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.740 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.740 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.741 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3868 3.16 3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head [stat,truncate 1998848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:45.741 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:45.741 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 18.229145931000001, 2024-09-14T06:16:45.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034445919999999998, 2024-09-14T06:16:45.741 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.741 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.741 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.741 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.741 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.741 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.741 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3868 2024-09-14T06:16:45.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.741 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:45.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:45.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:45.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1900000000000002e-07 2024-09-14T06:16:45.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502501+0000", 2024-09-14T06:16:45.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8340999999999999e-05 2024-09-14T06:16:45.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502512+0000", 2024-09-14T06:16:45.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1347e-05 2024-09-14T06:16:45.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502518+0000", 2024-09-14T06:16:45.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.947e-06 2024-09-14T06:16:45.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502570+0000", 2024-09-14T06:16:45.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1858999999999999e-05 2024-09-14T06:16:45.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502633+0000", 2024-09-14T06:16:45.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2974999999999999e-05 2024-09-14T06:16:45.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:16:45.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502795+0000", 2024-09-14T06:16:45.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016262899999999999 2024-09-14T06:16:45.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:45.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505179+0000", 2024-09-14T06:16:45.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002383844 2024-09-14T06:16:45.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:45.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505676+0000", 2024-09-14T06:16:45.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00049634899999999999 2024-09-14T06:16:45.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505816+0000", 2024-09-14T06:16:45.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00014026300000000001 2024-09-14T06:16:45.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505926+0000", 2024-09-14T06:16:45.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000110319 2024-09-14T06:16:45.746 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.746 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.746 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.747 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3902 3.5 3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:45.747 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.569016+0000", 2024-09-14T06:16:45.747 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 18.162612060000001, 2024-09-14T06:16:45.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003356197, 2024-09-14T06:16:45.747 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.747 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.747 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.747 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.747 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.747 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.747 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3902 2024-09-14T06:16:45.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.747 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569016+0000", 2024-09-14T06:16:45.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569016+0000", 2024-09-14T06:16:45.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569019+0000", 2024-09-14T06:16:45.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2279999999999998e-06 2024-09-14T06:16:45.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569029+0000", 2024-09-14T06:16:45.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8079999999999999e-06 2024-09-14T06:16:45.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569059+0000", 2024-09-14T06:16:45.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0198000000000001e-05 2024-09-14T06:16:45.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569074+0000", 2024-09-14T06:16:45.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4968e-05 2024-09-14T06:16:45.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569165+0000", 2024-09-14T06:16:45.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0867999999999998e-05 2024-09-14T06:16:45.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569282+0000", 2024-09-14T06:16:45.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000117602 2024-09-14T06:16:45.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:45.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569405+0000", 2024-09-14T06:16:45.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122471 2024-09-14T06:16:45.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:45.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.570585+0000", 2024-09-14T06:16:45.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001180459 2024-09-14T06:16:45.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:45.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.572315+0000", 2024-09-14T06:16:45.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0017295329999999999 2024-09-14T06:16:45.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.572345+0000", 2024-09-14T06:16:45.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0784999999999997e-05 2024-09-14T06:16:45.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.572372+0000", 2024-09-14T06:16:45.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6276999999999999e-05 2024-09-14T06:16:45.752 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.752 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.752 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.753 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3916 3.16 3:6ab8b1d8:::rbd_data.1152bd92e04f.0000000000000002:head [stat,truncate 1590169] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:45.753 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:45.753 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 18.152426500000001, 2024-09-14T06:16:45.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022958078999999999, 2024-09-14T06:16:45.753 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.753 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.753 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.753 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.753 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.753 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.753 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3916 2024-09-14T06:16:45.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.754 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:45.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:45.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579203+0000", 2024-09-14T06:16:45.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.748e-06 2024-09-14T06:16:45.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579208+0000", 2024-09-14T06:16:45.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:16:45.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579223+0000", 2024-09-14T06:16:45.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.508e-05 2024-09-14T06:16:45.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579232+0000", 2024-09-14T06:16:45.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9719999999999998e-06 2024-09-14T06:16:45.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579307+0000", 2024-09-14T06:16:45.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5028000000000001e-05 2024-09-14T06:16:45.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579412+0000", 2024-09-14T06:16:45.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010525400000000001 2024-09-14T06:16:45.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:16:45.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579535+0000", 2024-09-14T06:16:45.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122726 2024-09-14T06:16:45.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:45.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.581415+0000", 2024-09-14T06:16:45.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018800360000000001 2024-09-14T06:16:45.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.758 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:45.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602017+0000", 2024-09-14T06:16:45.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.020602425000000001 2024-09-14T06:16:45.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.758 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602096+0000", 2024-09-14T06:16:45.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8968999999999996e-05 2024-09-14T06:16:45.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602159+0000", 2024-09-14T06:16:45.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2824999999999996e-05 2024-09-14T06:16:45.759 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.759 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.759 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.759 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4704 3.5 3:a428c282:::rbd_data.1152bd92e04f.0000000000000047:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:45.759 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.943845+0000", 2024-09-14T06:16:45.759 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 17.787782594999999, 2024-09-14T06:16:45.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003139476, 2024-09-14T06:16:45.759 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.760 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.760 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.760 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.760 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.760 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.760 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4704 2024-09-14T06:16:45.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.760 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.943845+0000", 2024-09-14T06:16:45.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.943845+0000", 2024-09-14T06:16:45.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.943846+0000", 2024-09-14T06:16:45.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2049999999999999e-06 2024-09-14T06:16:45.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.943851+0000", 2024-09-14T06:16:45.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2909999999999998e-06 2024-09-14T06:16:45.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944219+0000", 2024-09-14T06:16:45.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036789100000000002 2024-09-14T06:16:45.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944232+0000", 2024-09-14T06:16:45.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2564e-05 2024-09-14T06:16:45.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944331+0000", 2024-09-14T06:16:45.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8628999999999994e-05 2024-09-14T06:16:45.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944481+0000", 2024-09-14T06:16:45.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015043700000000001 2024-09-14T06:16:45.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:45.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.944672+0000", 2024-09-14T06:16:45.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000191237 2024-09-14T06:16:45.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:45.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.945897+0000", 2024-09-14T06:16:45.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0012252020000000001 2024-09-14T06:16:45.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:45.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.946788+0000", 2024-09-14T06:16:45.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000890134 2024-09-14T06:16:45.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.946894+0000", 2024-09-14T06:16:45.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000106062 2024-09-14T06:16:45.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.946984+0000", 2024-09-14T06:16:45.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0823999999999996e-05 2024-09-14T06:16:45.765 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.765 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.765 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.765 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4873 3.e 3:76656e66:::rbd_data.1152bd92e04f.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=2b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:45.765 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:45.765 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 17.537979961000001, 2024-09-14T06:16:45.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048796120000000002, 2024-09-14T06:16:45.765 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.766 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.766 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.766 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.766 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.766 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.766 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4873 2024-09-14T06:16:45.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.766 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:45.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:45.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193651+0000", 2024-09-14T06:16:45.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:16:45.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193663+0000", 2024-09-14T06:16:45.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2047e-05 2024-09-14T06:16:45.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193692+0000", 2024-09-14T06:16:45.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9346e-05 2024-09-14T06:16:45.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193708+0000", 2024-09-14T06:16:45.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6123000000000001e-05 2024-09-14T06:16:45.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193798+0000", 2024-09-14T06:16:45.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9909999999999998e-05 2024-09-14T06:16:45.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193863+0000", 2024-09-14T06:16:45.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4583000000000002e-05 2024-09-14T06:16:45.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:45.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.194717+0000", 2024-09-14T06:16:45.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00085473299999999999 2024-09-14T06:16:45.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:45.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198400+0000", 2024-09-14T06:16:45.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036830890000000001 2024-09-14T06:16:45.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:45.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198461+0000", 2024-09-14T06:16:45.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0844000000000002e-05 2024-09-14T06:16:45.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198496+0000", 2024-09-14T06:16:45.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4820000000000002e-05 2024-09-14T06:16:45.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198527+0000", 2024-09-14T06:16:45.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1229999999999997e-05 2024-09-14T06:16:45.771 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.771 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.771 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.771 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:6925 3.5 3:a15e9af1:::rbd_data.1152f87cbac6.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:16:45.771 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:45.771 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.616848423, 2024-09-14T06:16:45.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0049096490000000003, 2024-09-14T06:16:45.771 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.771 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.772 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.772 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.772 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.772 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.772 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 6925 2024-09-14T06:16:45.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.772 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:45.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:45.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114780+0000", 2024-09-14T06:16:45.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.09e-07 2024-09-14T06:16:45.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114784+0000", 2024-09-14T06:16:45.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4669999999999998e-06 2024-09-14T06:16:45.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114799+0000", 2024-09-14T06:16:45.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4491000000000001e-05 2024-09-14T06:16:45.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114810+0000", 2024-09-14T06:16:45.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1243000000000001e-05 2024-09-14T06:16:45.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115515+0000", 2024-09-14T06:16:45.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00070521899999999998 2024-09-14T06:16:45.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115610+0000", 2024-09-14T06:16:45.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4260999999999998e-05 2024-09-14T06:16:45.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:45.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115742+0000", 2024-09-14T06:16:45.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000132414 2024-09-14T06:16:45.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:45.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.116650+0000", 2024-09-14T06:16:45.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00090814900000000002 2024-09-14T06:16:45.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:45.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119568+0000", 2024-09-14T06:16:45.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029178440000000002 2024-09-14T06:16:45.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119636+0000", 2024-09-14T06:16:45.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8365e-05 2024-09-14T06:16:45.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119689+0000", 2024-09-14T06:16:45.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2386999999999998e-05 2024-09-14T06:16:45.777 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.777 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.777 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.778 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:7151 3.e 3:7793b17d:::rbd_data.1152f87cbac6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:16:45.778 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:45.778 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.402030633000001, 2024-09-14T06:16:45.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.029556783, 2024-09-14T06:16:45.778 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.778 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.778 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.778 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:45.778 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:45.778 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:45.778 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 7151 2024-09-14T06:16:45.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.778 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:45.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:45.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329599+0000", 2024-09-14T06:16:45.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5270000000000001e-06 2024-09-14T06:16:45.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329610+0000", 2024-09-14T06:16:45.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0431000000000001e-05 2024-09-14T06:16:45.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329634+0000", 2024-09-14T06:16:45.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3617000000000001e-05 2024-09-14T06:16:45.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329644+0000", 2024-09-14T06:16:45.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.095e-05 2024-09-14T06:16:45.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329712+0000", 2024-09-14T06:16:45.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7067e-05 2024-09-14T06:16:45.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329827+0000", 2024-09-14T06:16:45.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011498 2024-09-14T06:16:45.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:45.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.334271+0000", 2024-09-14T06:16:45.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044445839999999997 2024-09-14T06:16:45.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:45.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.358967+0000", 2024-09-14T06:16:45.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024696356999999999 2024-09-14T06:16:45.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:45.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359027+0000", 2024-09-14T06:16:45.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0006000000000001e-05 2024-09-14T06:16:45.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359099+0000", 2024-09-14T06:16:45.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1052999999999999e-05 2024-09-14T06:16:45.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359154+0000", 2024-09-14T06:16:45.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5210999999999997e-05 2024-09-14T06:16:45.783 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.783 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.784 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.784 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:7151 3.e e32/29 ondisk, result = 0)", 2024-09-14T06:16:45.784 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:45.784 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.382086199, 2024-09-14T06:16:45.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0096251340000000005, 2024-09-14T06:16:45.784 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.784 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.784 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:45.784 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:45.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:45.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349543+0000", 2024-09-14T06:16:45.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:16:45.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349552+0000", 2024-09-14T06:16:45.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1039999999999996e-06 2024-09-14T06:16:45.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349579+0000", 2024-09-14T06:16:45.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6449999999999999e-05 2024-09-14T06:16:45.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349593+0000", 2024-09-14T06:16:45.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4099e-05 2024-09-14T06:16:45.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.787 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359003+0000", 2024-09-14T06:16:45.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0094096119999999995 2024-09-14T06:16:45.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.787 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359014+0000", 2024-09-14T06:16:45.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0871e-05 2024-09-14T06:16:45.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.787 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359167+0000", 2024-09-14T06:16:45.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015300700000000001 2024-09-14T06:16:45.788 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.788 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.788 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.788 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15516 3.1 e36/29 3:860a8b0d:::rbd_data.115223d7013d.00000000000000af:head v 36'26, mlcod=36'24)", 2024-09-14T06:16:45.788 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:45.788 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.4743318490000004, 2024-09-14T06:16:45.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035459770000000001, 2024-09-14T06:16:45.788 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.788 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.788 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.789 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:45.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:45.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257297+0000", 2024-09-14T06:16:45.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4900000000000004e-07 2024-09-14T06:16:45.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257304+0000", 2024-09-14T06:16:45.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:16:45.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257321+0000", 2024-09-14T06:16:45.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6311e-05 2024-09-14T06:16:45.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.791 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.791 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257329+0000", 2024-09-14T06:16:45.791 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0399999999999993e-06 2024-09-14T06:16:45.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257414+0000", 2024-09-14T06:16:45.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4846999999999996e-05 2024-09-14T06:16:45.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257436+0000", 2024-09-14T06:16:45.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2048999999999998e-05 2024-09-14T06:16:45.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.260796+0000", 2024-09-14T06:16:45.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033607559999999999 2024-09-14T06:16:45.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.260842+0000", 2024-09-14T06:16:45.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5226999999999999e-05 2024-09-14T06:16:45.793 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.793 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.793 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.794 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15613 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'121, mlcod=36'116)", 2024-09-14T06:16:45.794 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:45.794 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.4387233439999996, 2024-09-14T06:16:45.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053619389999999996, 2024-09-14T06:16:45.794 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.794 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.794 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.794 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:45.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:45.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292905+0000", 2024-09-14T06:16:45.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6800000000000009e-07 2024-09-14T06:16:45.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292915+0000", 2024-09-14T06:16:45.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:16:45.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.796 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292927+0000", 2024-09-14T06:16:45.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-09-14T06:16:45.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.796 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292935+0000", 2024-09-14T06:16:45.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2459999999999997e-06 2024-09-14T06:16:45.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.796 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292992+0000", 2024-09-14T06:16:45.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6873999999999998e-05 2024-09-14T06:16:45.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293016+0000", 2024-09-14T06:16:45.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4037e-05 2024-09-14T06:16:45.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298217+0000", 2024-09-14T06:16:45.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052008710000000001 2024-09-14T06:16:45.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298266+0000", 2024-09-14T06:16:45.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9691000000000002e-05 2024-09-14T06:16:45.798 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.798 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.798 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.798 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15615 3.11 e36/29 3:8d07d636:::rbd_data.115223d7013d.0000000000000002:head v 36'122, mlcod=36'116)", 2024-09-14T06:16:45.798 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:45.798 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.4378269849999992, 2024-09-14T06:16:45.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053069229999999998, 2024-09-14T06:16:45.798 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.799 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.799 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.799 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:45.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:45.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.800 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:45.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5799999999999998e-07 2024-09-14T06:16:45.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.800 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294003+0000", 2024-09-14T06:16:45.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020149600000000001 2024-09-14T06:16:45.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.800 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294016+0000", 2024-09-14T06:16:45.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3424999999999999e-05 2024-09-14T06:16:45.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294024+0000", 2024-09-14T06:16:45.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5240000000000003e-06 2024-09-14T06:16:45.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294673+0000", 2024-09-14T06:16:45.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064906700000000003 2024-09-14T06:16:45.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294698+0000", 2024-09-14T06:16:45.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5301999999999999e-05 2024-09-14T06:16:45.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299064+0000", 2024-09-14T06:16:45.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043652170000000002 2024-09-14T06:16:45.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299108+0000", 2024-09-14T06:16:45.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4033999999999999e-05 2024-09-14T06:16:45.802 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.802 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.802 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.803 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15619 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'124, mlcod=36'119)", 2024-09-14T06:16:45.803 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:45.803 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.4337730369999999, 2024-09-14T06:16:45.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061545389999999997, 2024-09-14T06:16:45.803 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.803 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.803 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.803 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:45.804 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.804 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.804 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:45.804 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.804 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.804 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:45.804 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3399999999999998e-07 2024-09-14T06:16:45.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.805 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.805 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297861+0000", 2024-09-14T06:16:45.805 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7640000000000002e-06 2024-09-14T06:16:45.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.805 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.805 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297876+0000", 2024-09-14T06:16:45.805 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4881e-05 2024-09-14T06:16:45.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.805 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.805 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297884+0000", 2024-09-14T06:16:45.806 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.588e-06 2024-09-14T06:16:45.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.806 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.806 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.806 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298294+0000", 2024-09-14T06:16:45.806 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00041011200000000002 2024-09-14T06:16:45.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.806 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.806 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.806 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298317+0000", 2024-09-14T06:16:45.806 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3417999999999998e-05 2024-09-14T06:16:45.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.806 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.806 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.807 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303972+0000", 2024-09-14T06:16:45.807 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056548680000000004 2024-09-14T06:16:45.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.807 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.807 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304009+0000", 2024-09-14T06:16:45.807 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7073999999999998e-05 2024-09-14T06:16:45.807 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.807 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.807 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.807 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15621 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'125, mlcod=36'119)", 2024-09-14T06:16:45.808 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:45.808 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.4322620199999996, 2024-09-14T06:16:45.808 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0069496169999999999, 2024-09-14T06:16:45.808 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.808 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.808 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.808 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.808 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.808 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.808 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:45.808 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.808 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.809 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.809 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:45.809 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.809 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.809 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.809 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:45.809 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2200000000000003e-07 2024-09-14T06:16:45.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.809 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.809 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.809 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299372+0000", 2024-09-14T06:16:45.809 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3850000000000001e-06 2024-09-14T06:16:45.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.810 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.810 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.810 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299387+0000", 2024-09-14T06:16:45.810 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4905e-05 2024-09-14T06:16:45.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.810 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.810 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.810 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299393+0000", 2024-09-14T06:16:45.810 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9090000000000003e-06 2024-09-14T06:16:45.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.810 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.810 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.810 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303591+0000", 2024-09-14T06:16:45.811 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041976640000000003 2024-09-14T06:16:45.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.811 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.811 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303619+0000", 2024-09-14T06:16:45.811 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7651e-05 2024-09-14T06:16:45.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.811 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.811 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306287+0000", 2024-09-14T06:16:45.811 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026686689999999998 2024-09-14T06:16:45.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.811 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.812 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306315+0000", 2024-09-14T06:16:45.812 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7712000000000002e-05 2024-09-14T06:16:45.812 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.812 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.812 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.812 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.812 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15623 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'126, mlcod=36'119)", 2024-09-14T06:16:45.812 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:45.812 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.4289552069999996, 2024-09-14T06:16:45.812 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003658859, 2024-09-14T06:16:45.812 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.812 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.813 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.813 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.813 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.813 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.813 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:45.813 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.813 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.813 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.813 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:45.813 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.813 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.814 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.814 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302673+0000", 2024-09-14T06:16:45.814 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4199999999999995e-07 2024-09-14T06:16:45.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.814 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.814 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302679+0000", 2024-09-14T06:16:45.814 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7599999999999999e-06 2024-09-14T06:16:45.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.814 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.814 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302689+0000", 2024-09-14T06:16:45.814 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0335e-05 2024-09-14T06:16:45.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.815 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.815 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302697+0000", 2024-09-14T06:16:45.815 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3420000000000002e-06 2024-09-14T06:16:45.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.815 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.815 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304047+0000", 2024-09-14T06:16:45.815 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0013503530000000001 2024-09-14T06:16:45.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.815 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.815 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304059+0000", 2024-09-14T06:16:45.816 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2285e-05 2024-09-14T06:16:45.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.816 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.816 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.816 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306318+0000", 2024-09-14T06:16:45.816 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022586580000000002 2024-09-14T06:16:45.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.816 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.816 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.816 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306331+0000", 2024-09-14T06:16:45.816 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3383999999999999e-05 2024-09-14T06:16:45.816 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.816 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.817 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.817 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.817 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.817 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15626 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'127, mlcod=36'120)", 2024-09-14T06:16:45.817 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:45.817 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.4285908119999995, 2024-09-14T06:16:45.817 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038671249999999999, 2024-09-14T06:16:45.817 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.817 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.817 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.817 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.817 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.817 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.818 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:45.818 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.818 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.818 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.818 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:45.818 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.818 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.818 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.818 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:45.818 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0100000000000005e-07 2024-09-14T06:16:45.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.818 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.819 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.819 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303043+0000", 2024-09-14T06:16:45.819 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.293e-06 2024-09-14T06:16:45.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.819 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.819 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.819 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303072+0000", 2024-09-14T06:16:45.819 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9135000000000001e-05 2024-09-14T06:16:45.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.819 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.819 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.819 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303081+0000", 2024-09-14T06:16:45.819 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7080000000000003e-06 2024-09-14T06:16:45.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.820 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.820 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.820 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.305965+0000", 2024-09-14T06:16:45.820 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028841769999999999 2024-09-14T06:16:45.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.820 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.820 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.820 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.305993+0000", 2024-09-14T06:16:45.820 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8121000000000001e-05 2024-09-14T06:16:45.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.820 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.820 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.820 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306861+0000", 2024-09-14T06:16:45.821 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00086771099999999996 2024-09-14T06:16:45.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.821 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.821 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.821 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306904+0000", 2024-09-14T06:16:45.821 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3378999999999999e-05 2024-09-14T06:16:45.821 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.821 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.821 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.821 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.821 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:16521 3.e e36/34 3:77c8e3e5:::rbd_data.1152fc09e7e6.0000000000000003:head v 36'83, mlcod=36'81)", 2024-09-14T06:16:45.821 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:45.822 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.8056087200000004, 2024-09-14T06:16:45.822 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044948899999999996, 2024-09-14T06:16:45.822 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.822 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.822 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.822 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.822 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.822 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:45.822 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.822 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.822 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:45.823 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.823 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.823 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926020+0000", 2024-09-14T06:16:45.823 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.545e-06 2024-09-14T06:16:45.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.823 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.823 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927192+0000", 2024-09-14T06:16:45.823 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011711919999999999 2024-09-14T06:16:45.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.824 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.824 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927215+0000", 2024-09-14T06:16:45.824 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3677e-05 2024-09-14T06:16:45.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.824 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.824 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927226+0000", 2024-09-14T06:16:45.824 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0665e-05 2024-09-14T06:16:45.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.824 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.824 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927314+0000", 2024-09-14T06:16:45.824 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8314999999999997e-05 2024-09-14T06:16:45.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.825 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.825 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927344+0000", 2024-09-14T06:16:45.825 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9941e-05 2024-09-14T06:16:45.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.825 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.825 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.930411+0000", 2024-09-14T06:16:45.825 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030670649999999999 2024-09-14T06:16:45.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.825 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.825 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.930514+0000", 2024-09-14T06:16:45.825 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010249 2024-09-14T06:16:45.826 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.826 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.826 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.826 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.826 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:18404 3.11 e40/29 3:88c7e550:::rbd_data.115273da7901.0000000000000003:head v 40'141, mlcod=39'138)", 2024-09-14T06:16:45.826 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:42.116636+0000", 2024-09-14T06:16:45.826 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.614991785, 2024-09-14T06:16:45.826 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003180619, 2024-09-14T06:16:45.826 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.826 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.826 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.826 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.827 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.827 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.116636+0000", 2024-09-14T06:16:45.827 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.827 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.827 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.116636+0000", 2024-09-14T06:16:45.827 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.827 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.827 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.116637+0000", 2024-09-14T06:16:45.827 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9699999999999995e-07 2024-09-14T06:16:45.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.828 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.828 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.828 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.116649+0000", 2024-09-14T06:16:45.828 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2346e-05 2024-09-14T06:16:45.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.828 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.828 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.828 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.116677+0000", 2024-09-14T06:16:45.828 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8297999999999999e-05 2024-09-14T06:16:45.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.828 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.828 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.828 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.116694+0000", 2024-09-14T06:16:45.829 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7336000000000001e-05 2024-09-14T06:16:45.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.829 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.829 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.116779+0000", 2024-09-14T06:16:45.829 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4571000000000001e-05 2024-09-14T06:16:45.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.829 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.829 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.116800+0000", 2024-09-14T06:16:45.829 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1435999999999999e-05 2024-09-14T06:16:45.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.829 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.830 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.119590+0000", 2024-09-14T06:16:45.830 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027893039999999998 2024-09-14T06:16:45.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.830 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.830 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.119816+0000", 2024-09-14T06:16:45.830 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022663100000000001 2024-09-14T06:16:45.830 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.830 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.830 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.830 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:18539 3.1e e40/29 3:7e28f783:::rbd_data.1152227be94f.0000000000000000:head v 40'59, mlcod=40'57)", 2024-09-14T06:16:45.830 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:16:45.831 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.448586669, 2024-09-14T06:16:45.831 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0059097430000000003, 2024-09-14T06:16:45.831 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:45.831 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:45.831 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:45.831 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:45.831 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.831 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:45.831 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:16:45.831 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.831 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.831 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:45.831 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:16:45.832 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:45.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.832 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.832 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:45.832 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283043+0000", 2024-09-14T06:16:45.832 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4499999999999998e-06 2024-09-14T06:16:45.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.832 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.832 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:45.832 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285878+0000", 2024-09-14T06:16:45.832 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028348039999999998 2024-09-14T06:16:45.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.832 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.833 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:45.833 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285908+0000", 2024-09-14T06:16:45.833 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9756000000000001e-05 2024-09-14T06:16:45.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.833 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:45.833 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285922+0000", 2024-09-14T06:16:45.833 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.43e-05 2024-09-14T06:16:45.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.833 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:45.833 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.286030+0000", 2024-09-14T06:16:45.833 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000107954 2024-09-14T06:16:45.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.834 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:45.834 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.286058+0000", 2024-09-14T06:16:45.834 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7796e-05 2024-09-14T06:16:45.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.834 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:45.834 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.288888+0000", 2024-09-14T06:16:45.834 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028295799999999999 2024-09-14T06:16:45.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:45.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:45.834 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:45.834 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.288951+0000", 2024-09-14T06:16:45.835 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3102999999999995e-05 2024-09-14T06:16:45.835 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.835 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.835 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.835 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:45.835 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:45.835 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:45.836 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-14T06:16:45.837 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:16:45.837 INFO:teuthology.orchestra.run.smithi039.stdout:1692 read 0x34a867 thru 0x352392 (0x7b2c bytes) 2024-09-14T06:16:45.837 INFO:teuthology.orchestra.run.smithi039.stdout:1693 trunc from 0xe3d2cc4 to 0xb946a47 2024-09-14T06:16:45.838 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:45.835+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:45.862 INFO:teuthology.orchestra.run.smithi039.stdout:1694 writesame 0x6dec7a1 thru 0x6df132c data_size 0xa(0x4b8c bytes) 2024-09-14T06:16:45.876 INFO:teuthology.orchestra.run.smithi039.stdout:1696 writesame 0x8e5043d thru 0x8e59fe5 data_size 0x25(0x9ba9 bytes) 2024-09-14T06:16:45.897 INFO:teuthology.orchestra.run.smithi039.stdout:1697 write 0xdda56e8 thru 0xdda6287 (0xba0 bytes) 2024-09-14T06:16:45.900 INFO:teuthology.orchestra.run.smithi039.stdout:1699 clone 11 order 22 su 131072 sc 9 2024-09-14T06:16:45.937 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:45.935+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:45.986 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:45.986 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:45.986 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:45.986 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:45.999 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:16:46.043 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:46.043+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:46.140 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:46.139+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:46.166 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:46.166 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:46.166 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:46.166 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:46.166 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:46.179 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:16:46.271 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:46.271+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:46.342 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:46.343+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:46.345 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:46.345 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11217 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 925696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.352503187, 2024-09-14T06:16:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049306313999999997, 2024-09-14T06:16:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11217 2024-09-14T06:16:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.29e-07 2024-09-14T06:16:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991345+0000", 2024-09-14T06:16:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1480000000000001e-06 2024-09-14T06:16:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991349+0000", 2024-09-14T06:16:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9840000000000003e-06 2024-09-14T06:16:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991359+0000", 2024-09-14T06:16:46.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4050000000000006e-06 2024-09-14T06:16:46.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.030826+0000", 2024-09-14T06:16:46.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.039467214 2024-09-14T06:16:46.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.030894+0000", 2024-09-14T06:16:46.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8400999999999998e-05 2024-09-14T06:16:46.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040650+0000", 2024-09-14T06:16:46.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0097559329999999996 2024-09-14T06:16:46.350 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.350 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.350 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.350 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11262 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 81920~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.350 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:46.351 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.352317150999999, 2024-09-14T06:16:46.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049588879000000002, 2024-09-14T06:16:46.351 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.351 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.351 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.351 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.351 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.351 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.351 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11262 2024-09-14T06:16:46.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.351 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:46.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:46.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:46.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9599999999999999e-07 2024-09-14T06:16:46.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991533+0000", 2024-09-14T06:16:46.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.362e-06 2024-09-14T06:16:46.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991539+0000", 2024-09-14T06:16:46.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:16:46.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991544+0000", 2024-09-14T06:16:46.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:16:46.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040755+0000", 2024-09-14T06:16:46.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049210938000000003 2024-09-14T06:16:46.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040833+0000", 2024-09-14T06:16:46.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7812999999999995e-05 2024-09-14T06:16:46.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041119+0000", 2024-09-14T06:16:46.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0002855 2024-09-14T06:16:46.355 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.355 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.355 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.355 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11263 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.355 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:46.355 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.352294372999999, 2024-09-14T06:16:46.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049944574999999998, 2024-09-14T06:16:46.356 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.356 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.356 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.356 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.356 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.356 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.356 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11263 2024-09-14T06:16:46.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.356 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.356 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:46.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:46.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:46.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.51e-07 2024-09-14T06:16:46.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991560+0000", 2024-09-14T06:16:46.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6719999999999998e-06 2024-09-14T06:16:46.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991567+0000", 2024-09-14T06:16:46.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1470000000000004e-06 2024-09-14T06:16:46.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991572+0000", 2024-09-14T06:16:46.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.237e-06 2024-09-14T06:16:46.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041149+0000", 2024-09-14T06:16:46.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049576988000000002 2024-09-14T06:16:46.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041277+0000", 2024-09-14T06:16:46.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000127687 2024-09-14T06:16:46.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041497+0000", 2024-09-14T06:16:46.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022039299999999999 2024-09-14T06:16:46.360 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.360 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.360 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.360 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11264 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 327680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.360 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:46.360 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.352268032, 2024-09-14T06:16:46.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050248567000000001, 2024-09-14T06:16:46.360 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.360 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.361 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.361 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.361 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.361 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.361 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11264 2024-09-14T06:16:46.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.361 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:46.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:46.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:46.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.41e-07 2024-09-14T06:16:46.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991581+0000", 2024-09-14T06:16:46.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3740000000000001e-06 2024-09-14T06:16:46.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991585+0000", 2024-09-14T06:16:46.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9400000000000004e-06 2024-09-14T06:16:46.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991587+0000", 2024-09-14T06:16:46.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.694e-06 2024-09-14T06:16:46.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041527+0000", 2024-09-14T06:16:46.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049939349000000001 2024-09-14T06:16:46.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041598+0000", 2024-09-14T06:16:46.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1633999999999999e-05 2024-09-14T06:16:46.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041827+0000", 2024-09-14T06:16:46.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000229235 2024-09-14T06:16:46.365 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.365 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.365 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.365 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.365 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:46.365 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.352254692000001, 2024-09-14T06:16:46.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:16:46.365 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.365 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.366 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.366 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.366 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.366 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:16:46.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.366 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:46.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:46.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:46.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:16:46.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:16:46.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:16:46.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:16:46.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:16:46.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:16:46.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:16:46.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:16:46.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:16:46.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:16:46.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:16:46.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:16:46.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:16:46.370 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.370 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.370 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.370 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.370 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:46.370 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.352241619999999, 2024-09-14T06:16:46.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:16:46.370 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.370 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.370 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.371 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.371 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.371 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.371 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:16:46.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.371 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.371 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:46.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:46.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:16:46.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:16:46.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:16:46.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:16:46.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:16:46.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:16:46.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:16:46.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:16:46.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:16:46.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:16:46.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:16:46.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:16:46.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:16:46.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:16:46.374 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.374 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.374 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.374 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.375 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:46.375 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.352224187999999, 2024-09-14T06:16:46.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:16:46.375 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.375 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.375 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.375 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.375 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.375 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.375 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:16:46.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.376 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:46.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:46.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:46.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:16:46.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:16:46.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:16:46.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:16:46.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:16:46.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.378 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.378 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:16:46.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:16:46.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.378 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.378 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:16:46.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:16:46.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.378 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:16:46.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:16:46.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:16:46.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:16:46.379 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.379 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.379 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.379 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.380 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:46.380 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.352203082999999, 2024-09-14T06:16:46.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:16:46.380 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.380 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.380 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.380 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.380 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.380 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.380 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:16:46.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.381 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:46.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:46.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.382 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:46.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:16:46.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.382 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:16:46.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:16:46.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:16:46.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:16:46.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:16:46.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:16:46.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:16:46.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:16:46.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:16:46.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:16:46.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:16:46.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:16:46.384 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.385 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.385 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.385 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.385 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:46.385 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.352181975000001, 2024-09-14T06:16:46.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:16:46.385 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.385 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.385 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.385 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.385 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.386 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.386 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:16:46.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.386 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.386 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.386 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:46.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.386 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.386 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:46.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.387 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:46.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:16:46.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.387 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:16:46.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:16:46.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:16:46.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:16:46.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:16:46.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:16:46.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:16:46.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:16:46.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:16:46.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:16:46.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:16:46.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:16:46.389 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.389 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.390 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.390 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13590 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 491520~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.390 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:46.390 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.206207093, 2024-09-14T06:16:46.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042553466999999998, 2024-09-14T06:16:46.390 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.390 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.390 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.390 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.390 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.390 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.390 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13590 2024-09-14T06:16:46.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.391 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.391 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.391 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:46.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.391 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.391 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:46.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:46.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.34e-07 2024-09-14T06:16:46.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137642+0000", 2024-09-14T06:16:46.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0150000000000002e-06 2024-09-14T06:16:46.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137649+0000", 2024-09-14T06:16:46.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4010000000000002e-06 2024-09-14T06:16:46.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137653+0000", 2024-09-14T06:16:46.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:16:46.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.179985+0000", 2024-09-14T06:16:46.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042331580000000001 2024-09-14T06:16:46.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180076+0000", 2024-09-14T06:16:46.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0796000000000003e-05 2024-09-14T06:16:46.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180193+0000", 2024-09-14T06:16:46.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000117471 2024-09-14T06:16:46.394 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.394 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.394 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.394 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13592 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 606208~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.394 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:46.394 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.206182524999999, 2024-09-14T06:16:46.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042709440000000001, 2024-09-14T06:16:46.395 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.395 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.395 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.395 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.395 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.395 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.395 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13592 2024-09-14T06:16:46.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.395 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:46.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.396 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.396 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:46.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.396 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.396 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137665+0000", 2024-09-14T06:16:46.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4299999999999998e-07 2024-09-14T06:16:46.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137667+0000", 2024-09-14T06:16:46.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5780000000000001e-06 2024-09-14T06:16:46.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137675+0000", 2024-09-14T06:16:46.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0870000000000001e-06 2024-09-14T06:16:46.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137680+0000", 2024-09-14T06:16:46.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4999999999999999e-06 2024-09-14T06:16:46.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180223+0000", 2024-09-14T06:16:46.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042542808000000001 2024-09-14T06:16:46.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180294+0000", 2024-09-14T06:16:46.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1497999999999999e-05 2024-09-14T06:16:46.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180374+0000", 2024-09-14T06:16:46.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9525999999999997e-05 2024-09-14T06:16:46.399 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.399 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.399 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.399 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13594 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 729088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.399 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:46.399 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.206158961, 2024-09-14T06:16:46.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042830604000000001, 2024-09-14T06:16:46.399 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.399 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.399 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.399 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.399 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.399 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.400 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13594 2024-09-14T06:16:46.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.400 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.400 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.400 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:46.400 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.400 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.400 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:46.400 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.401 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.401 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:46.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0699999999999998e-07 2024-09-14T06:16:46.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.401 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.401 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137690+0000", 2024-09-14T06:16:46.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.012e-06 2024-09-14T06:16:46.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.401 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.401 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137698+0000", 2024-09-14T06:16:46.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4379999999999998e-06 2024-09-14T06:16:46.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137703+0000", 2024-09-14T06:16:46.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1250000000000001e-06 2024-09-14T06:16:46.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180404+0000", 2024-09-14T06:16:46.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042701465000000001 2024-09-14T06:16:46.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180456+0000", 2024-09-14T06:16:46.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1227999999999998e-05 2024-09-14T06:16:46.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180519+0000", 2024-09-14T06:16:46.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-09-14T06:16:46.403 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.403 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.403 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.403 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13595 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 851968~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.404 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:46.404 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.206136455999999, 2024-09-14T06:16:46.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043012433000000003, 2024-09-14T06:16:46.404 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.404 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.404 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.404 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.404 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.404 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.404 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13595 2024-09-14T06:16:46.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.404 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:46.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:46.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137711+0000", 2024-09-14T06:16:46.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:16:46.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137713+0000", 2024-09-14T06:16:46.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5170000000000002e-06 2024-09-14T06:16:46.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137720+0000", 2024-09-14T06:16:46.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-09-14T06:16:46.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137726+0000", 2024-09-14T06:16:46.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9580000000000004e-06 2024-09-14T06:16:46.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180551+0000", 2024-09-14T06:16:46.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042825611999999999 2024-09-14T06:16:46.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180623+0000", 2024-09-14T06:16:46.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1673000000000003e-05 2024-09-14T06:16:46.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180723+0000", 2024-09-14T06:16:46.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.9928999999999998e-05 2024-09-14T06:16:46.408 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.408 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.408 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.408 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13597 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 974848~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.408 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:46.408 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.206113570999999, 2024-09-14T06:16:46.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043180442999999999, 2024-09-14T06:16:46.408 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.408 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.408 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.409 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.409 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.409 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.409 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13597 2024-09-14T06:16:46.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.409 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:46.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:46.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137734+0000", 2024-09-14T06:16:46.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4499999999999997e-07 2024-09-14T06:16:46.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137736+0000", 2024-09-14T06:16:46.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0470000000000001e-06 2024-09-14T06:16:46.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137742+0000", 2024-09-14T06:16:46.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2269999999999998e-06 2024-09-14T06:16:46.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137747+0000", 2024-09-14T06:16:46.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6670000000000004e-06 2024-09-14T06:16:46.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180753+0000", 2024-09-14T06:16:46.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043006132000000002 2024-09-14T06:16:46.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180820+0000", 2024-09-14T06:16:46.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7516999999999997e-05 2024-09-14T06:16:46.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180914+0000", 2024-09-14T06:16:46.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3407999999999997e-05 2024-09-14T06:16:46.412 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.412 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.412 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.413 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13684 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 999424~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.413 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:46.413 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.203881028, 2024-09-14T06:16:46.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042643702999999998, 2024-09-14T06:16:46.413 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.413 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.413 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.413 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13684 2024-09-14T06:16:46.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.413 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:46.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:46.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139967+0000", 2024-09-14T06:16:46.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1200000000000003e-07 2024-09-14T06:16:46.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141080+0000", 2024-09-14T06:16:46.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011133029999999999 2024-09-14T06:16:46.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141084+0000", 2024-09-14T06:16:46.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.544e-06 2024-09-14T06:16:46.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141088+0000", 2024-09-14T06:16:46.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0280000000000002e-06 2024-09-14T06:16:46.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182457+0000", 2024-09-14T06:16:46.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041368690999999999 2024-09-14T06:16:46.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182528+0000", 2024-09-14T06:16:46.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0881000000000001e-05 2024-09-14T06:16:46.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182610+0000", 2024-09-14T06:16:46.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1644000000000006e-05 2024-09-14T06:16:46.417 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.417 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.417 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.417 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13721 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 532480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.417 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:46.417 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.202580294000001, 2024-09-14T06:16:46.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042577526999999997, 2024-09-14T06:16:46.417 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.417 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.417 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.418 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.418 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.418 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.418 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13721 2024-09-14T06:16:46.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.418 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:46.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:46.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:46.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2999999999999999e-07 2024-09-14T06:16:46.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141268+0000", 2024-09-14T06:16:46.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0130000000000001e-06 2024-09-14T06:16:46.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141271+0000", 2024-09-14T06:16:46.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0970000000000002e-06 2024-09-14T06:16:46.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141274+0000", 2024-09-14T06:16:46.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5809999999999999e-06 2024-09-14T06:16:46.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183522+0000", 2024-09-14T06:16:46.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042248591000000002 2024-09-14T06:16:46.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183691+0000", 2024-09-14T06:16:46.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016885399999999999 2024-09-14T06:16:46.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183844+0000", 2024-09-14T06:16:46.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015316099999999999 2024-09-14T06:16:46.421 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.422 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.422 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.422 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13722 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 655360~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.422 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:46.422 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.202569992000001, 2024-09-14T06:16:46.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042741610999999999, 2024-09-14T06:16:46.422 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.422 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.422 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.422 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.422 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.422 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.423 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13722 2024-09-14T06:16:46.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.423 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:46.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:46.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:46.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:16:46.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141278+0000", 2024-09-14T06:16:46.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.066e-06 2024-09-14T06:16:46.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141282+0000", 2024-09-14T06:16:46.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7100000000000001e-06 2024-09-14T06:16:46.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.425 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141285+0000", 2024-09-14T06:16:46.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.554e-06 2024-09-14T06:16:46.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.425 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183824+0000", 2024-09-14T06:16:46.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042539043999999998 2024-09-14T06:16:46.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.426 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183906+0000", 2024-09-14T06:16:46.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2298000000000004e-05 2024-09-14T06:16:46.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.426 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184019+0000", 2024-09-14T06:16:46.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000112724 2024-09-14T06:16:46.426 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.426 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.426 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.427 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13723 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 778240~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.427 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:46.427 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.202559022999999, 2024-09-14T06:16:46.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042995901000000003, 2024-09-14T06:16:46.427 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.427 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.427 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.427 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.427 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.427 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.427 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13723 2024-09-14T06:16:46.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.428 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:46.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:46.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:46.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9799999999999999e-07 2024-09-14T06:16:46.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141289+0000", 2024-09-14T06:16:46.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0589999999999999e-06 2024-09-14T06:16:46.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141292+0000", 2024-09-14T06:16:46.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.208e-06 2024-09-14T06:16:46.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.430 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.430 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.430 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141295+0000", 2024-09-14T06:16:46.430 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5160000000000001e-06 2024-09-14T06:16:46.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.430 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.430 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.430 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184069+0000", 2024-09-14T06:16:46.430 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042774176999999997 2024-09-14T06:16:46.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.430 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.430 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.431 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184169+0000", 2024-09-14T06:16:46.431 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000100073 2024-09-14T06:16:46.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.431 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.431 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184284+0000", 2024-09-14T06:16:46.431 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011457 2024-09-14T06:16:46.431 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.431 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.431 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.431 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13724 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 901120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.431 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:16:46.431 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.202548629000001, 2024-09-14T06:16:46.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043197179000000002, 2024-09-14T06:16:46.432 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.432 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.432 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.432 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.432 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.432 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.432 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13724 2024-09-14T06:16:46.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.432 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:16:46.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:16:46.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141299+0000", 2024-09-14T06:16:46.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.03e-07 2024-09-14T06:16:46.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141300+0000", 2024-09-14T06:16:46.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0389999999999999e-06 2024-09-14T06:16:46.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.434 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.434 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141303+0000", 2024-09-14T06:16:46.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0479999999999999e-06 2024-09-14T06:16:46.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.434 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.434 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141305+0000", 2024-09-14T06:16:46.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5739999999999998e-06 2024-09-14T06:16:46.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.435 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.435 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.435 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184317+0000", 2024-09-14T06:16:46.435 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043011701999999999 2024-09-14T06:16:46.435 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.435 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.435 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.435 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184394+0000", 2024-09-14T06:16:46.435 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7324999999999996e-05 2024-09-14T06:16:46.435 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.435 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.435 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.435 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184496+0000", 2024-09-14T06:16:46.435 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010118799999999999 2024-09-14T06:16:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13726 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 1024000~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 10.202538356, 2024-09-14T06:16:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043397185999999997, 2024-09-14T06:16:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:46.437 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.437 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13726 2024-09-14T06:16:46.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.437 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:46.437 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.437 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:46.437 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:46.437 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.437 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.437 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:46.437 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:46.437 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:46.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.438 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:46.438 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:46.438 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:16:46.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.438 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:46.438 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141310+0000", 2024-09-14T06:16:46.438 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:16:46.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.438 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:46.438 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141313+0000", 2024-09-14T06:16:46.439 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0309999999999999e-06 2024-09-14T06:16:46.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.439 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.439 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.439 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141315+0000", 2024-09-14T06:16:46.439 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.52e-06 2024-09-14T06:16:46.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.439 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.439 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:46.439 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184527+0000", 2024-09-14T06:16:46.439 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043211855 2024-09-14T06:16:46.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.439 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.440 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:46.440 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184606+0000", 2024-09-14T06:16:46.440 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8287000000000004e-05 2024-09-14T06:16:46.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:46.440 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:46.440 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:46.440 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184706+0000", 2024-09-14T06:16:46.440 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010021700000000001 2024-09-14T06:16:46.440 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.440 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.440 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.440 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:46.440 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:46.441 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:46.442 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:16:46.442 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:46.439+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:46.485 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:16:46.543 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:46.543+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:46.576 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:46.577 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:46.577 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:16:46.577 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:46.592 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:16:46.603 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone10 from 0xdda6288 (overlap 0xb946a47) to 0x5ca3523 2024-09-14T06:16:46.678 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:46.679+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:46.746 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:46.743+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:46.748 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-14T06:16:46.756 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:46.756 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:46.756 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:16:46.757 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:16:46.757 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:46.771 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:16:46.847 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:46.847+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:46.946 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:46.946 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:16:46.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:16:46.947 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:16:46.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.947 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13229 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.947 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:16:46.947 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.845825204, 2024-09-14T06:16:46.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081798868999999996, 2024-09-14T06:16:46.947 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:46.947 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:46.947 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:46.947 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:46.947 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:46.947 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.947 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13229 2024-09-14T06:16:46.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.948 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:46.948 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.948 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:46.948 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:16:46.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.948 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.948 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:46.948 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:16:46.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.949 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:46.949 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098710+0000", 2024-09-14T06:16:46.949 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8199999999999999e-07 2024-09-14T06:16:46.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.949 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:46.949 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098714+0000", 2024-09-14T06:16:46.949 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:16:46.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.949 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:46.949 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098727+0000", 2024-09-14T06:16:46.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2520999999999999e-05 2024-09-14T06:16:46.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098749+0000", 2024-09-14T06:16:46.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2773999999999999e-05 2024-09-14T06:16:46.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:46.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160234+0000", 2024-09-14T06:16:46.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.061484430999999999 2024-09-14T06:16:46.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:46.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160281+0000", 2024-09-14T06:16:46.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7437e-05 2024-09-14T06:16:46.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:46.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160569+0000", 2024-09-14T06:16:46.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028788700000000002 2024-09-14T06:16:46.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:46.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163121+0000", 2024-09-14T06:16:46.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025514090000000001 2024-09-14T06:16:46.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.952 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:46.952 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180410+0000", 2024-09-14T06:16:46.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.017289098999999999 2024-09-14T06:16:46.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.952 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:46.952 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180472+0000", 2024-09-14T06:16:46.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2547999999999998e-05 2024-09-14T06:16:46.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.952 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:46.953 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180508+0000", 2024-09-14T06:16:46.953 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5534000000000002e-05 2024-09-14T06:16:46.953 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:46.953 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:46.953 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:46.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.953 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.953 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13230 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.953 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:16:46.953 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.845744002, 2024-09-14T06:16:46.953 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081851558000000005, 2024-09-14T06:16:46.953 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:46.953 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:46.954 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:46.954 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:46.954 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:46.954 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.954 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13230 2024-09-14T06:16:46.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.954 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:46.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:46.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:16:46.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:46.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:16:46.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:46.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098791+0000", 2024-09-14T06:16:46.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8800000000000001e-07 2024-09-14T06:16:46.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:46.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098795+0000", 2024-09-14T06:16:46.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.391e-06 2024-09-14T06:16:46.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:46.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098804+0000", 2024-09-14T06:16:46.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3440000000000001e-06 2024-09-14T06:16:46.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098810+0000", 2024-09-14T06:16:46.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2339999999999999e-06 2024-09-14T06:16:46.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.957 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:46.957 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160930+0000", 2024-09-14T06:16:46.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062120018999999999 2024-09-14T06:16:46.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.957 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:46.957 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161017+0000", 2024-09-14T06:16:46.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6868999999999998e-05 2024-09-14T06:16:46.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.957 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:46.957 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161300+0000", 2024-09-14T06:16:46.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028349499999999998 2024-09-14T06:16:46.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.958 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:46.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164612+0000", 2024-09-14T06:16:46.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033119680000000002 2024-09-14T06:16:46.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.958 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:46.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180559+0000", 2024-09-14T06:16:46.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015946612999999998 2024-09-14T06:16:46.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.958 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:46.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180605+0000", 2024-09-14T06:16:46.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5890000000000003e-05 2024-09-14T06:16:46.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:46.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180642+0000", 2024-09-14T06:16:46.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:16:46.959 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:46.959 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:46.959 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:46.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.959 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13231 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.959 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:16:46.959 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.845481588, 2024-09-14T06:16:46.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081750685000000003, 2024-09-14T06:16:46.960 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:46.960 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:46.960 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:46.960 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:46.960 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:46.960 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.960 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13231 2024-09-14T06:16:46.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.960 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:46.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.960 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:46.960 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:16:46.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.961 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.961 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:46.961 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:16:46.961 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.961 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.961 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.961 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:46.961 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099053+0000", 2024-09-14T06:16:46.961 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:16:46.961 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.961 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.961 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:46.961 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099059+0000", 2024-09-14T06:16:46.962 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1789999999999999e-06 2024-09-14T06:16:46.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.962 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.962 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:46.962 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099072+0000", 2024-09-14T06:16:46.962 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3179999999999999e-05 2024-09-14T06:16:46.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.962 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.962 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.962 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099080+0000", 2024-09-14T06:16:46.962 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9030000000000004e-06 2024-09-14T06:16:46.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:46.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161681+0000", 2024-09-14T06:16:46.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062601641999999999 2024-09-14T06:16:46.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:46.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161766+0000", 2024-09-14T06:16:46.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4909999999999998e-05 2024-09-14T06:16:46.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:46.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162109+0000", 2024-09-14T06:16:46.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00034264899999999999 2024-09-14T06:16:46.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:46.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164615+0000", 2024-09-14T06:16:46.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025061979999999998 2024-09-14T06:16:46.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:46.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180718+0000", 2024-09-14T06:16:46.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016103240000000001 2024-09-14T06:16:46.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.965 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.965 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:46.965 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180768+0000", 2024-09-14T06:16:46.965 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9781999999999999e-05 2024-09-14T06:16:46.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.965 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.965 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:46.965 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180803+0000", 2024-09-14T06:16:46.965 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5021999999999998e-05 2024-09-14T06:16:46.965 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:46.965 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:46.965 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:46.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.965 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.965 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13232 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.966 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:16:46.966 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.845439814000001, 2024-09-14T06:16:46.966 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081861981, 2024-09-14T06:16:46.966 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:46.966 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:46.966 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:46.966 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:46.966 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:46.966 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.966 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13232 2024-09-14T06:16:46.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.967 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:46.967 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.967 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:46.967 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:16:46.967 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.967 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.967 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:46.967 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:16:46.967 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.967 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.967 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:46.967 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099095+0000", 2024-09-14T06:16:46.967 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:16:46.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.968 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.968 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:46.968 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099098+0000", 2024-09-14T06:16:46.968 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5420000000000001e-06 2024-09-14T06:16:46.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.968 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.968 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:46.968 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099106+0000", 2024-09-14T06:16:46.968 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.824e-06 2024-09-14T06:16:46.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.968 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.968 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.968 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099112+0000", 2024-09-14T06:16:46.968 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9789999999999998e-06 2024-09-14T06:16:46.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.969 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:46.969 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162576+0000", 2024-09-14T06:16:46.969 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.063463723999999999 2024-09-14T06:16:46.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.969 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:46.969 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162627+0000", 2024-09-14T06:16:46.969 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1530000000000003e-05 2024-09-14T06:16:46.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.969 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:46.969 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162854+0000", 2024-09-14T06:16:46.970 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000226159 2024-09-14T06:16:46.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.970 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.970 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:46.970 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166090+0000", 2024-09-14T06:16:46.970 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032362319999999999 2024-09-14T06:16:46.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.970 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.970 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:46.970 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180877+0000", 2024-09-14T06:16:46.970 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014786726 2024-09-14T06:16:46.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.970 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.970 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:46.971 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180920+0000", 2024-09-14T06:16:46.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3427000000000003e-05 2024-09-14T06:16:46.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.971 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.971 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:46.971 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180956+0000", 2024-09-14T06:16:46.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6195999999999997e-05 2024-09-14T06:16:46.971 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:46.971 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:46.971 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:46.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.971 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.971 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13233 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.971 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:16:46.971 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.845414550999999, 2024-09-14T06:16:46.972 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081924860000000002, 2024-09-14T06:16:46.972 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:46.972 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:46.972 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:46.972 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:46.972 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:46.972 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.972 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13233 2024-09-14T06:16:46.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.972 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:46.972 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.972 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:46.972 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:16:46.972 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.973 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:46.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:16:46.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.973 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:46.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099120+0000", 2024-09-14T06:16:46.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:16:46.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.973 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:46.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099122+0000", 2024-09-14T06:16:46.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:16:46.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.974 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.974 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:46.974 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099126+0000", 2024-09-14T06:16:46.974 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2540000000000003e-06 2024-09-14T06:16:46.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.974 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.974 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.974 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099130+0000", 2024-09-14T06:16:46.974 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4060000000000001e-06 2024-09-14T06:16:46.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.974 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.974 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:46.974 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163169+0000", 2024-09-14T06:16:46.975 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064039415000000002 2024-09-14T06:16:46.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.975 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.975 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:46.975 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163260+0000", 2024-09-14T06:16:46.975 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1038000000000005e-05 2024-09-14T06:16:46.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.975 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.975 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:46.975 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163640+0000", 2024-09-14T06:16:46.975 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037954699999999999 2024-09-14T06:16:46.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.975 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.975 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:46.975 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166093+0000", 2024-09-14T06:16:46.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024533229999999999 2024-09-14T06:16:46.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.976 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:46.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180993+0000", 2024-09-14T06:16:46.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01489997 2024-09-14T06:16:46.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.976 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:46.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181020+0000", 2024-09-14T06:16:46.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6806999999999999e-05 2024-09-14T06:16:46.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.977 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:46.977 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181044+0000", 2024-09-14T06:16:46.977 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4405000000000001e-05 2024-09-14T06:16:46.977 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:46.977 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:46.977 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:46.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.978 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13234 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.978 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:16:46.978 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.845372346, 2024-09-14T06:16:46.978 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082015597999999995, 2024-09-14T06:16:46.978 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:46.978 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:46.978 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:46.978 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:46.978 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:46.978 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.978 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13234 2024-09-14T06:16:46.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.978 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:46.978 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:46.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:16:46.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:46.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:16:46.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:46.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:16:46.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0999999999999998e-07 2024-09-14T06:16:46.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.980 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.980 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:46.980 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099166+0000", 2024-09-14T06:16:46.980 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:16:46.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.980 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.980 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:46.980 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099171+0000", 2024-09-14T06:16:46.980 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6949999999999997e-06 2024-09-14T06:16:46.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.980 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.981 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.981 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099175+0000", 2024-09-14T06:16:46.981 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9029999999999997e-06 2024-09-14T06:16:46.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.981 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.981 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:46.981 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164063+0000", 2024-09-14T06:16:46.981 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064887608999999999 2024-09-14T06:16:46.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.981 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.981 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:46.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164111+0000", 2024-09-14T06:16:46.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7648999999999997e-05 2024-09-14T06:16:46.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.982 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:46.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164351+0000", 2024-09-14T06:16:46.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024072899999999999 2024-09-14T06:16:46.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.982 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:46.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167561+0000", 2024-09-14T06:16:46.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032094179999999999 2024-09-14T06:16:46.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.983 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:46.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181093+0000", 2024-09-14T06:16:46.983 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013532245 2024-09-14T06:16:46.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.983 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:46.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181139+0000", 2024-09-14T06:16:46.983 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6483999999999997e-05 2024-09-14T06:16:46.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.983 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:46.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181177+0000", 2024-09-14T06:16:46.984 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7861000000000002e-05 2024-09-14T06:16:46.984 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:46.984 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:46.984 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:46.984 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.984 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.984 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13235 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.984 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:16:46.984 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.845265072, 2024-09-14T06:16:46.984 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081988088000000001, 2024-09-14T06:16:46.984 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:46.984 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:46.984 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:46.985 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:46.985 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:46.985 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.985 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13235 2024-09-14T06:16:46.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.985 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:46.985 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.985 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:46.985 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:16:46.985 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.985 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.985 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:46.986 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:16:46.986 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.986 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.986 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.986 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:46.986 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099270+0000", 2024-09-14T06:16:46.986 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7400000000000003e-07 2024-09-14T06:16:46.986 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.986 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.986 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:46.986 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099276+0000", 2024-09-14T06:16:46.987 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:16:46.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.987 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.987 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:46.987 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099284+0000", 2024-09-14T06:16:46.987 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7979999999999995e-06 2024-09-14T06:16:46.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.987 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.987 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.987 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099289+0000", 2024-09-14T06:16:46.988 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:16:46.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.988 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.988 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:46.988 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164664+0000", 2024-09-14T06:16:46.988 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065374901999999999 2024-09-14T06:16:46.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.988 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.988 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:46.988 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164755+0000", 2024-09-14T06:16:46.988 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0790000000000003e-05 2024-09-14T06:16:46.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.988 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.989 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:46.989 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165143+0000", 2024-09-14T06:16:46.989 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000387904 2024-09-14T06:16:46.989 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.989 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.989 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:46.989 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167564+0000", 2024-09-14T06:16:46.989 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024208989999999998 2024-09-14T06:16:46.989 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.989 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.989 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:46.989 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181212+0000", 2024-09-14T06:16:46.989 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013648664 2024-09-14T06:16:46.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.990 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.990 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:46.990 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181237+0000", 2024-09-14T06:16:46.990 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5177999999999998e-05 2024-09-14T06:16:46.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.990 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.990 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:46.990 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181257+0000", 2024-09-14T06:16:46.990 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9660000000000002e-05 2024-09-14T06:16:46.990 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:46.990 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:46.990 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:46.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.991 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.991 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13236 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.991 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:16:46.991 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.845158252999999, 2024-09-14T06:16:46.991 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082010939000000005, 2024-09-14T06:16:46.991 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:46.991 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:46.991 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:46.991 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:46.991 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:46.991 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.991 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13236 2024-09-14T06:16:46.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.992 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:46.992 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.992 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:46.992 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:16:46.992 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.992 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.992 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.992 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:46.992 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:16:46.992 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.992 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.992 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.993 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:46.993 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:16:46.993 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.61e-07 2024-09-14T06:16:46.993 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.993 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.993 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:46.993 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099382+0000", 2024-09-14T06:16:46.993 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3589999999999999e-06 2024-09-14T06:16:46.993 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.994 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:46.994 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099388+0000", 2024-09-14T06:16:46.994 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9619999999999998e-06 2024-09-14T06:16:46.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.994 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:46.994 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099393+0000", 2024-09-14T06:16:46.994 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3410000000000002e-06 2024-09-14T06:16:46.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.994 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:46.994 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165553+0000", 2024-09-14T06:16:46.994 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066159604999999996 2024-09-14T06:16:46.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:46.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165603+0000", 2024-09-14T06:16:46.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0427999999999999e-05 2024-09-14T06:16:46.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:46.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165827+0000", 2024-09-14T06:16:46.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022358299999999999 2024-09-14T06:16:46.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:46.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177005+0000", 2024-09-14T06:16:46.996 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011178044 2024-09-14T06:16:46.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.996 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.996 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:46.996 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181302+0000", 2024-09-14T06:16:46.996 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042970669999999999 2024-09-14T06:16:46.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.996 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.996 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:46.996 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181349+0000", 2024-09-14T06:16:46.996 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7287999999999999e-05 2024-09-14T06:16:46.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.996 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.996 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:46.996 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181387+0000", 2024-09-14T06:16:46.997 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7601000000000001e-05 2024-09-14T06:16:46.997 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:46.997 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:46.997 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:46.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.997 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.997 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13237 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:46.997 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:16:46.997 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.845000106000001, 2024-09-14T06:16:46.997 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081927299999999995, 2024-09-14T06:16:46.998 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:46.998 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:46.998 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:46.998 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:46.998 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:46.998 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:46.998 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13237 2024-09-14T06:16:46.998 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.998 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:46.998 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.998 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:46.998 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:16:46.998 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.998 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.999 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.999 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:46.999 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:16:46.999 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:46.999 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.999 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:46.999 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:46.999 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099535+0000", 2024-09-14T06:16:46.999 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:16:46.999 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:46.999 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.000 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:47.000 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099538+0000", 2024-09-14T06:16:47.000 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:16:47.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.000 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:47.000 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099545+0000", 2024-09-14T06:16:47.000 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:16:47.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.000 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:47.000 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099551+0000", 2024-09-14T06:16:47.000 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1770000000000003e-06 2024-09-14T06:16:47.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:47.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166131+0000", 2024-09-14T06:16:47.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066580499000000001 2024-09-14T06:16:47.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:47.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166221+0000", 2024-09-14T06:16:47.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0000000000000006e-05 2024-09-14T06:16:47.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:47.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166622+0000", 2024-09-14T06:16:47.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040029799999999999 2024-09-14T06:16:47.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.002 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:47.002 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177008+0000", 2024-09-14T06:16:47.002 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010386772000000001 2024-09-14T06:16:47.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.002 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:47.002 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181420+0000", 2024-09-14T06:16:47.002 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044119190000000003 2024-09-14T06:16:47.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.002 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:47.002 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181443+0000", 2024-09-14T06:16:47.003 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2603999999999999e-05 2024-09-14T06:16:47.003 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.003 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.003 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:47.003 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181461+0000", 2024-09-14T06:16:47.003 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8405000000000001e-05 2024-09-14T06:16:47.003 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.003 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:47.003 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.003 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.003 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.003 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13238 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:47.003 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:16:47.003 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.844963798, 2024-09-14T06:16:47.004 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082014927000000001, 2024-09-14T06:16:47.004 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:47.004 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:47.004 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:47.004 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:47.004 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:47.004 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:47.004 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13238 2024-09-14T06:16:47.004 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.004 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:47.004 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.004 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:47.004 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:16:47.005 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.005 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.005 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.005 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:47.005 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:16:47.005 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.005 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.005 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.005 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:47.005 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099571+0000", 2024-09-14T06:16:47.005 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9699999999999996e-07 2024-09-14T06:16:47.006 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.006 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.006 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:47.006 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099577+0000", 2024-09-14T06:16:47.006 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2890000000000003e-06 2024-09-14T06:16:47.006 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.006 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.006 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:47.006 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099587+0000", 2024-09-14T06:16:47.006 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0002e-05 2024-09-14T06:16:47.006 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.006 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.006 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:47.006 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099593+0000", 2024-09-14T06:16:47.007 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4539999999999998e-06 2024-09-14T06:16:47.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.007 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.007 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:47.007 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167024+0000", 2024-09-14T06:16:47.007 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067431480000000002 2024-09-14T06:16:47.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.007 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.007 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:47.007 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167074+0000", 2024-09-14T06:16:47.007 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9996e-05 2024-09-14T06:16:47.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.007 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.007 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:47.007 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167288+0000", 2024-09-14T06:16:47.008 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021410900000000001 2024-09-14T06:16:47.008 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.008 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.008 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:47.008 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177997+0000", 2024-09-14T06:16:47.008 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010708756 2024-09-14T06:16:47.008 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.008 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.008 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:47.008 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181508+0000", 2024-09-14T06:16:47.008 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003511363 2024-09-14T06:16:47.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.009 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.009 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:47.009 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181552+0000", 2024-09-14T06:16:47.009 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3609999999999998e-05 2024-09-14T06:16:47.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.009 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.009 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:47.009 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181585+0000", 2024-09-14T06:16:47.009 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3271e-05 2024-09-14T06:16:47.009 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.009 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:47.009 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.010 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.010 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13239 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:47.010 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:16:47.010 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.844871980000001, 2024-09-14T06:16:47.010 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081998772999999997, 2024-09-14T06:16:47.010 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:47.010 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:47.010 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:47.010 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:47.010 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:47.010 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:47.010 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13239 2024-09-14T06:16:47.010 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.010 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:47.011 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.011 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:47.011 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:16:47.011 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.011 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.011 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.011 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:47.011 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:16:47.011 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.011 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.011 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.011 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:47.011 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099663+0000", 2024-09-14T06:16:47.011 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6100000000000001e-07 2024-09-14T06:16:47.011 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.012 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.012 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:47.012 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099666+0000", 2024-09-14T06:16:47.012 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.056e-06 2024-09-14T06:16:47.012 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.012 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.012 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:47.012 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099672+0000", 2024-09-14T06:16:47.012 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8120000000000002e-06 2024-09-14T06:16:47.012 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.012 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.012 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:47.012 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099676+0000", 2024-09-14T06:16:47.012 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:16:47.013 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.013 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.013 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:47.013 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167602+0000", 2024-09-14T06:16:47.013 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067925867000000001 2024-09-14T06:16:47.013 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.013 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.013 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:47.013 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167693+0000", 2024-09-14T06:16:47.013 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0736000000000007e-05 2024-09-14T06:16:47.013 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.013 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.013 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:47.013 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168065+0000", 2024-09-14T06:16:47.013 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037245399999999998 2024-09-14T06:16:47.014 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.014 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.014 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:47.014 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178003+0000", 2024-09-14T06:16:47.014 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0099372360000000003 2024-09-14T06:16:47.014 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.014 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.014 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:47.014 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181621+0000", 2024-09-14T06:16:47.014 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036178489999999998 2024-09-14T06:16:47.014 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.014 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.014 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:47.014 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181643+0000", 2024-09-14T06:16:47.014 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2155e-05 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181661+0000", 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8147000000000001e-05 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13240 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.844752887, 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081942607000000001, 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:47.015 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:47.016 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:47.016 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:47.016 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:47.016 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13240 2024-09-14T06:16:47.016 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.016 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:47.016 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.016 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:47.016 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:16:47.016 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.016 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.016 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.016 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:47.016 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:16:47.016 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.016 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.017 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.017 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:47.017 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099782+0000", 2024-09-14T06:16:47.017 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3900000000000004e-07 2024-09-14T06:16:47.017 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.017 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.017 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:47.020 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099787+0000", 2024-09-14T06:16:47.020 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6189999999999999e-06 2024-09-14T06:16:47.020 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.020 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.020 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:47.020 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099793+0000", 2024-09-14T06:16:47.020 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.998e-06 2024-09-14T06:16:47.021 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.021 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.021 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:47.021 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099797+0000", 2024-09-14T06:16:47.021 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8449999999999996e-06 2024-09-14T06:16:47.021 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.021 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.021 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:47.021 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168494+0000", 2024-09-14T06:16:47.021 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.068696500999999993 2024-09-14T06:16:47.021 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.021 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.021 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:47.022 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168590+0000", 2024-09-14T06:16:47.022 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6733999999999999e-05 2024-09-14T06:16:47.022 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.022 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.022 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:47.022 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168973+0000", 2024-09-14T06:16:47.022 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038229399999999998 2024-09-14T06:16:47.022 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.022 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.022 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:47.022 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178013+0000", 2024-09-14T06:16:47.022 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0090397199999999994 2024-09-14T06:16:47.022 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.022 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.023 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:47.023 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181683+0000", 2024-09-14T06:16:47.023 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036705819999999999 2024-09-14T06:16:47.023 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.023 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.023 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:47.023 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181705+0000", 2024-09-14T06:16:47.023 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1551000000000001e-05 2024-09-14T06:16:47.023 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.023 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.023 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:47.023 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181724+0000", 2024-09-14T06:16:47.024 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9123999999999999e-05 2024-09-14T06:16:47.024 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.024 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:47.024 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.024 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.024 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.024 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13243 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:47.024 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:16:47.024 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.844219156999999, 2024-09-14T06:16:47.024 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081767815999999993, 2024-09-14T06:16:47.024 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:47.024 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:47.024 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:47.024 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:47.025 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:47.025 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:47.025 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13243 2024-09-14T06:16:47.025 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.025 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:47.025 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.025 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:47.025 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:16:47.025 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.025 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.025 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.025 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:47.025 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:16:47.025 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.025 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.026 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.026 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:47.026 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:16:47.026 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1e-07 2024-09-14T06:16:47.026 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.026 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.026 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:47.026 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100317+0000", 2024-09-14T06:16:47.026 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.092e-06 2024-09-14T06:16:47.026 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.026 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.026 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:47.026 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100321+0000", 2024-09-14T06:16:47.026 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7809999999999999e-06 2024-09-14T06:16:47.026 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.026 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.027 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:47.027 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100324+0000", 2024-09-14T06:16:47.027 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.117e-06 2024-09-14T06:16:47.027 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.027 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.027 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:47.027 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171113+0000", 2024-09-14T06:16:47.027 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.070788816000000004 2024-09-14T06:16:47.027 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.027 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.027 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:47.028 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171202+0000", 2024-09-14T06:16:47.028 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8843000000000003e-05 2024-09-14T06:16:47.028 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.028 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.028 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:47.028 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171584+0000", 2024-09-14T06:16:47.028 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038231099999999998 2024-09-14T06:16:47.028 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.028 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.028 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:47.028 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178101+0000", 2024-09-14T06:16:47.028 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0065168329999999997 2024-09-14T06:16:47.028 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.028 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.028 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:47.029 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182041+0000", 2024-09-14T06:16:47.029 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039401810000000001 2024-09-14T06:16:47.029 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.029 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.029 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:47.029 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182065+0000", 2024-09-14T06:16:47.029 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3623000000000001e-05 2024-09-14T06:16:47.029 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.029 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.029 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:47.029 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182083+0000", 2024-09-14T06:16:47.029 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7909000000000001e-05 2024-09-14T06:16:47.029 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.029 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:47.030 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.030 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.030 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.030 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13244 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:47.030 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:16:47.030 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.844204337000001, 2024-09-14T06:16:47.030 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081837800000000002, 2024-09-14T06:16:47.030 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:47.030 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:47.030 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:47.030 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:47.030 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:47.030 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:47.030 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13244 2024-09-14T06:16:47.030 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.031 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:47.031 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.031 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:47.031 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:16:47.031 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.031 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.031 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.031 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:47.031 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:16:47.031 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.031 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.031 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.031 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:47.031 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:16:47.031 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:16:47.032 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.032 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.032 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:47.032 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100337+0000", 2024-09-14T06:16:47.032 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.939e-06 2024-09-14T06:16:47.032 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.032 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.032 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:47.032 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100341+0000", 2024-09-14T06:16:47.032 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2320000000000003e-06 2024-09-14T06:16:47.032 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.032 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.032 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:47.032 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100344+0000", 2024-09-14T06:16:47.032 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0759999999999999e-06 2024-09-14T06:16:47.033 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.033 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.033 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:47.033 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171996+0000", 2024-09-14T06:16:47.033 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.071651919999999994 2024-09-14T06:16:47.033 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.033 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.033 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:47.033 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172079+0000", 2024-09-14T06:16:47.033 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2579999999999999e-05 2024-09-14T06:16:47.033 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.033 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.033 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:47.033 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172457+0000", 2024-09-14T06:16:47.033 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037785100000000001 2024-09-14T06:16:47.034 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.034 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.034 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:47.034 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178104+0000", 2024-09-14T06:16:47.034 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056471730000000001 2024-09-14T06:16:47.034 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.034 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.034 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:47.034 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182123+0000", 2024-09-14T06:16:47.034 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040190019999999998 2024-09-14T06:16:47.034 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.034 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.034 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:47.034 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182147+0000", 2024-09-14T06:16:47.034 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4241000000000001e-05 2024-09-14T06:16:47.035 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.035 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.035 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:47.035 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182168+0000", 2024-09-14T06:16:47.035 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0492999999999998e-05 2024-09-14T06:16:47.035 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.035 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:47.035 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.035 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.035 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.035 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13245 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:47.035 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:16:47.035 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.84418483, 2024-09-14T06:16:47.035 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081893899000000006, 2024-09-14T06:16:47.036 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:47.036 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:47.036 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:47.036 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:47.036 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:47.036 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:47.036 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13245 2024-09-14T06:16:47.036 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.036 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:47.036 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.036 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:47.037 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:16:47.037 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.037 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.037 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.037 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:47.037 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:16:47.037 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.037 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.037 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.037 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:47.037 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100350+0000", 2024-09-14T06:16:47.037 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5499999999999999e-07 2024-09-14T06:16:47.037 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.037 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.037 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:47.038 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100353+0000", 2024-09-14T06:16:47.038 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:16:47.038 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.038 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.038 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:47.038 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100357+0000", 2024-09-14T06:16:47.038 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8500000000000004e-06 2024-09-14T06:16:47.038 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.038 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.038 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:47.038 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100360+0000", 2024-09-14T06:16:47.038 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8700000000000001e-06 2024-09-14T06:16:47.038 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.038 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.038 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:47.039 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172862+0000", 2024-09-14T06:16:47.039 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.072501992000000001 2024-09-14T06:16:47.039 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.039 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.039 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:47.039 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172950+0000", 2024-09-14T06:16:47.039 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8988999999999998e-05 2024-09-14T06:16:47.039 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.039 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.039 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:47.039 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173324+0000", 2024-09-14T06:16:47.039 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037366199999999997 2024-09-14T06:16:47.039 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.039 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.039 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:47.039 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178107+0000", 2024-09-14T06:16:47.040 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047830520000000003 2024-09-14T06:16:47.040 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.040 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.040 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:47.040 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182202+0000", 2024-09-14T06:16:47.040 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040949280000000003 2024-09-14T06:16:47.040 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.040 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.040 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:47.040 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182226+0000", 2024-09-14T06:16:47.040 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3535000000000001e-05 2024-09-14T06:16:47.040 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.040 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.040 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:47.040 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182243+0000", 2024-09-14T06:16:47.041 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7445e-05 2024-09-14T06:16:47.041 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.041 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:47.041 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.041 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.041 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.041 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13246 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:47.041 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:16:47.041 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.844170507999999, 2024-09-14T06:16:47.041 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081962276000000001, 2024-09-14T06:16:47.041 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:47.041 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:47.042 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:47.042 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:47.042 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:47.042 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:47.042 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13246 2024-09-14T06:16:47.042 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.042 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:47.042 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.042 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:47.042 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:16:47.042 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.042 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.042 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.042 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:47.042 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:16:47.043 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.043 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.043 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.043 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:47.043 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:16:47.043 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0800000000000001e-07 2024-09-14T06:16:47.043 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.043 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.043 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:47.043 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100367+0000", 2024-09-14T06:16:47.043 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8439999999999998e-06 2024-09-14T06:16:47.043 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.043 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.043 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:47.043 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100372+0000", 2024-09-14T06:16:47.044 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4009999999999996e-06 2024-09-14T06:16:47.044 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.044 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.044 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:47.044 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100378+0000", 2024-09-14T06:16:47.044 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7180000000000001e-06 2024-09-14T06:16:47.044 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.044 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.044 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:47.044 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173780+0000", 2024-09-14T06:16:47.044 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.073401991 2024-09-14T06:16:47.044 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.045 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.045 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:47.045 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173879+0000", 2024-09-14T06:16:47.045 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9449000000000003e-05 2024-09-14T06:16:47.045 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.045 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.045 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:47.045 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174240+0000", 2024-09-14T06:16:47.045 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036118100000000002 2024-09-14T06:16:47.045 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.045 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.045 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:47.045 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178109+0000", 2024-09-14T06:16:47.045 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038690619999999999 2024-09-14T06:16:47.046 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.046 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.046 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:47.046 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182281+0000", 2024-09-14T06:16:47.046 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041715900000000002 2024-09-14T06:16:47.046 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.046 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.046 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:47.046 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182306+0000", 2024-09-14T06:16:47.046 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4477e-05 2024-09-14T06:16:47.046 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.046 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.046 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:47.046 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182326+0000", 2024-09-14T06:16:47.046 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0254999999999998e-05 2024-09-14T06:16:47.047 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.047 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:47.047 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.047 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.047 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.047 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13247 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:47.047 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:16:47.047 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.844073331000001, 2024-09-14T06:16:47.047 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966593000000004, 2024-09-14T06:16:47.047 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:47.047 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:47.047 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:47.047 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:47.047 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:47.047 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:47.048 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13247 2024-09-14T06:16:47.048 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.048 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:47.048 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.048 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:47.048 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:16:47.048 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.048 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.048 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.048 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:47.048 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:16:47.048 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.048 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.048 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.048 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:47.049 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100462+0000", 2024-09-14T06:16:47.049 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9800000000000002e-07 2024-09-14T06:16:47.049 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.049 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.049 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:47.049 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100466+0000", 2024-09-14T06:16:47.049 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4039999999999997e-06 2024-09-14T06:16:47.049 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.049 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.049 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:47.049 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100475+0000", 2024-09-14T06:16:47.049 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3680000000000008e-06 2024-09-14T06:16:47.049 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.049 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.050 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:47.050 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100482+0000", 2024-09-14T06:16:47.050 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8630000000000002e-06 2024-09-14T06:16:47.050 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.050 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.050 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:47.050 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174669+0000", 2024-09-14T06:16:47.050 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074186849999999999 2024-09-14T06:16:47.050 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.050 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.050 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:47.050 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174758+0000", 2024-09-14T06:16:47.050 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9021e-05 2024-09-14T06:16:47.051 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.051 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.051 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:47.051 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175137+0000", 2024-09-14T06:16:47.051 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378511 2024-09-14T06:16:47.051 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.051 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.051 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:47.051 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178158+0000", 2024-09-14T06:16:47.051 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003020956 2024-09-14T06:16:47.051 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.051 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.051 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:47.051 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182386+0000", 2024-09-14T06:16:47.052 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042278840000000003 2024-09-14T06:16:47.052 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.052 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.052 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:47.052 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182410+0000", 2024-09-14T06:16:47.052 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4026e-05 2024-09-14T06:16:47.052 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.052 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.052 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:47.052 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182427+0000", 2024-09-14T06:16:47.052 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7711999999999999e-05 2024-09-14T06:16:47.052 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.052 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:47.052 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.053 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.053 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.053 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13248 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:47.053 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:16:47.053 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.84399125, 2024-09-14T06:16:47.053 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966527999999997, 2024-09-14T06:16:47.053 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:47.053 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:47.053 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:47.053 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:47.053 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:47.053 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:47.053 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13248 2024-09-14T06:16:47.053 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.053 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:47.054 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.054 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:47.054 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:16:47.054 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.054 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.054 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.054 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:47.054 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:16:47.054 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.054 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.054 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.054 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:47.054 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:16:47.055 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:16:47.055 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.055 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.055 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:47.055 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100548+0000", 2024-09-14T06:16:47.055 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:16:47.055 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.055 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.055 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:47.055 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100557+0000", 2024-09-14T06:16:47.055 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:16:47.055 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.055 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.055 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:47.055 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100563+0000", 2024-09-14T06:16:47.055 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2519999999999996e-06 2024-09-14T06:16:47.056 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.056 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.056 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:47.056 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175572+0000", 2024-09-14T06:16:47.056 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075008459 2024-09-14T06:16:47.056 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.056 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.056 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:47.056 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175656+0000", 2024-09-14T06:16:47.056 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3906999999999995e-05 2024-09-14T06:16:47.056 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.056 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.056 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:47.056 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176010+0000", 2024-09-14T06:16:47.057 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00035447499999999998 2024-09-14T06:16:47.057 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.057 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.057 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:47.057 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178304+0000", 2024-09-14T06:16:47.057 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0022941379999999998 2024-09-14T06:16:47.057 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.057 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.057 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:47.057 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182467+0000", 2024-09-14T06:16:47.057 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041632020000000004 2024-09-14T06:16:47.057 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.057 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.057 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:47.058 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182491+0000", 2024-09-14T06:16:47.058 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3733000000000001e-05 2024-09-14T06:16:47.058 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.058 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.058 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:47.058 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182509+0000", 2024-09-14T06:16:47.058 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8113000000000001e-05 2024-09-14T06:16:47.058 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.058 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:47.058 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.058 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.058 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.058 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13249 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:47.059 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:16:47.059 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.843956316, 2024-09-14T06:16:47.059 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082037225000000005, 2024-09-14T06:16:47.059 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:47.059 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:47.059 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:47.059 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:47.059 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:47.059 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:47.059 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13249 2024-09-14T06:16:47.059 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.059 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:47.059 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.059 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:47.060 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:16:47.060 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.060 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.060 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.060 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:47.060 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:16:47.060 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.060 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.060 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.060 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:47.060 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:16:47.060 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4600000000000005e-07 2024-09-14T06:16:47.060 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.060 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.060 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:47.060 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100584+0000", 2024-09-14T06:16:47.061 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.947e-06 2024-09-14T06:16:47.061 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.061 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.061 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:47.061 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100593+0000", 2024-09-14T06:16:47.061 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9760000000000001e-06 2024-09-14T06:16:47.061 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.061 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.061 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:47.061 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100597+0000", 2024-09-14T06:16:47.061 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9689999999999996e-06 2024-09-14T06:16:47.061 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.061 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.061 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:47.062 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176461+0000", 2024-09-14T06:16:47.062 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075864132000000001 2024-09-14T06:16:47.062 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.062 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.062 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:47.062 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176516+0000", 2024-09-14T06:16:47.062 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4416999999999998e-05 2024-09-14T06:16:47.062 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.062 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.062 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:47.062 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176734+0000", 2024-09-14T06:16:47.063 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021859399999999999 2024-09-14T06:16:47.063 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.063 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.063 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:47.063 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178308+0000", 2024-09-14T06:16:47.063 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015733660000000001 2024-09-14T06:16:47.063 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.063 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.063 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:47.063 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182573+0000", 2024-09-14T06:16:47.063 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042649200000000002 2024-09-14T06:16:47.063 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.063 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.063 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:47.063 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182597+0000", 2024-09-14T06:16:47.064 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4355e-05 2024-09-14T06:16:47.064 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.064 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.064 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:47.064 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182615+0000", 2024-09-14T06:16:47.064 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8003e-05 2024-09-14T06:16:47.064 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.064 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:47.064 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.064 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.064 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.064 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13250 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:47.065 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:16:47.065 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.843791711, 2024-09-14T06:16:47.065 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081983890000000004, 2024-09-14T06:16:47.065 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:47.065 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:47.065 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:47.065 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:47.065 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:47.065 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:47.065 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13250 2024-09-14T06:16:47.065 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.065 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:47.065 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.065 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:47.066 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:16:47.066 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.066 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.066 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.066 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:47.066 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:16:47.066 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:47.066 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.066 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.066 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:47.066 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100743+0000", 2024-09-14T06:16:47.066 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:16:47.066 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.067 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.067 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:47.067 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100753+0000", 2024-09-14T06:16:47.067 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9979999999999998e-06 2024-09-14T06:16:47.067 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.067 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.067 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:47.067 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100772+0000", 2024-09-14T06:16:47.067 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8664e-05 2024-09-14T06:16:47.067 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.067 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.067 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:47.067 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100779+0000", 2024-09-14T06:16:47.067 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.046e-06 2024-09-14T06:16:47.067 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.068 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.068 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:47.068 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177049+0000", 2024-09-14T06:16:47.068 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076270819000000004 2024-09-14T06:16:47.068 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.068 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.068 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:47.068 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177140+0000", 2024-09-14T06:16:47.068 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0746000000000002e-05 2024-09-14T06:16:47.068 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.068 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.068 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:47.068 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177519+0000", 2024-09-14T06:16:47.068 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378656 2024-09-14T06:16:47.069 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.069 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.069 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:47.069 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178349+0000", 2024-09-14T06:16:47.069 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00083061599999999995 2024-09-14T06:16:47.069 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.069 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.069 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:47.069 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182679+0000", 2024-09-14T06:16:47.069 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043298379999999999 2024-09-14T06:16:47.069 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.069 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.069 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:47.069 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182705+0000", 2024-09-14T06:16:47.069 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5602999999999999e-05 2024-09-14T06:16:47.070 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:47.070 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:47.070 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:47.070 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182726+0000", 2024-09-14T06:16:47.070 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1297e-05 2024-09-14T06:16:47.070 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.070 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:47.070 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.070 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:47.070 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:47.070 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:47.072 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:16:47.072 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:47.071+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:47.146 INFO:teuthology.orchestra.run.smithi039.stdout:1700 write 0xa64bb1d thru 0xa655e1a (0xa2fe bytes) 2024-09-14T06:16:47.152 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:16:47.160 INFO:teuthology.orchestra.run.smithi039.stdout:1702 writesame 0xaab7755 thru 0xaabaa08 data_size 0x16(0x32b4 bytes) 2024-09-14T06:16:47.173 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:47.171+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:47.182 INFO:teuthology.orchestra.run.smithi039.stdout:1703 trunc from 0xdda6288 to 0x3fc07ca 2024-09-14T06:16:47.237 INFO:teuthology.orchestra.run.smithi039.stdout:1704 write 0x7733924 thru 0x773d084 (0x9761 bytes) 2024-09-14T06:16:47.255 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:47.255 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:47.255 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:16:47.255 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:47.257 INFO:teuthology.orchestra.run.smithi039.stdout:1705 write 0xd81c587 thru 0xd81f653 (0x30cd bytes) 2024-09-14T06:16:47.261 INFO:teuthology.orchestra.run.smithi039.stdout:1706 write 0x922d6b5 thru 0x9234fee (0x793a bytes) 2024-09-14T06:16:47.266 INFO:teuthology.orchestra.run.smithi039.stdout:1707 write 0xa868c04 thru 0xa874dd2 (0xc1cf bytes) 2024-09-14T06:16:47.271 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:16:47.272 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:47.272 INFO:teuthology.orchestra.run.smithi039.stdout:1709 punch from 0x6035890 to 0x6036594, (0xd04 bytes) 2024-09-14T06:16:47.272 INFO:teuthology.orchestra.run.smithi039.stdout:1710 write 0x5a971e4 thru 0x5a9c9df (0x57fc bytes) 2024-09-14T06:16:47.277 INFO:teuthology.orchestra.run.smithi039.stdout:1712 read 0x341fc27 thru 0x3427325 (0x76ff bytes) 2024-09-14T06:16:47.277 INFO:teuthology.orchestra.run.smithi039.stdout:1713 write 0x213a328 thru 0x213ec40 (0x4919 bytes) 2024-09-14T06:16:47.285 INFO:teuthology.orchestra.run.smithi039.stdout:1714 write 0x7cfc3fe thru 0x7cfc4da (0xdd bytes) 2024-09-14T06:16:47.288 INFO:teuthology.orchestra.run.smithi039.stdout:1716 read 0xaaa8c4a thru 0xaab73f7 (0xe7ae bytes) 2024-09-14T06:16:47.289 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:47.293 INFO:teuthology.orchestra.run.smithi039.stdout:1718 trunc from 0xd81f654 to 0xb267799 2024-09-14T06:16:47.307 INFO:teuthology.orchestra.run.smithi039.stdout:1721 write 0xaa8032d thru 0xaa8a0ad (0x9d81 bytes) 2024-09-14T06:16:47.312 INFO:teuthology.orchestra.run.smithi039.stdout:1723 writesame 0x85a5fba thru 0x85b2f2e data_size 0x3f(0xcf75 bytes) 2024-09-14T06:16:47.314 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:47.315+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:47.323 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:47.323 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:47.324 INFO:teuthology.orchestra.run.smithi039.stdout:1724 write 0xb8298a1 thru 0xb83610b (0xc86b bytes) 2024-09-14T06:16:47.327 INFO:teuthology.orchestra.run.smithi039.stdout:1725 read 0x2f33771 thru 0x2f3619b (0x2a2b bytes) 2024-09-14T06:16:47.327 INFO:teuthology.orchestra.run.smithi039.stdout:1726 write 0x7e35275 thru 0x7e37793 (0x251f bytes) 2024-09-14T06:16:47.330 INFO:teuthology.orchestra.run.smithi039.stdout:1727 punch from 0x6cbadf6 to 0x6cca01b, (0xf225 bytes) 2024-09-14T06:16:47.339 INFO:teuthology.orchestra.run.smithi039.stdout:1728 write 0xc003c69 thru 0xc00f509 (0xb8a1 bytes) 2024-09-14T06:16:47.341 INFO:teuthology.orchestra.run.smithi039.stdout:1729 writesame 0x8ffa8ef thru 0x90035ce data_size 0x70(0x8ce0 bytes) 2024-09-14T06:16:47.341 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:47.341 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:47.345 INFO:teuthology.orchestra.run.smithi039.stdout:1731 writesame 0x208e942 thru 0x20959ea data_size 0x1(0x70a9 bytes) 2024-09-14T06:16:47.360 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:47.360 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:47.374 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:47.375+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:47.376 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:47.377 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:47.382 INFO:teuthology.orchestra.run.smithi039.stdout:1732 writesame 0x8790cbc thru 0x879237e data_size 0x1(0x16c3 bytes) 2024-09-14T06:16:47.390 INFO:teuthology.orchestra.run.smithi039.stdout:1733 write 0x3ed770b thru 0x3edb310 (0x3c06 bytes) 2024-09-14T06:16:47.392 INFO:teuthology.orchestra.run.smithi039.stdout:1734 read 0x6b791df thru 0x6b7f8c6 (0x66e8 bytes) 2024-09-14T06:16:47.393 INFO:teuthology.orchestra.run.smithi039.stdout:1736 write 0x82e1d20 thru 0x82eaf86 (0x9267 bytes) 2024-09-14T06:16:47.395 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:47.395 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:47.397 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:47.399 INFO:teuthology.orchestra.run.smithi039.stdout:1739 trunc from 0xc00f50a to 0xd57f7d7 2024-09-14T06:16:47.406 INFO:teuthology.orchestra.run.smithi039.stdout:1740 writesame 0x7bc494e thru 0x7bc5dd1 data_size 0x1a(0x1484 bytes) 2024-09-14T06:16:47.409 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:47.409 INFO:teuthology.orchestra.run.smithi039.stdout:1742 write 0x4181b66 thru 0x41846a8 (0x2b43 bytes) 2024-09-14T06:16:47.411 INFO:teuthology.orchestra.run.smithi039.stdout:1743 write 0x2ba97f9 thru 0x2bb8d4a (0xf552 bytes) 2024-09-14T06:16:47.413 INFO:teuthology.orchestra.run.smithi039.stdout:1744 punch from 0x5d2ce6c to 0x5d2debb, (0x104f bytes) 2024-09-14T06:16:47.415 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:47.415 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:47.420 INFO:teuthology.orchestra.run.smithi039.stdout:1745 write 0xdb043ab thru 0xdb05caa (0x1900 bytes) 2024-09-14T06:16:47.424 INFO:teuthology.orchestra.run.smithi039.stdout:1746 write 0xcb78165 thru 0xcb7feeb (0x7d87 bytes) 2024-09-14T06:16:47.431 INFO:teuthology.orchestra.run.smithi039.stdout:1748 trunc from 0xdb05cab to 0xec68164 2024-09-14T06:16:47.437 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:47.437 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:47.438 INFO:teuthology.orchestra.run.smithi039.stdout:1749 write 0x5e14eb8 thru 0x5e16a33 (0x1b7c bytes) 2024-09-14T06:16:47.442 INFO:teuthology.orchestra.run.smithi039.stdout:1750 punch from 0x73dc701 to 0x73ea79d, (0xe09c bytes) 2024-09-14T06:16:47.442 INFO:teuthology.orchestra.run.smithi039.stdout:1751 write 0xb749736 thru 0xb7577fe (0xe0c9 bytes) 2024-09-14T06:16:47.446 INFO:teuthology.orchestra.run.smithi039.stdout:1752 punch from 0xa1729fe to 0xa172f13, (0x515 bytes) 2024-09-14T06:16:47.446 INFO:teuthology.orchestra.run.smithi039.stdout:1754 punch from 0x4276770 to 0x42828bd, (0xc14d bytes) 2024-09-14T06:16:47.446 INFO:teuthology.orchestra.run.smithi039.stdout:1755 read 0xde7f2fb thru 0xde8d933 (0xe639 bytes) 2024-09-14T06:16:47.446 INFO:teuthology.orchestra.run.smithi039.stdout:1757 write 0x5e503c8 thru 0x5e5af44 (0xab7d bytes) 2024-09-14T06:16:47.450 INFO:teuthology.orchestra.run.smithi039.stdout:1758 read 0xe3c71c1 thru 0xe3ce27a (0x70ba bytes) 2024-09-14T06:16:47.450 INFO:teuthology.orchestra.run.smithi039.stdout:1759 writesame 0x20634e9 thru 0x2072585 data_size 0x1(0xf09d bytes) 2024-09-14T06:16:47.452 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:47.452 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:47.458 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:47.468 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:16:47.489 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-09-14T06:16:47.501 INFO:teuthology.orchestra.run.smithi039.stdout:1760 write 0x1c1172a thru 0x1c1539a (0x3c71 bytes) 2024-09-14T06:16:47.506 INFO:teuthology.orchestra.run.smithi039.stdout:1761 writesame 0x2100675 thru 0x21061b0 data_size 0x2(0x5b3c bytes) 2024-09-14T06:16:47.510 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:47.511+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:47.530 INFO:teuthology.orchestra.run.smithi039.stdout:1763 write 0xb6db771 thru 0xb6e7ff2 (0xc882 bytes) 2024-09-14T06:16:47.535 INFO:teuthology.orchestra.run.smithi039.stdout:1764 read 0x217cc4e thru 0x2186a0f (0x9dc2 bytes) 2024-09-14T06:16:47.541 INFO:teuthology.orchestra.run.smithi039.stdout:1765 trunc from 0xec68164 to 0xbc2f426 2024-09-14T06:16:47.564 INFO:teuthology.orchestra.run.smithi039.stdout:1766 write 0x7a31f41 thru 0x7a388c2 (0x6982 bytes) 2024-09-14T06:16:47.567 INFO:teuthology.orchestra.run.smithi039.stdout:1768 write 0x8b99f94 thru 0x8b9e1c6 (0x4233 bytes) 2024-09-14T06:16:47.569 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:47.569 INFO:teuthology.orchestra.run.smithi039.stdout:1771 read 0x2d92168 thru 0x2d98d66 (0x6bff bytes) 2024-09-14T06:16:47.569 INFO:teuthology.orchestra.run.smithi039.stdout:1772 write 0x205a919 thru 0x205acfd (0x3e5 bytes) 2024-09-14T06:16:47.571 INFO:teuthology.orchestra.run.smithi039.stdout:1776 write 0x19b2fa9 thru 0x19b4c19 (0x1c71 bytes) 2024-09-14T06:16:47.573 INFO:teuthology.orchestra.run.smithi039.stdout:1777 read 0x46ad37b thru 0x46ae405 (0x108b bytes) 2024-09-14T06:16:47.574 INFO:teuthology.orchestra.run.smithi039.stdout:1778 punch from 0xb1d836 to 0xb29a2c, (0xc1f6 bytes) 2024-09-14T06:16:47.574 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:47.574 INFO:teuthology.orchestra.run.smithi039.stdout:1780 punch from 0xb27e36c to 0xb287d0e, (0x99a2 bytes) 2024-09-14T06:16:47.574 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:47.574 INFO:teuthology.orchestra.run.smithi039.stdout:1782 read 0x409eaf9 thru 0x40a5a7c (0x6f84 bytes) 2024-09-14T06:16:47.575 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:47.575+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:47.583 INFO:teuthology.orchestra.run.smithi039.stdout:1783 write 0xcba2904 thru 0xcba2c34 (0x331 bytes) 2024-09-14T06:16:47.587 INFO:teuthology.orchestra.run.smithi039.stdout:1784 read 0x5e2efb1 thru 0x5e3aace (0xbb1e bytes) 2024-09-14T06:16:47.612 INFO:teuthology.orchestra.run.smithi039.stdout:1785 trunc from 0xcba2c35 to 0x1e72dbb 2024-09-14T06:16:47.651 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:47.651 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:47.671 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:16:47.677 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:47.675+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:47.679 INFO:teuthology.orchestra.run.smithi039.stdout:1786 punch from 0x128f929 to 0x129e8ba, (0xef91 bytes) 2024-09-14T06:16:47.679 INFO:teuthology.orchestra.run.smithi039.stdout:1787 punch from 0x169c30d to 0x16a0e84, (0x4b77 bytes) 2024-09-14T06:16:47.679 INFO:teuthology.orchestra.run.smithi039.stdout:1788 read 0xbb124a thru 0xbb680c (0x55c3 bytes) 2024-09-14T06:16:47.694 INFO:teuthology.orchestra.run.smithi039.stdout:1789 write 0x44b3dfd thru 0x44bc6a1 (0x88a5 bytes) 2024-09-14T06:16:47.723 INFO:teuthology.orchestra.run.smithi039.stdout:1791 write 0xa5613d6 thru 0xa56be7f (0xaaaa bytes) 2024-09-14T06:16:47.729 INFO:teuthology.orchestra.run.smithi039.stdout:1792 write 0x3c13c31 thru 0x3c1f4a9 (0xb879 bytes) 2024-09-14T06:16:47.735 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:47.735 INFO:teuthology.orchestra.run.smithi039.stdout:1796 write 0x7aa0000 thru 0x7aa4b52 (0x4b53 bytes) 2024-09-14T06:16:47.754 INFO:teuthology.orchestra.run.smithi039.stdout:1797 trunc from 0xa56be80 to 0x5917b7f 2024-09-14T06:16:47.777 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:47.775+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:47.804 INFO:teuthology.orchestra.run.smithi039.stdout:1798 write 0xc8833d8 thru 0xc8903e5 (0xd00e bytes) 2024-09-14T06:16:47.811 INFO:teuthology.orchestra.run.smithi039.stdout:1799 write 0xbc0ca4b thru 0xbc1a623 (0xdbd9 bytes) 2024-09-14T06:16:47.818 INFO:teuthology.orchestra.run.smithi039.stdout:1800 writesame 0x1fe2720 thru 0x1fe6b8e data_size 0x1(0x446f bytes) 2024-09-14T06:16:47.837 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:47.837 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:47.843 INFO:teuthology.orchestra.run.smithi039.stdout:1801 punch from 0xf05a73 to 0xf0ed53, (0x92e0 bytes) 2024-09-14T06:16:47.843 INFO:teuthology.orchestra.run.smithi039.stdout:1802 write 0x5af73d3 thru 0x5afd9bc (0x65ea bytes) 2024-09-14T06:16:47.851 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:16:47.851 INFO:teuthology.orchestra.run.smithi039.stdout:1803 writesame 0x74b324 thru 0x74c0fa data_size 0x1(0xdd7 bytes) 2024-09-14T06:16:47.856 INFO:teuthology.orchestra.run.smithi039.stdout:1804 read 0x3d18025 thru 0x3d20922 (0x88fe bytes) 2024-09-14T06:16:47.875 INFO:teuthology.orchestra.run.smithi039.stdout:1805 write 0xee52b34 thru 0xee57de2 (0x52af bytes) 2024-09-14T06:16:47.878 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:47.875+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:47.919 INFO:teuthology.orchestra.run.smithi039.stdout:1806 trunc from 0xee57de3 to 0x1073086 2024-09-14T06:16:47.948 INFO:teuthology.orchestra.run.smithi039.stdout:1811 read 0x279ade thru 0x27f068 (0x558b bytes) 2024-09-14T06:16:47.948 INFO:teuthology.orchestra.run.smithi039.stdout:1813 read 0xd51f5f thru 0xd557e0 (0x3882 bytes) 2024-09-14T06:16:47.965 INFO:teuthology.orchestra.run.smithi039.stdout:1816 write 0x2139dcd thru 0x2149a84 (0xfcb8 bytes) 2024-09-14T06:16:47.969 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:47.978 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:47.979+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:47.990 INFO:teuthology.orchestra.run.smithi039.stdout:1818 trunc from 0x2149a85 to 0x9622cbd 2024-09-14T06:16:48.003 INFO:teuthology.orchestra.run.smithi039.stdout:1819 write 0x3afec8 thru 0x3bd32d (0xd466 bytes) 2024-09-14T06:16:48.019 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:48.020 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:48.026 INFO:teuthology.orchestra.run.smithi039.stdout:1820 write 0xc83e70f thru 0xc83ecf6 (0x5e8 bytes) 2024-09-14T06:16:48.032 INFO:teuthology.orchestra.run.smithi039.stdout:1822 writesame 0x35b82c thru 0x361572 data_size 0x1(0x5d47 bytes) 2024-09-14T06:16:48.035 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:16:48.043 INFO:teuthology.orchestra.run.smithi039.stdout:1823 write 0x354c2b thru 0x356b72 (0x1f48 bytes) 2024-09-14T06:16:48.046 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:48.046 INFO:teuthology.orchestra.run.smithi039.stdout:1826 write 0xa66e3c0 thru 0xa674c8b (0x68cc bytes) 2024-09-14T06:16:48.052 INFO:teuthology.orchestra.run.smithi039.stdout:1827 punch from 0x853a4f6 to 0x8548076, (0xdb80 bytes) 2024-09-14T06:16:48.064 INFO:teuthology.orchestra.run.smithi039.stdout:1828 writesame 0xd626b8e thru 0xd63063c data_size 0x1(0x9aaf bytes) 2024-09-14T06:16:48.065 INFO:teuthology.orchestra.run.smithi039.stdout:1829 trunc from 0xd63063d to 0xd491796 2024-09-14T06:16:48.078 INFO:teuthology.orchestra.run.smithi039.stdout:1830 write 0x56f87cb thru 0x56ff6dd (0x6f13 bytes) 2024-09-14T06:16:48.079 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:48.079+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:48.084 INFO:teuthology.orchestra.run.smithi039.stdout:1831 read 0x7acaa39 thru 0x7ada128 (0xf6f0 bytes) 2024-09-14T06:16:48.084 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:48.085 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:48.085 INFO:teuthology.orchestra.run.smithi039.stdout:1837 punch from 0x833c94b to 0x8343d7c, (0x7431 bytes) 2024-09-14T06:16:48.087 INFO:teuthology.orchestra.run.smithi039.stdout:1840 trunc from 0xd491796 to 0xe3e956e 2024-09-14T06:16:48.097 INFO:teuthology.orchestra.run.smithi039.stdout:1842 punch from 0x8699e1e to 0x86a235c, (0x853e bytes) 2024-09-14T06:16:48.097 INFO:teuthology.orchestra.run.smithi039.stdout:1844 writesame 0x7cc629e thru 0x7cc8d09 data_size 0x1e(0x2a6c bytes) 2024-09-14T06:16:48.103 INFO:teuthology.orchestra.run.smithi039.stdout:1845 write 0x86048e0 thru 0x860c095 (0x77b6 bytes) 2024-09-14T06:16:48.109 INFO:teuthology.orchestra.run.smithi039.stdout:1848 writesame 0x732a285 thru 0x732c9cb data_size 0x1(0x2747 bytes) 2024-09-14T06:16:48.120 INFO:teuthology.orchestra.run.smithi039.stdout:1849 write 0xc71458 thru 0xc7f961 (0xe50a bytes) 2024-09-14T06:16:48.125 INFO:teuthology.orchestra.run.smithi039.stdout:1850 write 0xac0ed35 thru 0xac1283c (0x3b08 bytes) 2024-09-14T06:16:48.128 INFO:teuthology.orchestra.run.smithi039.stdout:1851 punch from 0x2dd8609 to 0x2ddf45a, (0x6e51 bytes) 2024-09-14T06:16:48.129 INFO:teuthology.orchestra.run.smithi039.stdout:1852 read 0x55090af thru 0x5516c93 (0xdbe5 bytes) 2024-09-14T06:16:48.129 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:48.163 INFO:teuthology.orchestra.run.smithi039.stdout:1855 trunc from 0xe3e956e to 0x1b39b1e 2024-09-14T06:16:48.179 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:48.179+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:48.191 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:48.192 INFO:teuthology.orchestra.run.smithi039.stdout:1858 punch from 0xca78f9 to 0xcb36a5, (0xbdac bytes) 2024-09-14T06:16:48.201 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:48.201 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:48.215 INFO:teuthology.orchestra.run.smithi039.stdout:1860 write 0x58728c0 thru 0x587444f (0x1b90 bytes) 2024-09-14T06:16:48.216 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:16:48.243 INFO:teuthology.orchestra.run.smithi039.stdout:1861 trunc from 0x5874450 to 0xde60adf 2024-09-14T06:16:48.256 INFO:teuthology.orchestra.run.smithi039.stdout:1862 write 0xaabc50b thru 0xaac87b1 (0xc2a7 bytes) 2024-09-14T06:16:48.293 INFO:teuthology.orchestra.run.smithi039.stdout:1863 trunc from 0xde60adf to 0x37ee856 2024-09-14T06:16:48.302 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:48.303+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:48.348 INFO:teuthology.orchestra.run.smithi039.stdout:1865 write 0xe10cda3 thru 0xe110750 (0x39ae bytes) 2024-09-14T06:16:48.354 INFO:teuthology.orchestra.run.smithi039.stdout:1866 punch from 0x7ba91ad to 0x7bb3cf9, (0xab4c bytes) 2024-09-14T06:16:48.355 INFO:teuthology.orchestra.run.smithi039.stdout:1867 write 0x4afb373 thru 0x4afcb84 (0x1812 bytes) 2024-09-14T06:16:48.379 INFO:teuthology.orchestra.run.smithi039.stdout:1868 trunc from 0xe110751 to 0x7558068 2024-09-14T06:16:48.380 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:48.379+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:48.388 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:48.388 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:48.403 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:16:48.404 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:48.414 INFO:teuthology.orchestra.run.smithi039.stdout:1870 write 0x7da2fd7 thru 0x7da3e15 (0xe3f bytes) 2024-09-14T06:16:48.420 INFO:teuthology.orchestra.run.smithi039.stdout:1871 writesame 0x354bb97 thru 0x35529ba data_size 0x6a(0x6e24 bytes) 2024-09-14T06:16:48.425 INFO:teuthology.orchestra.run.smithi039.stdout:1873 write 0x3b68465 thru 0x3b6de9b (0x5a37 bytes) 2024-09-14T06:16:48.432 INFO:teuthology.orchestra.run.smithi039.stdout:1875 read 0x3237b4d thru 0x323934d (0x1801 bytes) 2024-09-14T06:16:48.432 INFO:teuthology.orchestra.run.smithi039.stdout:1877 writesame 0x34ae98d thru 0x34b3478 data_size 0x46(0x4aec bytes) 2024-09-14T06:16:48.447 INFO:teuthology.orchestra.run.smithi039.stdout:1878 writesame 0x8090508 thru 0x80962ba data_size 0x11(0x5db3 bytes) 2024-09-14T06:16:48.447 INFO:teuthology.orchestra.run.smithi039.stdout:1879 read 0x2796d11 thru 0x279d6b3 (0x69a3 bytes) 2024-09-14T06:16:48.448 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:48.448 INFO:teuthology.orchestra.run.smithi039.stdout:1881 read 0x771958b thru 0x7723c87 (0xa6fd bytes) 2024-09-14T06:16:48.448 INFO:teuthology.orchestra.run.smithi039.stdout:1882 trunc from 0x80962bb to 0x8070a6c 2024-09-14T06:16:48.459 INFO:teuthology.orchestra.run.smithi039.stdout:1883 read 0x5021004 thru 0x5027091 (0x608e bytes) 2024-09-14T06:16:48.459 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:48.477 INFO:teuthology.orchestra.run.smithi039.stdout:1890 write 0xa7e8a72 thru 0xa7f86e1 (0xfc70 bytes) 2024-09-14T06:16:48.483 INFO:teuthology.orchestra.run.smithi039.stdout:1892 read 0x1058e92 thru 0x1061207 (0x8376 bytes) 2024-09-14T06:16:48.483 INFO:teuthology.orchestra.run.smithi039.stdout:1893 read 0x6f20476 thru 0x6f26f47 (0x6ad2 bytes) 2024-09-14T06:16:48.483 INFO:teuthology.orchestra.run.smithi039.stdout:1894 clone 12 order 20 su 131072 sc 15 2024-09-14T06:16:48.498 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:48.499+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:48.582 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:48.582 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:48.582 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:48.579+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:48.595 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:16:48.677 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone11 from 0xa7f86e2 (overlap 0x1073086) to 0xecdedf 2024-09-14T06:16:48.684 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:48.683+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:48.769 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:48.770 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:48.784 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:48.783+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:48.784 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:16:48.861 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-14T06:16:48.885 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:48.883+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:48.935 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:48.936 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:48.962 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:16:48.986 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:48.983+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:49.086 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:49.087+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:49.093 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:49.093 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:49.096 INFO:teuthology.orchestra.run.smithi039.stdout:1895 punch from 0x6603dd6 to 0x6612839, (0xea63 bytes) 2024-09-14T06:16:49.109 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:16:49.110 INFO:teuthology.orchestra.run.smithi039.stdout:1896 trunc from 0xa7f86e2 to 0xdfe597c 2024-09-14T06:16:49.123 INFO:teuthology.orchestra.run.smithi039.stdout:1897 write 0x81593e3 thru 0x81693b6 (0xffd4 bytes) 2024-09-14T06:16:49.131 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:49.131 INFO:teuthology.orchestra.run.smithi039.stdout:1900 read 0xa870d57 thru 0xa880113 (0xf3bd bytes) 2024-09-14T06:16:49.131 INFO:teuthology.orchestra.run.smithi039.stdout:1901 read 0x340faab thru 0x341d348 (0xd89e bytes) 2024-09-14T06:16:49.140 INFO:teuthology.orchestra.run.smithi039.stdout:1902 trunc from 0xdfe597c to 0x7f3e64e 2024-09-14T06:16:49.182 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:49.183 INFO:teuthology.orchestra.run.smithi039.stdout:1905 read 0x40cd955 thru 0x40d4c29 (0x72d5 bytes) 2024-09-14T06:16:49.183 INFO:teuthology.orchestra.run.smithi039.stdout:1906 write 0x26b50f5 thru 0x26b590d (0x819 bytes) 2024-09-14T06:16:49.187 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:49.187+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:49.189 INFO:teuthology.orchestra.run.smithi039.stdout:1907 punch from 0x371f52f to 0x372e3bf, (0xee90 bytes) 2024-09-14T06:16:49.189 INFO:teuthology.orchestra.run.smithi039.stdout:1908 writesame 0x510c658 thru 0x5119546 data_size 0x41(0xceef bytes) 2024-09-14T06:16:49.197 INFO:teuthology.orchestra.run.smithi039.stdout:1909 trunc from 0x7f3e64e to 0x8cf962e 2024-09-14T06:16:49.208 INFO:teuthology.orchestra.run.smithi039.stdout:1911 write 0x7da458f thru 0x7db3f90 (0xfa02 bytes) 2024-09-14T06:16:49.210 INFO:teuthology.orchestra.run.smithi039.stdout:1912 write 0x15e588e thru 0x15ecebd (0x7630 bytes) 2024-09-14T06:16:49.217 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:49.217 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:49.217 INFO:teuthology.orchestra.run.smithi039.stdout:1917 read 0x39d4554 thru 0x39e2085 (0xdb32 bytes) 2024-09-14T06:16:49.236 INFO:teuthology.orchestra.run.smithi039.stdout:1918 write 0xb055deb thru 0xb05a7b4 (0x49ca bytes) 2024-09-14T06:16:49.240 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:49.240 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:49.242 INFO:teuthology.orchestra.run.smithi039.stdout:1922 read 0xa7d271b thru 0xa7de80a (0xc0f0 bytes) 2024-09-14T06:16:49.242 INFO:teuthology.orchestra.run.smithi039.stdout:1924 punch from 0x982efe6 to 0x983e252, (0xf26c bytes) 2024-09-14T06:16:49.242 INFO:teuthology.orchestra.run.smithi039.stdout:1926 read 0xacc75fd thru 0xacc86f9 (0x10fd bytes) 2024-09-14T06:16:49.242 INFO:teuthology.orchestra.run.smithi039.stdout:1927 read 0x10e4174 thru 0x10eef81 (0xae0e bytes) 2024-09-14T06:16:49.243 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:49.243 INFO:teuthology.orchestra.run.smithi039.stdout:1931 read 0xa2f6cee thru 0xa2f98de (0x2bf1 bytes) 2024-09-14T06:16:49.244 INFO:teuthology.orchestra.run.smithi039.stdout:1933 punch from 0x8807e3c to 0x880ca38, (0x4bfc bytes) 2024-09-14T06:16:49.244 INFO:teuthology.orchestra.run.smithi039.stdout:1934 read 0xf750f8 thru 0xf8156a (0xc473 bytes) 2024-09-14T06:16:49.260 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:16:49.268 INFO:teuthology.orchestra.run.smithi039.stdout:1935 write 0xda4a9e8 thru 0xda5a4e5 (0xfafe bytes) 2024-09-14T06:16:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:1936 punch from 0x5cec10a to 0x5cfb8f1, (0xf7e7 bytes) 2024-09-14T06:16:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:1937 punch from 0x2ff0253 to 0x2ffa702, (0xa4af bytes) 2024-09-14T06:16:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:1941 read 0xaab7294 thru 0xaab8e65 (0x1bd2 bytes) 2024-09-14T06:16:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:1942 read 0x62dcaff thru 0x62e6145 (0x9647 bytes) 2024-09-14T06:16:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:1943 write 0x42dc134 thru 0x42eaee2 (0xedaf bytes) 2024-09-14T06:16:49.286 INFO:teuthology.orchestra.run.smithi039.stdout:1944 read 0xbd6eb82 thru 0xbd70bbf (0x203e bytes) 2024-09-14T06:16:49.286 INFO:teuthology.orchestra.run.smithi039.stdout:1945 read 0x329fa24 thru 0x32adf1e (0xe4fb bytes) 2024-09-14T06:16:49.287 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:49.287+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:49.295 INFO:teuthology.orchestra.run.smithi039.stdout:1946 trunc from 0xda5a4e6 to 0x86cc675 2024-09-14T06:16:49.316 INFO:teuthology.orchestra.run.smithi039.stdout:1948 read 0x128e72c thru 0x12980ea (0x99bf bytes) 2024-09-14T06:16:49.316 INFO:teuthology.orchestra.run.smithi039.stdout:1949 punch from 0x486b2fa to 0x487b0b2, (0xfdb8 bytes) 2024-09-14T06:16:49.317 INFO:teuthology.orchestra.run.smithi039.stdout:1950 read 0x1bfc1a7 thru 0x1bffc48 (0x3aa2 bytes) 2024-09-14T06:16:49.318 INFO:teuthology.orchestra.run.smithi039.stdout:1951 read 0x3cb4393 thru 0x3cc2f9f (0xec0d bytes) 2024-09-14T06:16:49.340 INFO:teuthology.orchestra.run.smithi039.stdout:1952 writesame 0xa2f78ea thru 0xa2fd912 data_size 0x1(0x6029 bytes) 2024-09-14T06:16:49.340 INFO:teuthology.orchestra.run.smithi039.stdout:1953 writesame 0x32ac066 thru 0x32b45ff data_size 0x7(0x859a bytes) 2024-09-14T06:16:49.348 INFO:teuthology.orchestra.run.smithi039.stdout:1954 read 0x52473ab thru 0x5248b5f (0x17b5 bytes) 2024-09-14T06:16:49.348 INFO:teuthology.orchestra.run.smithi039.stdout:1956 read 0x9890806 thru 0x989f0c6 (0xe8c1 bytes) 2024-09-14T06:16:49.348 INFO:teuthology.orchestra.run.smithi039.stdout:1957 write 0xa00d7e thru 0xa0d166 (0xc3e9 bytes) 2024-09-14T06:16:49.361 INFO:teuthology.orchestra.run.smithi039.stdout:1958 punch from 0x637ec61 to 0x638c429, (0xd7c8 bytes) 2024-09-14T06:16:49.361 INFO:teuthology.orchestra.run.smithi039.stdout:1959 read 0xcf3642 thru 0xd008de (0xd29d bytes) 2024-09-14T06:16:49.362 INFO:teuthology.orchestra.run.smithi039.stdout:1960 write 0x729e3c8 thru 0x72a8352 (0x9f8b bytes) 2024-09-14T06:16:49.373 INFO:teuthology.orchestra.run.smithi039.stdout:1961 read 0x8fc5f63 thru 0x8fcf2bc (0x935a bytes) 2024-09-14T06:16:49.373 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:49.375 INFO:teuthology.orchestra.run.smithi039.stdout:1966 trunc from 0xa2fd913 to 0xc099eba 2024-09-14T06:16:49.388 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:49.387+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:49.390 INFO:teuthology.orchestra.run.smithi039.stdout:1967 write 0x3ca55c8 thru 0x3cb43e6 (0xee1f bytes) 2024-09-14T06:16:49.392 INFO:teuthology.orchestra.run.smithi039.stdout:1968 read 0x19c6a89 thru 0x19d3bc6 (0xd13e bytes) 2024-09-14T06:16:49.393 INFO:teuthology.orchestra.run.smithi039.stdout:1969 writesame 0x27d8c9b thru 0x27dbfc5 data_size 0x1(0x332b bytes) 2024-09-14T06:16:49.401 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:49.401 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:49.416 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:16:49.417 INFO:teuthology.orchestra.run.smithi039.stdout:1970 read 0x2a22b20 thru 0x2a29de5 (0x72c6 bytes) 2024-09-14T06:16:49.417 INFO:teuthology.orchestra.run.smithi039.stdout:1971 punch from 0x3fe3320 to 0x3ff1dcc, (0xeaac bytes) 2024-09-14T06:16:49.417 INFO:teuthology.orchestra.run.smithi039.stdout:1972 read 0x363dcde thru 0x3643caf (0x5fd2 bytes) 2024-09-14T06:16:49.418 INFO:teuthology.orchestra.run.smithi039.stdout:1974 punch from 0x43dd40c to 0x43e0a93, (0x3687 bytes) 2024-09-14T06:16:49.435 INFO:teuthology.orchestra.run.smithi039.stdout:1975 writesame 0xe40b281 thru 0xe40c463 data_size 0x1(0x11e3 bytes) 2024-09-14T06:16:49.435 INFO:teuthology.orchestra.run.smithi039.stdout:1976 punch from 0x739b8df to 0x73a941c, (0xdb3d bytes) 2024-09-14T06:16:49.436 INFO:teuthology.orchestra.run.smithi039.stdout:1977 write 0xc25325e thru 0xc260f4e (0xdcf1 bytes) 2024-09-14T06:16:49.443 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:49.443 INFO:teuthology.orchestra.run.smithi039.stdout:1979 writesame 0xbbed686 thru 0xbbf7ee4 data_size 0x1(0xa85f bytes) 2024-09-14T06:16:49.472 INFO:teuthology.orchestra.run.smithi039.stdout:1981 write 0x7cc6623 thru 0x7ccc281 (0x5c5f bytes) 2024-09-14T06:16:49.472 INFO:teuthology.orchestra.run.smithi039.stdout:1982 write 0xb2c73bc thru 0xb2d2cbb (0xb900 bytes) 2024-09-14T06:16:49.479 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:16:49.479 INFO:teuthology.orchestra.run.smithi039.stdout:1985 write 0xdef8d7e thru 0xdf05172 (0xc3f5 bytes) 2024-09-14T06:16:49.500 INFO:teuthology.orchestra.run.smithi039.stdout:1986 trunc from 0xe40c464 to 0x580dffa 2024-09-14T06:16:49.500 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:49.499+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:49.513 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:16:49.573 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:49.574 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:49.574 INFO:teuthology.orchestra.run.smithi039.stdout:1987 write 0x78a659a thru 0x78b4a73 (0xe4da bytes) 2024-09-14T06:16:49.590 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:49.591+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:49.590 INFO:teuthology.orchestra.run.smithi039.stdout:1988 trunc from 0x78b4a74 to 0x1713123 2024-09-14T06:16:49.595 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:16:49.649 INFO:teuthology.orchestra.run.smithi039.stdout:1989 read 0xc29822 thru 0xc376d9 (0xdeb8 bytes) 2024-09-14T06:16:49.669 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:49.669 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:49.676 INFO:teuthology.orchestra.run.smithi039.stdout:1991 writesame 0x5c76f85 thru 0x5c77c47 data_size 0x3(0xcc3 bytes) 2024-09-14T06:16:49.678 INFO:teuthology.orchestra.run.smithi039.stdout:1992 trunc from 0x5c77c48 to 0x6bbec59 2024-09-14T06:16:49.688 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:49.689 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:49.689 INFO:teuthology.orchestra.run.smithi039.stdout:1993 write 0x406430f thru 0x4073543 (0xf235 bytes) 2024-09-14T06:16:49.690 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:49.691+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:49.694 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:49.694 INFO:teuthology.orchestra.run.smithi039.stdout:1997 read 0x35d34cc thru 0x35dcb0f (0x9644 bytes) 2024-09-14T06:16:49.708 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:49.708 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:49.713 INFO:teuthology.orchestra.run.smithi039.stdout:1998 write 0xc4874e7 thru 0xc49118d (0x9ca7 bytes) 2024-09-14T06:16:49.718 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:49.718 INFO:teuthology.orchestra.run.smithi039.stdout:2001 read 0x43bdad3 thru 0x43cca68 (0xef96 bytes) 2024-09-14T06:16:49.719 INFO:teuthology.orchestra.run.smithi039.stdout:2002 writesame 0x90f7d40 thru 0x90f944f data_size 0x30(0x1710 bytes) 2024-09-14T06:16:49.723 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:49.723 INFO:teuthology.orchestra.run.smithi039.stdout:2006 punch from 0x8d075d6 to 0x8d14f23, (0xd94d bytes) 2024-09-14T06:16:49.723 INFO:teuthology.orchestra.run.smithi039.stdout:2009 punch from 0x5a625f6 to 0x5a70756, (0xe160 bytes) 2024-09-14T06:16:49.723 INFO:teuthology.orchestra.run.smithi039.stdout:2011 read 0x4f3ff01 thru 0x4f45950 (0x5a50 bytes) 2024-09-14T06:16:49.723 INFO:teuthology.orchestra.run.smithi039.stdout:2014 read 0x5be463c thru 0x5bec6f8 (0x80bd bytes) 2024-09-14T06:16:49.724 INFO:teuthology.orchestra.run.smithi039.stdout:2015 trunc from 0xc49118e to 0xbf9e8d5 2024-09-14T06:16:49.729 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:49.729 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:49.742 INFO:teuthology.orchestra.run.smithi039.stdout:2016 writesame 0x3eb919e thru 0x3ec60a3 data_size 0x49(0xcf06 bytes) 2024-09-14T06:16:49.745 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:49.745 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:49.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:49.745 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:49.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.745 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:16:49.745 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:49.745 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 45.667132826, 2024-09-14T06:16:49.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.56009052699999995, 2024-09-14T06:16:49.745 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.746 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.746 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:49.746 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.746 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:16:49.746 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:16:49.746 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 1 2024-09-14T06:16:49.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.746 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:49.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:16:49.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077065+0000", 2024-09-14T06:16:49.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6429999999999999e-06 2024-09-14T06:16:49.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077079+0000", 2024-09-14T06:16:49.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4447000000000001e-05 2024-09-14T06:16:49.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077103+0000", 2024-09-14T06:16:49.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3856e-05 2024-09-14T06:16:49.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077120+0000", 2024-09-14T06:16:49.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7337e-05 2024-09-14T06:16:49.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077255+0000", 2024-09-14T06:16:49.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000134652 2024-09-14T06:16:49.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:16:49.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077274+0000", 2024-09-14T06:16:49.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9003000000000001e-05 2024-09-14T06:16:49.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634272+0000", 2024-09-14T06:16:49.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.55699813099999995 2024-09-14T06:16:49.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634591+0000", 2024-09-14T06:16:49.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031910800000000002 2024-09-14T06:16:49.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:49.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634892+0000", 2024-09-14T06:16:49.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00030066400000000002 2024-09-14T06:16:49.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:49.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636345+0000", 2024-09-14T06:16:49.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001452896 2024-09-14T06:16:49.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:49.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636974+0000", 2024-09-14T06:16:49.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00062927800000000004 2024-09-14T06:16:49.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:49.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637071+0000", 2024-09-14T06:16:49.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7178999999999999e-05 2024-09-14T06:16:49.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637153+0000", 2024-09-14T06:16:49.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1333000000000002e-05 2024-09-14T06:16:49.751 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.751 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.751 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.752 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13136 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.752 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.091799+0000", 2024-09-14T06:16:49.752 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.652395883000001, 2024-09-14T06:16:49.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022628747000000001, 2024-09-14T06:16:49.752 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.752 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.752 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.752 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.753 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.753 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.753 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13136 2024-09-14T06:16:49.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.753 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.753 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091799+0000", 2024-09-14T06:16:49.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.753 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091799+0000", 2024-09-14T06:16:49.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091800+0000", 2024-09-14T06:16:49.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.68e-07 2024-09-14T06:16:49.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091802+0000", 2024-09-14T06:16:49.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6699999999999998e-06 2024-09-14T06:16:49.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091810+0000", 2024-09-14T06:16:49.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6750000000000002e-06 2024-09-14T06:16:49.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.091816+0000", 2024-09-14T06:16:49.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1600000000000003e-06 2024-09-14T06:16:49.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.110345+0000", 2024-09-14T06:16:49.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018528393000000001 2024-09-14T06:16:49.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.111299+0000", 2024-09-14T06:16:49.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00095451100000000001 2024-09-14T06:16:49.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114428+0000", 2024-09-14T06:16:49.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031287699999999999 2024-09-14T06:16:49.756 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.756 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.757 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.757 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13154 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.757 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092189+0000", 2024-09-14T06:16:49.757 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.652006033999999, 2024-09-14T06:16:49.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022359604000000002, 2024-09-14T06:16:49.757 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.757 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.757 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.757 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.757 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.757 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.757 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13154 2024-09-14T06:16:49.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.758 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.758 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092189+0000", 2024-09-14T06:16:49.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.758 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092189+0000", 2024-09-14T06:16:49.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.758 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092189+0000", 2024-09-14T06:16:49.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9700000000000002e-07 2024-09-14T06:16:49.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092192+0000", 2024-09-14T06:16:49.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.317e-06 2024-09-14T06:16:49.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092198+0000", 2024-09-14T06:16:49.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.387e-06 2024-09-14T06:16:49.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092203+0000", 2024-09-14T06:16:49.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.259e-06 2024-09-14T06:16:49.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114338+0000", 2024-09-14T06:16:49.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022135207 2024-09-14T06:16:49.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114451+0000", 2024-09-14T06:16:49.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000112518 2024-09-14T06:16:49.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114548+0000", 2024-09-14T06:16:49.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7518999999999999e-05 2024-09-14T06:16:49.761 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.761 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.761 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.761 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13155 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.761 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092211+0000", 2024-09-14T06:16:49.761 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.65198442, 2024-09-14T06:16:49.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.02255184, 2024-09-14T06:16:49.761 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.761 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.761 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.762 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.762 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.762 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.762 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13155 2024-09-14T06:16:49.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.762 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092211+0000", 2024-09-14T06:16:49.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092211+0000", 2024-09-14T06:16:49.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092211+0000", 2024-09-14T06:16:49.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8500000000000002e-07 2024-09-14T06:16:49.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092213+0000", 2024-09-14T06:16:49.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1380000000000001e-06 2024-09-14T06:16:49.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092220+0000", 2024-09-14T06:16:49.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5250000000000002e-06 2024-09-14T06:16:49.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092224+0000", 2024-09-14T06:16:49.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.843e-06 2024-09-14T06:16:49.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114580+0000", 2024-09-14T06:16:49.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022355455999999999 2024-09-14T06:16:49.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114673+0000", 2024-09-14T06:16:49.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2987999999999998e-05 2024-09-14T06:16:49.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114762+0000", 2024-09-14T06:16:49.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9505000000000004e-05 2024-09-14T06:16:49.765 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.766 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.766 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.766 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13156 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 409600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.766 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092232+0000", 2024-09-14T06:16:49.766 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.651963334, 2024-09-14T06:16:49.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022738728999999999, 2024-09-14T06:16:49.766 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.766 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.767 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.767 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.767 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.767 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.767 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13156 2024-09-14T06:16:49.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.767 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092232+0000", 2024-09-14T06:16:49.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092232+0000", 2024-09-14T06:16:49.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092232+0000", 2024-09-14T06:16:49.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8099999999999998e-07 2024-09-14T06:16:49.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092234+0000", 2024-09-14T06:16:49.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0379999999999998e-06 2024-09-14T06:16:49.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092240+0000", 2024-09-14T06:16:49.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2859999999999996e-06 2024-09-14T06:16:49.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092248+0000", 2024-09-14T06:16:49.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5140000000000004e-06 2024-09-14T06:16:49.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114792+0000", 2024-09-14T06:16:49.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022543758000000001 2024-09-14T06:16:49.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114865+0000", 2024-09-14T06:16:49.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3652000000000006e-05 2024-09-14T06:16:49.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.114970+0000", 2024-09-14T06:16:49.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010509999999999999 2024-09-14T06:16:49.773 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.773 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.773 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.773 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13157 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 524288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.774 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092256+0000", 2024-09-14T06:16:49.774 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.651939407, 2024-09-14T06:16:49.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022964834999999999, 2024-09-14T06:16:49.774 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.774 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.774 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.774 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.775 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.775 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.775 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13157 2024-09-14T06:16:49.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.775 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092256+0000", 2024-09-14T06:16:49.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092256+0000", 2024-09-14T06:16:49.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092256+0000", 2024-09-14T06:16:49.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6199999999999998e-07 2024-09-14T06:16:49.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092258+0000", 2024-09-14T06:16:49.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3530000000000002e-06 2024-09-14T06:16:49.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092265+0000", 2024-09-14T06:16:49.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5069999999999997e-06 2024-09-14T06:16:49.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092270+0000", 2024-09-14T06:16:49.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2530000000000004e-06 2024-09-14T06:16:49.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115000+0000", 2024-09-14T06:16:49.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022729787000000001 2024-09-14T06:16:49.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115074+0000", 2024-09-14T06:16:49.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4506999999999997e-05 2024-09-14T06:16:49.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115220+0000", 2024-09-14T06:16:49.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000145966 2024-09-14T06:16:49.778 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.778 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.778 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.778 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13158 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.778 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092282+0000", 2024-09-14T06:16:49.779 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.651912534999999, 2024-09-14T06:16:49.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.023233365999999998, 2024-09-14T06:16:49.779 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.779 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.779 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.779 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.779 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.779 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.779 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13158 2024-09-14T06:16:49.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.779 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092282+0000", 2024-09-14T06:16:49.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092282+0000", 2024-09-14T06:16:49.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092283+0000", 2024-09-14T06:16:49.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:16:49.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092285+0000", 2024-09-14T06:16:49.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3460000000000001e-06 2024-09-14T06:16:49.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092291+0000", 2024-09-14T06:16:49.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0549999999999996e-06 2024-09-14T06:16:49.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092296+0000", 2024-09-14T06:16:49.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1800000000000004e-06 2024-09-14T06:16:49.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115253+0000", 2024-09-14T06:16:49.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022956764000000001 2024-09-14T06:16:49.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115347+0000", 2024-09-14T06:16:49.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3762000000000001e-05 2024-09-14T06:16:49.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115516+0000", 2024-09-14T06:16:49.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016881200000000001 2024-09-14T06:16:49.783 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.783 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.783 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.783 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13159 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.783 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092307+0000", 2024-09-14T06:16:49.783 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.651888399000001, 2024-09-14T06:16:49.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.023865384999999999, 2024-09-14T06:16:49.783 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.783 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.783 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.783 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.784 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.784 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.784 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13159 2024-09-14T06:16:49.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.784 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092307+0000", 2024-09-14T06:16:49.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092307+0000", 2024-09-14T06:16:49.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092307+0000", 2024-09-14T06:16:49.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:16:49.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092309+0000", 2024-09-14T06:16:49.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3080000000000002e-06 2024-09-14T06:16:49.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092316+0000", 2024-09-14T06:16:49.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7120000000000003e-06 2024-09-14T06:16:49.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092321+0000", 2024-09-14T06:16:49.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7729999999999999e-06 2024-09-14T06:16:49.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.787 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115546+0000", 2024-09-14T06:16:49.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.023225000999999999 2024-09-14T06:16:49.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.115621+0000", 2024-09-14T06:16:49.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4981999999999995e-05 2024-09-14T06:16:49.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116172+0000", 2024-09-14T06:16:49.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00055117199999999995 2024-09-14T06:16:49.789 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.789 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.789 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.790 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13160 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.790 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092329+0000", 2024-09-14T06:16:49.790 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.651866242000001, 2024-09-14T06:16:49.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024062849000000001, 2024-09-14T06:16:49.790 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.790 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.790 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.790 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.791 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.791 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.791 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13160 2024-09-14T06:16:49.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.791 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092329+0000", 2024-09-14T06:16:49.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092329+0000", 2024-09-14T06:16:49.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092329+0000", 2024-09-14T06:16:49.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8299999999999997e-07 2024-09-14T06:16:49.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092331+0000", 2024-09-14T06:16:49.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:16:49.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092338+0000", 2024-09-14T06:16:49.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:16:49.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092351+0000", 2024-09-14T06:16:49.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3059e-05 2024-09-14T06:16:49.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116204+0000", 2024-09-14T06:16:49.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.023853268 2024-09-14T06:16:49.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116287+0000", 2024-09-14T06:16:49.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3140999999999996e-05 2024-09-14T06:16:49.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116392+0000", 2024-09-14T06:16:49.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010451900000000001 2024-09-14T06:16:49.795 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.795 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.795 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.795 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13161 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.795 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092359+0000", 2024-09-14T06:16:49.795 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.651835844000001, 2024-09-14T06:16:49.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024248515000000002, 2024-09-14T06:16:49.796 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.796 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.796 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.796 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.796 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.796 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.796 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13161 2024-09-14T06:16:49.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.796 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.796 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092359+0000", 2024-09-14T06:16:49.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092359+0000", 2024-09-14T06:16:49.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092360+0000", 2024-09-14T06:16:49.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:16:49.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092368+0000", 2024-09-14T06:16:49.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1140000000000006e-06 2024-09-14T06:16:49.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092375+0000", 2024-09-14T06:16:49.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4399999999999999e-06 2024-09-14T06:16:49.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092380+0000", 2024-09-14T06:16:49.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3170000000000001e-06 2024-09-14T06:16:49.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116426+0000", 2024-09-14T06:16:49.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024045492000000002 2024-09-14T06:16:49.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116513+0000", 2024-09-14T06:16:49.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6936999999999998e-05 2024-09-14T06:16:49.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116608+0000", 2024-09-14T06:16:49.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4765000000000005e-05 2024-09-14T06:16:49.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.800 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.800 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13162 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.800 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092389+0000", 2024-09-14T06:16:49.800 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.651806032, 2024-09-14T06:16:49.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024432156, 2024-09-14T06:16:49.801 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.801 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.801 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.801 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.801 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.801 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.801 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13162 2024-09-14T06:16:49.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.801 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092389+0000", 2024-09-14T06:16:49.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092389+0000", 2024-09-14T06:16:49.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092389+0000", 2024-09-14T06:16:49.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:16:49.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092392+0000", 2024-09-14T06:16:49.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3460000000000001e-06 2024-09-14T06:16:49.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092398+0000", 2024-09-14T06:16:49.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6789999999999999e-06 2024-09-14T06:16:49.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092403+0000", 2024-09-14T06:16:49.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9389999999999996e-06 2024-09-14T06:16:49.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116657+0000", 2024-09-14T06:16:49.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024253896 2024-09-14T06:16:49.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.804 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.804 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116732+0000", 2024-09-14T06:16:49.804 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4727000000000005e-05 2024-09-14T06:16:49.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.804 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.804 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116821+0000", 2024-09-14T06:16:49.804 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:16:49.804 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.804 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.804 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.805 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13163 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.805 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092410+0000", 2024-09-14T06:16:49.805 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.651784574000001, 2024-09-14T06:16:49.805 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024727810999999999, 2024-09-14T06:16:49.805 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.806 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.806 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.806 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.806 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.806 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.807 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13163 2024-09-14T06:16:49.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.807 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.807 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.808 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092410+0000", 2024-09-14T06:16:49.808 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.808 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.808 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.808 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092410+0000", 2024-09-14T06:16:49.809 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.809 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.809 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.809 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092411+0000", 2024-09-14T06:16:49.809 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2899999999999999e-07 2024-09-14T06:16:49.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.810 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.810 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.810 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092413+0000", 2024-09-14T06:16:49.810 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3649999999999998e-06 2024-09-14T06:16:49.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.810 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.810 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.811 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092420+0000", 2024-09-14T06:16:49.811 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.5729999999999996e-06 2024-09-14T06:16:49.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.811 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.811 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092425+0000", 2024-09-14T06:16:49.811 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0669999999999999e-06 2024-09-14T06:16:49.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.811 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.811 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.116850+0000", 2024-09-14T06:16:49.811 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024424833999999999 2024-09-14T06:16:49.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.812 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.812 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117058+0000", 2024-09-14T06:16:49.812 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020800199999999999 2024-09-14T06:16:49.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.812 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.812 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.812 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117138+0000", 2024-09-14T06:16:49.812 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0541e-05 2024-09-14T06:16:49.812 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.812 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.813 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.813 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.813 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13164 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.813 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092432+0000", 2024-09-14T06:16:49.813 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.651762897999999, 2024-09-14T06:16:49.813 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024878806999999999, 2024-09-14T06:16:49.813 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.813 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.813 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.813 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.813 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.813 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.814 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13164 2024-09-14T06:16:49.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.814 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.814 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.814 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092432+0000", 2024-09-14T06:16:49.814 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.814 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.814 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092432+0000", 2024-09-14T06:16:49.814 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.814 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.815 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092432+0000", 2024-09-14T06:16:49.815 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:16:49.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.815 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.815 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092434+0000", 2024-09-14T06:16:49.815 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.968e-06 2024-09-14T06:16:49.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.815 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.815 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092440+0000", 2024-09-14T06:16:49.815 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.964e-06 2024-09-14T06:16:49.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.816 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.816 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092445+0000", 2024-09-14T06:16:49.816 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9529999999999998e-06 2024-09-14T06:16:49.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.816 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.816 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.816 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117166+0000", 2024-09-14T06:16:49.816 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024720925000000001 2024-09-14T06:16:49.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.816 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.816 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.816 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117239+0000", 2024-09-14T06:16:49.816 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2650000000000004e-05 2024-09-14T06:16:49.817 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.817 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.817 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.817 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117311+0000", 2024-09-14T06:16:49.817 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1945000000000003e-05 2024-09-14T06:16:49.817 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.817 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.817 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.817 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.817 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.817 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13165 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.817 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092453+0000", 2024-09-14T06:16:49.817 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.651742217000001, 2024-09-14T06:16:49.818 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025052076999999999, 2024-09-14T06:16:49.818 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.818 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.818 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.818 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.818 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.818 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.818 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13165 2024-09-14T06:16:49.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.818 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.818 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.818 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.818 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092453+0000", 2024-09-14T06:16:49.818 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.819 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.819 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.819 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092453+0000", 2024-09-14T06:16:49.819 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.819 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.819 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.819 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092453+0000", 2024-09-14T06:16:49.819 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7300000000000001e-07 2024-09-14T06:16:49.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.819 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.820 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.820 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092455+0000", 2024-09-14T06:16:49.820 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.204e-06 2024-09-14T06:16:49.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.820 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.820 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.820 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092461+0000", 2024-09-14T06:16:49.820 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0299999999999999e-06 2024-09-14T06:16:49.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.821 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.821 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.821 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092466+0000", 2024-09-14T06:16:49.821 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7820000000000001e-06 2024-09-14T06:16:49.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.821 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.821 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.821 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117343+0000", 2024-09-14T06:16:49.821 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024876334999999999 2024-09-14T06:16:49.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.821 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.821 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.821 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117416+0000", 2024-09-14T06:16:49.821 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3368000000000006e-05 2024-09-14T06:16:49.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.822 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.822 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117505+0000", 2024-09-14T06:16:49.822 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8885e-05 2024-09-14T06:16:49.822 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.822 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.822 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.822 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13166 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.822 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092490+0000", 2024-09-14T06:16:49.822 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.651705054000001, 2024-09-14T06:16:49.822 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025236825000000001, 2024-09-14T06:16:49.822 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.823 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.823 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.823 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.823 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.823 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.823 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13166 2024-09-14T06:16:49.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.823 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.823 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.823 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092490+0000", 2024-09-14T06:16:49.823 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.824 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.824 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092490+0000", 2024-09-14T06:16:49.824 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.824 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.824 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092490+0000", 2024-09-14T06:16:49.824 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6199999999999999e-07 2024-09-14T06:16:49.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.824 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.824 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092492+0000", 2024-09-14T06:16:49.824 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7910000000000001e-06 2024-09-14T06:16:49.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.825 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.825 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092497+0000", 2024-09-14T06:16:49.825 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4190000000000001e-06 2024-09-14T06:16:49.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.825 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.825 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092501+0000", 2024-09-14T06:16:49.825 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0230000000000003e-06 2024-09-14T06:16:49.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.825 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.825 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117533+0000", 2024-09-14T06:16:49.825 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025031623999999999 2024-09-14T06:16:49.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.826 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.826 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.826 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117638+0000", 2024-09-14T06:16:49.826 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010451900000000001 2024-09-14T06:16:49.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.826 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.826 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.826 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117727+0000", 2024-09-14T06:16:49.826 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9086999999999996e-05 2024-09-14T06:16:49.826 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.826 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.826 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.827 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13167 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.827 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092507+0000", 2024-09-14T06:16:49.827 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.651687566, 2024-09-14T06:16:49.827 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025412396, 2024-09-14T06:16:49.827 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.827 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.827 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.827 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.827 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.827 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.827 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13167 2024-09-14T06:16:49.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.827 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.828 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.828 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.828 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092507+0000", 2024-09-14T06:16:49.828 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.828 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.828 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.828 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092507+0000", 2024-09-14T06:16:49.828 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.828 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.828 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.828 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092508+0000", 2024-09-14T06:16:49.828 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.65e-07 2024-09-14T06:16:49.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.829 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.829 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092509+0000", 2024-09-14T06:16:49.829 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.426e-06 2024-09-14T06:16:49.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.829 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.829 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092512+0000", 2024-09-14T06:16:49.829 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.416e-06 2024-09-14T06:16:49.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.829 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.829 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092515+0000", 2024-09-14T06:16:49.830 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.971e-06 2024-09-14T06:16:49.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.830 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.830 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117755+0000", 2024-09-14T06:16:49.830 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.02523999 2024-09-14T06:16:49.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.830 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.830 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117830+0000", 2024-09-14T06:16:49.830 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4386000000000002e-05 2024-09-14T06:16:49.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.830 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.831 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117920+0000", 2024-09-14T06:16:49.831 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9942e-05 2024-09-14T06:16:49.831 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.831 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.831 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.831 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.831 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13168 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.831 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092519+0000", 2024-09-14T06:16:49.831 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.651675432999999, 2024-09-14T06:16:49.831 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025607092000000001, 2024-09-14T06:16:49.831 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.831 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.831 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.831 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.832 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.832 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.832 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13168 2024-09-14T06:16:49.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.832 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.832 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.832 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.832 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092519+0000", 2024-09-14T06:16:49.832 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.832 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.832 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.832 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092519+0000", 2024-09-14T06:16:49.832 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.833 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.833 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092520+0000", 2024-09-14T06:16:49.833 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4999999999999999e-07 2024-09-14T06:16:49.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.833 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.833 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092521+0000", 2024-09-14T06:16:49.833 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2440000000000001e-06 2024-09-14T06:16:49.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.833 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.833 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092524+0000", 2024-09-14T06:16:49.834 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3189999999999999e-06 2024-09-14T06:16:49.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.834 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.834 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092527+0000", 2024-09-14T06:16:49.834 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.638e-06 2024-09-14T06:16:49.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.834 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.834 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.117948+0000", 2024-09-14T06:16:49.834 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.025421293000000001 2024-09-14T06:16:49.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.834 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.835 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.118025+0000", 2024-09-14T06:16:49.835 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6410999999999997e-05 2024-09-14T06:16:49.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.835 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.835 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.118127+0000", 2024-09-14T06:16:49.835 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000101937 2024-09-14T06:16:49.835 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.835 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.835 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.835 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13169 3.10 3:0d935005:::rbd_data.1152f87cbac6.000000000000001a:6 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.835 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.092531+0000", 2024-09-14T06:16:49.836 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.651664236, 2024-09-14T06:16:49.836 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.030222784999999999, 2024-09-14T06:16:49.836 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.836 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.836 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:49.836 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.836 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.836 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.836 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13169 2024-09-14T06:16:49.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.836 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.836 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.837 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.837 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092531+0000", 2024-09-14T06:16:49.837 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.837 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.837 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.837 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092531+0000", 2024-09-14T06:16:49.837 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.838 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.838 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.838 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092531+0000", 2024-09-14T06:16:49.838 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.41e-07 2024-09-14T06:16:49.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.838 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.838 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.838 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092545+0000", 2024-09-14T06:16:49.838 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4256999999999999e-05 2024-09-14T06:16:49.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.838 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.838 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.838 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092553+0000", 2024-09-14T06:16:49.838 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.374e-06 2024-09-14T06:16:49.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.839 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.839 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.839 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.092558+0000", 2024-09-14T06:16:49.839 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2830000000000001e-06 2024-09-14T06:16:49.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.839 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.839 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.839 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.122584+0000", 2024-09-14T06:16:49.839 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.030026091000000001 2024-09-14T06:16:49.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.839 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.839 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.839 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.122672+0000", 2024-09-14T06:16:49.839 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7880999999999994e-05 2024-09-14T06:16:49.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.840 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.840 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.840 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.122753+0000", 2024-09-14T06:16:49.840 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1657999999999996e-05 2024-09-14T06:16:49.840 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.840 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.840 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.840 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.840 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:16984 3.1c 3:3aec2de0:::rbd_data.1152fc09e7e6.000000000000000d:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:49.840 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:37.655478+0000", 2024-09-14T06:16:49.840 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.088716901, 2024-09-14T06:16:49.840 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.034057654999999999, 2024-09-14T06:16:49.841 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.841 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.841 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:49.841 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.841 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.841 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.841 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 16984 2024-09-14T06:16:49.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.841 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.841 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.841 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.841 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655478+0000", 2024-09-14T06:16:49.841 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.842 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.842 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.842 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655478+0000", 2024-09-14T06:16:49.842 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.842 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.842 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.842 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655480+0000", 2024-09-14T06:16:49.842 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.074e-06 2024-09-14T06:16:49.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.842 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.842 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.842 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655489+0000", 2024-09-14T06:16:49.842 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2799999999999992e-06 2024-09-14T06:16:49.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.843 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.843 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.843 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655509+0000", 2024-09-14T06:16:49.843 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9494e-05 2024-09-14T06:16:49.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.843 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.843 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.843 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655518+0000", 2024-09-14T06:16:49.843 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7730000000000005e-06 2024-09-14T06:16:49.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.843 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.843 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.843 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655599+0000", 2024-09-14T06:16:49.843 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1306999999999999e-05 2024-09-14T06:16:49.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.844 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.844 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.844 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.655708+0000", 2024-09-14T06:16:49.844 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010870300000000001 2024-09-14T06:16:49.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.844 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.844 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:49.844 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.661376+0000", 2024-09-14T06:16:49.844 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056679460000000001 2024-09-14T06:16:49.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.844 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.844 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:49.844 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.689383+0000", 2024-09-14T06:16:49.845 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.028007632000000001 2024-09-14T06:16:49.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.845 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.845 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:49.845 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.689407+0000", 2024-09-14T06:16:49.845 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3743e-05 2024-09-14T06:16:49.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.845 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.845 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:49.845 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.689476+0000", 2024-09-14T06:16:49.845 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9423000000000003e-05 2024-09-14T06:16:49.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.845 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.845 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.846 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:37.689536+0000", 2024-09-14T06:16:49.846 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9280000000000002e-05 2024-09-14T06:16:49.846 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.846 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.846 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.846 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.846 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:18695 3.10 3:0c23de83:::rbd_data.1152227be94f.000000000000001a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:16:49.847 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:42.335500+0000", 2024-09-14T06:16:49.847 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 7.4086954120000001, 2024-09-14T06:16:49.847 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.060751287000000001, 2024-09-14T06:16:49.847 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:49.847 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:49.847 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:49.847 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:49.847 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:49.847 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:49.847 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 18695 2024-09-14T06:16:49.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.847 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:49.848 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.848 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:49.848 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.335500+0000", 2024-09-14T06:16:49.848 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.848 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.848 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:49.848 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.335500+0000", 2024-09-14T06:16:49.848 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:49.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.848 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.848 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:49.848 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.335502+0000", 2024-09-14T06:16:49.849 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9870000000000002e-06 2024-09-14T06:16:49.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.849 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.849 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:49.849 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.335512+0000", 2024-09-14T06:16:49.849 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0251999999999999e-05 2024-09-14T06:16:49.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.849 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.849 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:49.849 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.335540+0000", 2024-09-14T06:16:49.849 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8411999999999998e-05 2024-09-14T06:16:49.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.849 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.849 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:49.850 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.335557+0000", 2024-09-14T06:16:49.850 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7030000000000001e-05 2024-09-14T06:16:49.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.850 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.850 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:49.850 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.335654+0000", 2024-09-14T06:16:49.850 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7113000000000003e-05 2024-09-14T06:16:49.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.850 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.850 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:49.850 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.335737+0000", 2024-09-14T06:16:49.850 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2662000000000002e-05 2024-09-14T06:16:49.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.850 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.851 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:16:49.851 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.343481+0000", 2024-09-14T06:16:49.851 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0077441410000000004 2024-09-14T06:16:49.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.851 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.851 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:49.851 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.395863+0000", 2024-09-14T06:16:49.851 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052381843999999997 2024-09-14T06:16:49.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.851 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.851 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:49.851 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.395917+0000", 2024-09-14T06:16:49.851 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.397e-05 2024-09-14T06:16:49.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.852 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.852 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:49.852 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.396119+0000", 2024-09-14T06:16:49.852 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020236900000000001 2024-09-14T06:16:49.852 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:49.852 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:49.852 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:49.852 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.396251+0000", 2024-09-14T06:16:49.852 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000131507 2024-09-14T06:16:49.852 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.852 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.852 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.852 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:49.853 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:49.853 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:49.853 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:49.854 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:49.855 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:16:49.855 INFO:teuthology.orchestra.run.smithi039.stdout:2017 writesame 0xe7b97f4 thru 0xe7c1c31 data_size 0x2(0x843e bytes) 2024-09-14T06:16:49.855 INFO:teuthology.orchestra.run.smithi039.stdout:2018 read 0x1dca17a thru 0x1dd98fc (0xf783 bytes) 2024-09-14T06:16:49.855 INFO:teuthology.orchestra.run.smithi039.stdout:2019 trunc from 0xe7c1c32 to 0x89abec8 2024-09-14T06:16:49.856 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:49.856 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:49.856 INFO:teuthology.orchestra.run.smithi039.stdout:2020 write 0x20d49db thru 0x20daebc (0x64e2 bytes) 2024-09-14T06:16:49.857 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:49.857 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:49.857 INFO:teuthology.orchestra.run.smithi039.stdout:2022 write 0xd5613b1 thru 0xd5688ec (0x753c bytes) 2024-09-14T06:16:49.858 INFO:teuthology.orchestra.run.smithi039.stdout:2023 read 0x2462943 thru 0x2466219 (0x38d7 bytes) 2024-09-14T06:16:49.858 INFO:teuthology.orchestra.run.smithi039.stdout:2024 read 0x195fe3a thru 0x195ff25 (0xec bytes) 2024-09-14T06:16:49.858 INFO:teuthology.orchestra.run.smithi039.stdout:2025 writesame 0x74db08 thru 0x757468 data_size 0x1(0x9961 bytes) 2024-09-14T06:16:49.858 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:49.859 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:49.859 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:49.851+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:49.861 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:49.862 INFO:teuthology.orchestra.run.smithi039.stdout:2027 write 0x9feb3ea thru 0x9ff2cb8 (0x78cf bytes) 2024-09-14T06:16:49.867 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:49.867 INFO:teuthology.orchestra.run.smithi039.stdout:2029 read 0x44c669a thru 0x44c8d4e (0x26b5 bytes) 2024-09-14T06:16:49.867 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:49.867 INFO:teuthology.orchestra.run.smithi039.stdout:2031 punch from 0x7fb57a3 to 0x7fb950c, (0x3d69 bytes) 2024-09-14T06:16:49.867 INFO:teuthology.orchestra.run.smithi039.stdout:2032 write 0x85d28c2 thru 0x85d3b1c (0x125b bytes) 2024-09-14T06:16:49.877 INFO:teuthology.orchestra.run.smithi039.stdout:2033 trunc from 0xd5688ed to 0xb2535f2 2024-09-14T06:16:49.896 INFO:teuthology.orchestra.run.smithi039.stdout:2034 write 0x431b84a thru 0x431c9a6 (0x115d bytes) 2024-09-14T06:16:49.901 INFO:teuthology.orchestra.run.smithi039.stdout:2035 read 0x5f53ecf thru 0x5f62eae (0xefe0 bytes) 2024-09-14T06:16:49.901 INFO:teuthology.orchestra.run.smithi039.stdout:2036 write 0x159057d thru 0x159bb5a (0xb5de bytes) 2024-09-14T06:16:49.903 INFO:teuthology.orchestra.run.smithi039.stdout:2038 writesame 0x1dd0fef thru 0x1dd713f data_size 0x1(0x6151 bytes) 2024-09-14T06:16:49.937 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:49.941 INFO:teuthology.orchestra.run.smithi039.stdout:2040 trunc from 0xb2535f2 to 0x8904acb 2024-09-14T06:16:49.955 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:49.951+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:49.963 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:49.964 INFO:teuthology.orchestra.run.smithi039.stdout:2043 write 0x2abbf75 thru 0x2ac448b (0x8517 bytes) 2024-09-14T06:16:49.973 INFO:teuthology.orchestra.run.smithi039.stdout:2044 punch from 0x8735c85 to 0x873602c, (0x3a7 bytes) 2024-09-14T06:16:49.973 INFO:teuthology.orchestra.run.smithi039.stdout:2045 punch from 0x20d7f06 to 0x20d9fb8, (0x20b2 bytes) 2024-09-14T06:16:49.974 INFO:teuthology.orchestra.run.smithi039.stdout:2046 read 0x3d28b59 thru 0x3d2f430 (0x68d8 bytes) 2024-09-14T06:16:49.974 INFO:teuthology.orchestra.run.smithi039.stdout:2047 read 0x5ff67aa thru 0x6005549 (0xeda0 bytes) 2024-09-14T06:16:49.974 INFO:teuthology.orchestra.run.smithi039.stdout:2049 read 0x1d85155 thru 0x1d8aa9b (0x5947 bytes) 2024-09-14T06:16:49.974 INFO:teuthology.orchestra.run.smithi039.stdout:2051 write 0x6a524de thru 0x6a5467c (0x219f bytes) 2024-09-14T06:16:49.977 INFO:teuthology.orchestra.run.smithi039.stdout:2052 writesame 0x1cc16f thru 0x1daca2 data_size 0x2(0xeb34 bytes) 2024-09-14T06:16:49.985 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:49.985 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:49.985 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:49.985 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:50.000 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:16:50.036 INFO:teuthology.orchestra.run.smithi039.stdout:2053 write 0xc783ffc thru 0xc78e953 (0xa958 bytes) 2024-09-14T06:16:50.042 INFO:teuthology.orchestra.run.smithi039.stdout:2054 read 0x96687b0 thru 0x96772e2 (0xeb33 bytes) 2024-09-14T06:16:50.042 INFO:teuthology.orchestra.run.smithi039.stdout:2057 punch from 0x267c03a to 0x268072d, (0x46f3 bytes) 2024-09-14T06:16:50.042 INFO:teuthology.orchestra.run.smithi039.stdout:2058 punch from 0x2d0e2d9 to 0x2d17115, (0x8e3c bytes) 2024-09-14T06:16:50.042 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:50.042 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:50.054 INFO:teuthology.orchestra.run.smithi039.stdout:2064 trunc from 0xc78e954 to 0x50177e2 2024-09-14T06:16:50.056 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:50.055+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:50.072 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:50.072 INFO:teuthology.orchestra.run.smithi039.stdout:2066 punch from 0x6adbb3 to 0x6b50d5, (0x7522 bytes) 2024-09-14T06:16:50.072 INFO:teuthology.orchestra.run.smithi039.stdout:2067 punch from 0x1645ec5 to 0x165280d, (0xc948 bytes) 2024-09-14T06:16:50.095 INFO:teuthology.orchestra.run.smithi039.stdout:2068 write 0xbb1eecd thru 0xbb25f3d (0x7071 bytes) 2024-09-14T06:16:50.103 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:50.103 INFO:teuthology.orchestra.run.smithi039.stdout:2072 write 0x8b61aed thru 0x8b6731d (0x5831 bytes) 2024-09-14T06:16:50.110 INFO:teuthology.orchestra.run.smithi039.stdout:2073 write 0x7dd666f thru 0x7dd6eb8 (0x84a bytes) 2024-09-14T06:16:50.115 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:50.115 INFO:teuthology.orchestra.run.smithi039.stdout:2077 read 0x1a7b868 thru 0x1a818b0 (0x6049 bytes) 2024-09-14T06:16:50.116 INFO:teuthology.orchestra.run.smithi039.stdout:2078 read 0x6a9046c thru 0x6a96237 (0x5dcc bytes) 2024-09-14T06:16:50.116 INFO:teuthology.orchestra.run.smithi039.stdout:2080 read 0x25fc93b thru 0x2606696 (0x9d5c bytes) 2024-09-14T06:16:50.117 INFO:teuthology.orchestra.run.smithi039.stdout:2081 write 0x4d3a1e5 thru 0x4d494f1 (0xf30d bytes) 2024-09-14T06:16:50.126 INFO:teuthology.orchestra.run.smithi039.stdout:2082 write 0x6e049cc thru 0x6e096cf (0x4d04 bytes) 2024-09-14T06:16:50.131 INFO:teuthology.orchestra.run.smithi039.stdout:2083 write 0x6903c25 thru 0x691132e (0xd70a bytes) 2024-09-14T06:16:50.136 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:50.136 INFO:teuthology.orchestra.run.smithi039.stdout:2087 read 0x1221581 thru 0x122c390 (0xae10 bytes) 2024-09-14T06:16:50.137 INFO:teuthology.orchestra.run.smithi039.stdout:2088 punch from 0x1796231 to 0x17a2bab, (0xc97a bytes) 2024-09-14T06:16:50.137 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:50.152 INFO:teuthology.orchestra.run.smithi039.stdout:2090 writesame 0xd62f72d thru 0xd6319d0 data_size 0x6(0x22a4 bytes) 2024-09-14T06:16:50.152 INFO:teuthology.orchestra.run.smithi039.stdout:2092 read 0xf8b21a thru 0xf901c3 (0x4faa bytes) 2024-09-14T06:16:50.154 INFO:teuthology.orchestra.run.smithi039.stdout:2093 read 0x3bd8323 thru 0x3bd8bf5 (0x8d3 bytes) 2024-09-14T06:16:50.154 INFO:teuthology.orchestra.run.smithi039.stdout:2094 read 0x79da183 thru 0x79e2d5f (0x8bdd bytes) 2024-09-14T06:16:50.154 INFO:teuthology.orchestra.run.smithi039.stdout:2095 punch from 0x3536d78 to 0x353e748, (0x79d0 bytes) 2024-09-14T06:16:50.154 INFO:teuthology.orchestra.run.smithi039.stdout:2096 read 0x83fbb2a thru 0x840a513 (0xe9ea bytes) 2024-09-14T06:16:50.154 INFO:teuthology.orchestra.run.smithi039.stdout:2098 read 0x537a74e thru 0x5389e3f (0xf6f2 bytes) 2024-09-14T06:16:50.155 INFO:teuthology.orchestra.run.smithi039.stdout:2099 trunc from 0xd6319d1 to 0xcf555a9 2024-09-14T06:16:50.157 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:50.155+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:50.166 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:50.167 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:50.167 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:50.167 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:50.167 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:50.169 INFO:teuthology.orchestra.run.smithi039.stdout:2100 write 0xb0f2a70 thru 0xb0ffaa8 (0xd039 bytes) 2024-09-14T06:16:50.178 INFO:teuthology.orchestra.run.smithi039.stdout:2101 read 0x10549b8 thru 0x10612a0 (0xc8e9 bytes) 2024-09-14T06:16:50.179 INFO:teuthology.orchestra.run.smithi039.stdout:2102 read 0x23b7ffc thru 0x23c489f (0xc8a4 bytes) 2024-09-14T06:16:50.179 INFO:teuthology.orchestra.run.smithi039.stdout:2103 punch from 0x43410eb to 0x4347c6e, (0x6b83 bytes) 2024-09-14T06:16:50.182 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:16:50.189 INFO:teuthology.orchestra.run.smithi039.stdout:2105 trunc from 0xcf555a9 to 0x5c5beae 2024-09-14T06:16:50.211 INFO:teuthology.orchestra.run.smithi039.stdout:2107 read 0x4f6fb23 thru 0x4f77a18 (0x7ef6 bytes) 2024-09-14T06:16:50.211 INFO:teuthology.orchestra.run.smithi039.stdout:2108 write 0x6f76d thru 0x76b5d (0x73f1 bytes) 2024-09-14T06:16:50.220 INFO:teuthology.orchestra.run.smithi039.stdout:2110 write 0x24643fd thru 0x2469dc0 (0x59c4 bytes) 2024-09-14T06:16:50.225 INFO:teuthology.orchestra.run.smithi039.stdout:2111 punch from 0x15e6fcb to 0x15ea44e, (0x3483 bytes) 2024-09-14T06:16:50.243 INFO:teuthology.orchestra.run.smithi039.stdout:2112 write 0xa9a730f thru 0xa9b1743 (0xa435 bytes) 2024-09-14T06:16:50.249 INFO:teuthology.orchestra.run.smithi039.stdout:2113 punch from 0x83cf8e to 0x846c1c, (0x9c8e bytes) 2024-09-14T06:16:50.249 INFO:teuthology.orchestra.run.smithi039.stdout:2114 punch from 0x3f4fc75 to 0x3f55fbb, (0x6346 bytes) 2024-09-14T06:16:50.249 INFO:teuthology.orchestra.run.smithi039.stdout:2115 write 0x7fbfd1 thru 0x8061fc (0xa22c bytes) 2024-09-14T06:16:50.257 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:50.255+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:50.258 INFO:teuthology.orchestra.run.smithi039.stdout:2118 read 0x8a3a0bc thru 0x8a3f6c3 (0x5608 bytes) 2024-09-14T06:16:50.258 INFO:teuthology.orchestra.run.smithi039.stdout:2120 read 0x8c69366 thru 0x8c760ed (0xcd88 bytes) 2024-09-14T06:16:50.258 INFO:teuthology.orchestra.run.smithi039.stdout:2122 read 0x5436808 thru 0x543ad7c (0x4575 bytes) 2024-09-14T06:16:50.258 INFO:teuthology.orchestra.run.smithi039.stdout:2124 writesame 0x2393475 thru 0x239975c data_size 0x3c(0x62e8 bytes) 2024-09-14T06:16:50.278 INFO:teuthology.orchestra.run.smithi039.stdout:2125 trunc from 0xa9b1744 to 0x3477522 2024-09-14T06:16:50.299 INFO:teuthology.orchestra.run.smithi039.stdout:2126 read 0x299ab69 thru 0x29a2151 (0x75e9 bytes) 2024-09-14T06:16:50.327 INFO:teuthology.orchestra.run.smithi039.stdout:2128 write 0xa577666 thru 0xa578f20 (0x18bb bytes) 2024-09-14T06:16:50.329 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:50.330 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:50.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:50.330 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:50.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.330 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.5 30/29 [PushOp(3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head, version: 28'5, data_included: [0~212992,360448~311296], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head@28'5, size: 1032192, copy_subset: [(0, 1032192)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1032192, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:16:50.330 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:50.330 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.051245494, 2024-09-14T06:16:50.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041630249999999999, 2024-09-14T06:16:50.330 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.330 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.330 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:50.330 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:50.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:50.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275888+0000", 2024-09-14T06:16:50.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.367e-06 2024-09-14T06:16:50.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277098+0000", 2024-09-14T06:16:50.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001210661 2024-09-14T06:16:50.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277122+0000", 2024-09-14T06:16:50.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3413000000000001e-05 2024-09-14T06:16:50.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277141+0000", 2024-09-14T06:16:50.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9133000000000001e-05 2024-09-14T06:16:50.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.333 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277364+0000", 2024-09-14T06:16:50.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000223365 2024-09-14T06:16:50.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.333 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277405+0000", 2024-09-14T06:16:50.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1053000000000001e-05 2024-09-14T06:16:50.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.334 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.280049+0000", 2024-09-14T06:16:50.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026440330000000001 2024-09-14T06:16:50.334 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.334 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.334 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.334 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.16 30/29 [PushOp(3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head, version: 28'3, data_included: [0~212992,344064~327680], data_size: 540672, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head@28'3, size: 1032192, copy_subset: [(0, 1032192)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1032192, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:16:50.334 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.287894+0000", 2024-09-14T06:16:50.334 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.039237301, 2024-09-14T06:16:50.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032539600000000002, 2024-09-14T06:16:50.334 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.334 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.335 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:50.335 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.287894+0000", 2024-09-14T06:16:50.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.287894+0000", 2024-09-14T06:16:50.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.287899+0000", 2024-09-14T06:16:50.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7280000000000003e-06 2024-09-14T06:16:50.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.337 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.337 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.288805+0000", 2024-09-14T06:16:50.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00090622700000000001 2024-09-14T06:16:50.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.337 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.337 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.288855+0000", 2024-09-14T06:16:50.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9835e-05 2024-09-14T06:16:50.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.288902+0000", 2024-09-14T06:16:50.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6854999999999998e-05 2024-09-14T06:16:50.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.289120+0000", 2024-09-14T06:16:50.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000217495 2024-09-14T06:16:50.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.289142+0000", 2024-09-14T06:16:50.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2870999999999998e-05 2024-09-14T06:16:50.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.339 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.291148+0000", 2024-09-14T06:16:50.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020059489999999999 2024-09-14T06:16:50.339 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.339 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.339 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.339 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3868 3.16 3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head [stat,truncate 1998848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:50.339 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:50.339 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.824650124000001, 2024-09-14T06:16:50.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034445919999999998, 2024-09-14T06:16:50.339 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.340 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.340 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.340 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.340 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.340 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.340 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3868 2024-09-14T06:16:50.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.340 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:50.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:50.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:50.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1900000000000002e-07 2024-09-14T06:16:50.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502501+0000", 2024-09-14T06:16:50.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8340999999999999e-05 2024-09-14T06:16:50.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502512+0000", 2024-09-14T06:16:50.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1347e-05 2024-09-14T06:16:50.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502518+0000", 2024-09-14T06:16:50.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.947e-06 2024-09-14T06:16:50.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502570+0000", 2024-09-14T06:16:50.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1858999999999999e-05 2024-09-14T06:16:50.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502633+0000", 2024-09-14T06:16:50.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2974999999999999e-05 2024-09-14T06:16:50.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:16:50.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502795+0000", 2024-09-14T06:16:50.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016262899999999999 2024-09-14T06:16:50.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.344 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:50.344 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505179+0000", 2024-09-14T06:16:50.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002383844 2024-09-14T06:16:50.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.344 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:50.344 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505676+0000", 2024-09-14T06:16:50.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00049634899999999999 2024-09-14T06:16:50.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.344 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.344 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505816+0000", 2024-09-14T06:16:50.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00014026300000000001 2024-09-14T06:16:50.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.345 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.345 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505926+0000", 2024-09-14T06:16:50.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000110319 2024-09-14T06:16:50.345 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.345 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.345 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.345 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3902 3.5 3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:50.345 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.569016+0000", 2024-09-14T06:16:50.345 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.758116253000001, 2024-09-14T06:16:50.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003356197, 2024-09-14T06:16:50.346 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.346 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.346 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.346 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.346 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.346 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.346 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3902 2024-09-14T06:16:50.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.346 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569016+0000", 2024-09-14T06:16:50.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569016+0000", 2024-09-14T06:16:50.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569019+0000", 2024-09-14T06:16:50.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2279999999999998e-06 2024-09-14T06:16:50.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569029+0000", 2024-09-14T06:16:50.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8079999999999999e-06 2024-09-14T06:16:50.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569059+0000", 2024-09-14T06:16:50.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0198000000000001e-05 2024-09-14T06:16:50.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569074+0000", 2024-09-14T06:16:50.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4968e-05 2024-09-14T06:16:50.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569165+0000", 2024-09-14T06:16:50.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0867999999999998e-05 2024-09-14T06:16:50.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569282+0000", 2024-09-14T06:16:50.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000117602 2024-09-14T06:16:50.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:50.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.569405+0000", 2024-09-14T06:16:50.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122471 2024-09-14T06:16:50.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:50.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.570585+0000", 2024-09-14T06:16:50.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001180459 2024-09-14T06:16:50.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:50.351 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.572315+0000", 2024-09-14T06:16:50.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0017295329999999999 2024-09-14T06:16:50.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.351 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.351 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.572345+0000", 2024-09-14T06:16:50.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0784999999999997e-05 2024-09-14T06:16:50.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.351 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.351 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.572372+0000", 2024-09-14T06:16:50.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6276999999999999e-05 2024-09-14T06:16:50.352 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.352 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.352 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.352 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3916 3.16 3:6ab8b1d8:::rbd_data.1152bd92e04f.0000000000000002:head [stat,truncate 1590169] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:50.352 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:50.352 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.747930693000001, 2024-09-14T06:16:50.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022958078999999999, 2024-09-14T06:16:50.352 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.352 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.352 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.352 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.353 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.353 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.353 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3916 2024-09-14T06:16:50.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.353 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:50.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:50.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579203+0000", 2024-09-14T06:16:50.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.748e-06 2024-09-14T06:16:50.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579208+0000", 2024-09-14T06:16:50.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:16:50.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579223+0000", 2024-09-14T06:16:50.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.508e-05 2024-09-14T06:16:50.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579232+0000", 2024-09-14T06:16:50.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9719999999999998e-06 2024-09-14T06:16:50.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.356 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579307+0000", 2024-09-14T06:16:50.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5028000000000001e-05 2024-09-14T06:16:50.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.356 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579412+0000", 2024-09-14T06:16:50.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010525400000000001 2024-09-14T06:16:50.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:16:50.356 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579535+0000", 2024-09-14T06:16:50.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122726 2024-09-14T06:16:50.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:50.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.581415+0000", 2024-09-14T06:16:50.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018800360000000001 2024-09-14T06:16:50.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:50.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602017+0000", 2024-09-14T06:16:50.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.020602425000000001 2024-09-14T06:16:50.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602096+0000", 2024-09-14T06:16:50.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8968999999999996e-05 2024-09-14T06:16:50.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602159+0000", 2024-09-14T06:16:50.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2824999999999996e-05 2024-09-14T06:16:50.358 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.358 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.358 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.358 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4873 3.e 3:76656e66:::rbd_data.1152bd92e04f.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=2b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:50.358 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:50.359 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.133484154000001, 2024-09-14T06:16:50.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048796120000000002, 2024-09-14T06:16:50.359 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.359 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.359 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.359 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.359 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.359 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.359 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4873 2024-09-14T06:16:50.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.359 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:50.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:50.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193651+0000", 2024-09-14T06:16:50.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:16:50.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193663+0000", 2024-09-14T06:16:50.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2047e-05 2024-09-14T06:16:50.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193692+0000", 2024-09-14T06:16:50.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9346e-05 2024-09-14T06:16:50.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193708+0000", 2024-09-14T06:16:50.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6123000000000001e-05 2024-09-14T06:16:50.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193798+0000", 2024-09-14T06:16:50.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9909999999999998e-05 2024-09-14T06:16:50.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193863+0000", 2024-09-14T06:16:50.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4583000000000002e-05 2024-09-14T06:16:50.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:50.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.194717+0000", 2024-09-14T06:16:50.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00085473299999999999 2024-09-14T06:16:50.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:50.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198400+0000", 2024-09-14T06:16:50.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036830890000000001 2024-09-14T06:16:50.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:50.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198461+0000", 2024-09-14T06:16:50.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0844000000000002e-05 2024-09-14T06:16:50.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198496+0000", 2024-09-14T06:16:50.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4820000000000002e-05 2024-09-14T06:16:50.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198527+0000", 2024-09-14T06:16:50.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1229999999999997e-05 2024-09-14T06:16:50.364 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.364 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.364 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.365 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:6925 3.5 3:a15e9af1:::rbd_data.1152f87cbac6.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:16:50.365 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:50.365 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.212352616, 2024-09-14T06:16:50.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0049096490000000003, 2024-09-14T06:16:50.365 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.365 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.365 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.365 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.365 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.365 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 6925 2024-09-14T06:16:50.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.365 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:50.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:50.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114780+0000", 2024-09-14T06:16:50.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.09e-07 2024-09-14T06:16:50.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114784+0000", 2024-09-14T06:16:50.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4669999999999998e-06 2024-09-14T06:16:50.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114799+0000", 2024-09-14T06:16:50.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4491000000000001e-05 2024-09-14T06:16:50.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114810+0000", 2024-09-14T06:16:50.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1243000000000001e-05 2024-09-14T06:16:50.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115515+0000", 2024-09-14T06:16:50.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00070521899999999998 2024-09-14T06:16:50.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115610+0000", 2024-09-14T06:16:50.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4260999999999998e-05 2024-09-14T06:16:50.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:50.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115742+0000", 2024-09-14T06:16:50.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000132414 2024-09-14T06:16:50.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:50.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.116650+0000", 2024-09-14T06:16:50.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00090814900000000002 2024-09-14T06:16:50.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:50.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119568+0000", 2024-09-14T06:16:50.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029178440000000002 2024-09-14T06:16:50.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.370 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.370 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119636+0000", 2024-09-14T06:16:50.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8365e-05 2024-09-14T06:16:50.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.370 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.370 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119689+0000", 2024-09-14T06:16:50.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2386999999999998e-05 2024-09-14T06:16:50.370 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.370 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.370 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.371 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:7151 3.e 3:7793b17d:::rbd_data.1152f87cbac6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:16:50.371 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:50.371 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 18.997534825999999, 2024-09-14T06:16:50.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.029556783, 2024-09-14T06:16:50.371 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.371 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.371 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.371 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.371 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.371 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.371 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 7151 2024-09-14T06:16:50.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.372 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:50.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:50.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329599+0000", 2024-09-14T06:16:50.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5270000000000001e-06 2024-09-14T06:16:50.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329610+0000", 2024-09-14T06:16:50.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0431000000000001e-05 2024-09-14T06:16:50.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329634+0000", 2024-09-14T06:16:50.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3617000000000001e-05 2024-09-14T06:16:50.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329644+0000", 2024-09-14T06:16:50.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.095e-05 2024-09-14T06:16:50.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329712+0000", 2024-09-14T06:16:50.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7067e-05 2024-09-14T06:16:50.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329827+0000", 2024-09-14T06:16:50.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011498 2024-09-14T06:16:50.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:50.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.334271+0000", 2024-09-14T06:16:50.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044445839999999997 2024-09-14T06:16:50.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:50.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.358967+0000", 2024-09-14T06:16:50.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024696356999999999 2024-09-14T06:16:50.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:50.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359027+0000", 2024-09-14T06:16:50.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0006000000000001e-05 2024-09-14T06:16:50.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359099+0000", 2024-09-14T06:16:50.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1052999999999999e-05 2024-09-14T06:16:50.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359154+0000", 2024-09-14T06:16:50.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5210999999999997e-05 2024-09-14T06:16:50.377 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.377 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.377 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.377 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:7151 3.e e32/29 ondisk, result = 0)", 2024-09-14T06:16:50.377 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:50.377 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 18.977590392, 2024-09-14T06:16:50.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0096251340000000005, 2024-09-14T06:16:50.378 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.378 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.378 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:50.378 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.378 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.378 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:50.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.378 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.378 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:50.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349543+0000", 2024-09-14T06:16:50.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:16:50.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349552+0000", 2024-09-14T06:16:50.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1039999999999996e-06 2024-09-14T06:16:50.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349579+0000", 2024-09-14T06:16:50.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6449999999999999e-05 2024-09-14T06:16:50.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349593+0000", 2024-09-14T06:16:50.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4099e-05 2024-09-14T06:16:50.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359003+0000", 2024-09-14T06:16:50.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0094096119999999995 2024-09-14T06:16:50.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359014+0000", 2024-09-14T06:16:50.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0871e-05 2024-09-14T06:16:50.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359167+0000", 2024-09-14T06:16:50.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015300700000000001 2024-09-14T06:16:50.381 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.381 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.381 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.381 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15516 3.1 e36/29 3:860a8b0d:::rbd_data.115223d7013d.00000000000000af:head v 36'26, mlcod=36'24)", 2024-09-14T06:16:50.382 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:50.382 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.069836042, 2024-09-14T06:16:50.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035459770000000001, 2024-09-14T06:16:50.382 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.382 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.382 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.382 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.382 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:50.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.382 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:50.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257297+0000", 2024-09-14T06:16:50.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4900000000000004e-07 2024-09-14T06:16:50.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257304+0000", 2024-09-14T06:16:50.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:16:50.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257321+0000", 2024-09-14T06:16:50.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6311e-05 2024-09-14T06:16:50.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257329+0000", 2024-09-14T06:16:50.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0399999999999993e-06 2024-09-14T06:16:50.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257414+0000", 2024-09-14T06:16:50.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4846999999999996e-05 2024-09-14T06:16:50.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257436+0000", 2024-09-14T06:16:50.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2048999999999998e-05 2024-09-14T06:16:50.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.260796+0000", 2024-09-14T06:16:50.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033607559999999999 2024-09-14T06:16:50.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.260842+0000", 2024-09-14T06:16:50.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5226999999999999e-05 2024-09-14T06:16:50.385 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.386 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.386 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.386 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15613 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'121, mlcod=36'116)", 2024-09-14T06:16:50.386 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:50.386 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.034227537, 2024-09-14T06:16:50.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053619389999999996, 2024-09-14T06:16:50.386 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.386 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.386 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.386 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.387 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:50.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.387 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:50.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.387 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292905+0000", 2024-09-14T06:16:50.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6800000000000009e-07 2024-09-14T06:16:50.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292915+0000", 2024-09-14T06:16:50.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:16:50.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292927+0000", 2024-09-14T06:16:50.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-09-14T06:16:50.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292935+0000", 2024-09-14T06:16:50.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2459999999999997e-06 2024-09-14T06:16:50.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292992+0000", 2024-09-14T06:16:50.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6873999999999998e-05 2024-09-14T06:16:50.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293016+0000", 2024-09-14T06:16:50.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4037e-05 2024-09-14T06:16:50.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298217+0000", 2024-09-14T06:16:50.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052008710000000001 2024-09-14T06:16:50.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298266+0000", 2024-09-14T06:16:50.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9691000000000002e-05 2024-09-14T06:16:50.391 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.391 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.391 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.391 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15615 3.11 e36/29 3:8d07d636:::rbd_data.115223d7013d.0000000000000002:head v 36'122, mlcod=36'116)", 2024-09-14T06:16:50.391 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:50.391 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.033331177999999, 2024-09-14T06:16:50.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053069229999999998, 2024-09-14T06:16:50.391 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.391 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.392 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.392 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:50.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:50.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:50.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5799999999999998e-07 2024-09-14T06:16:50.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294003+0000", 2024-09-14T06:16:50.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020149600000000001 2024-09-14T06:16:50.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294016+0000", 2024-09-14T06:16:50.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3424999999999999e-05 2024-09-14T06:16:50.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294024+0000", 2024-09-14T06:16:50.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5240000000000003e-06 2024-09-14T06:16:50.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294673+0000", 2024-09-14T06:16:50.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064906700000000003 2024-09-14T06:16:50.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294698+0000", 2024-09-14T06:16:50.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5301999999999999e-05 2024-09-14T06:16:50.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299064+0000", 2024-09-14T06:16:50.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043652170000000002 2024-09-14T06:16:50.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299108+0000", 2024-09-14T06:16:50.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4033999999999999e-05 2024-09-14T06:16:50.396 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.396 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.396 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.396 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15619 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'124, mlcod=36'119)", 2024-09-14T06:16:50.396 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:50.396 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.02927723, 2024-09-14T06:16:50.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061545389999999997, 2024-09-14T06:16:50.396 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.396 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.396 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.396 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:50.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:50.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:50.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3399999999999998e-07 2024-09-14T06:16:50.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297861+0000", 2024-09-14T06:16:50.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7640000000000002e-06 2024-09-14T06:16:50.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297876+0000", 2024-09-14T06:16:50.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4881e-05 2024-09-14T06:16:50.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297884+0000", 2024-09-14T06:16:50.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.588e-06 2024-09-14T06:16:50.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298294+0000", 2024-09-14T06:16:50.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00041011200000000002 2024-09-14T06:16:50.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298317+0000", 2024-09-14T06:16:50.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3417999999999998e-05 2024-09-14T06:16:50.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.400 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303972+0000", 2024-09-14T06:16:50.400 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056548680000000004 2024-09-14T06:16:50.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.400 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.400 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304009+0000", 2024-09-14T06:16:50.400 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7073999999999998e-05 2024-09-14T06:16:50.400 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.400 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15621 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'125, mlcod=36'119)", 2024-09-14T06:16:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.027766213, 2024-09-14T06:16:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0069496169999999999, 2024-09-14T06:16:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.402 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:50.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:50.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:50.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2200000000000003e-07 2024-09-14T06:16:50.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299372+0000", 2024-09-14T06:16:50.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3850000000000001e-06 2024-09-14T06:16:50.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299387+0000", 2024-09-14T06:16:50.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4905e-05 2024-09-14T06:16:50.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299393+0000", 2024-09-14T06:16:50.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9090000000000003e-06 2024-09-14T06:16:50.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303591+0000", 2024-09-14T06:16:50.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041976640000000003 2024-09-14T06:16:50.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303619+0000", 2024-09-14T06:16:50.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7651e-05 2024-09-14T06:16:50.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306287+0000", 2024-09-14T06:16:50.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026686689999999998 2024-09-14T06:16:50.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306315+0000", 2024-09-14T06:16:50.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7712000000000002e-05 2024-09-14T06:16:50.405 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.405 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.405 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.406 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15623 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'126, mlcod=36'119)", 2024-09-14T06:16:50.406 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:50.406 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.0244594, 2024-09-14T06:16:50.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003658859, 2024-09-14T06:16:50.406 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.406 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.406 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.406 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:50.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:50.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302673+0000", 2024-09-14T06:16:50.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4199999999999995e-07 2024-09-14T06:16:50.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302679+0000", 2024-09-14T06:16:50.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7599999999999999e-06 2024-09-14T06:16:50.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302689+0000", 2024-09-14T06:16:50.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0335e-05 2024-09-14T06:16:50.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302697+0000", 2024-09-14T06:16:50.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3420000000000002e-06 2024-09-14T06:16:50.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304047+0000", 2024-09-14T06:16:50.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0013503530000000001 2024-09-14T06:16:50.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304059+0000", 2024-09-14T06:16:50.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2285e-05 2024-09-14T06:16:50.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306318+0000", 2024-09-14T06:16:50.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022586580000000002 2024-09-14T06:16:50.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306331+0000", 2024-09-14T06:16:50.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3383999999999999e-05 2024-09-14T06:16:50.410 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.410 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.411 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.411 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15626 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'127, mlcod=36'120)", 2024-09-14T06:16:50.411 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:50.411 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.024095005, 2024-09-14T06:16:50.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038671249999999999, 2024-09-14T06:16:50.411 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.411 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.411 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.411 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:50.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:50.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:50.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0100000000000005e-07 2024-09-14T06:16:50.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.413 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.413 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303043+0000", 2024-09-14T06:16:50.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.293e-06 2024-09-14T06:16:50.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.413 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.413 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303072+0000", 2024-09-14T06:16:50.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9135000000000001e-05 2024-09-14T06:16:50.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.413 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.413 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303081+0000", 2024-09-14T06:16:50.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7080000000000003e-06 2024-09-14T06:16:50.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.305965+0000", 2024-09-14T06:16:50.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028841769999999999 2024-09-14T06:16:50.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.305993+0000", 2024-09-14T06:16:50.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8121000000000001e-05 2024-09-14T06:16:50.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306861+0000", 2024-09-14T06:16:50.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00086771099999999996 2024-09-14T06:16:50.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306904+0000", 2024-09-14T06:16:50.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3378999999999999e-05 2024-09-14T06:16:50.415 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.415 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.415 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.415 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:16521 3.e e36/34 3:77c8e3e5:::rbd_data.1152fc09e7e6.0000000000000003:head v 36'83, mlcod=36'81)", 2024-09-14T06:16:50.415 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:50.415 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.401112913, 2024-09-14T06:16:50.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044948899999999996, 2024-09-14T06:16:50.416 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.416 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.416 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.416 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:50.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:50.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926020+0000", 2024-09-14T06:16:50.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.545e-06 2024-09-14T06:16:50.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927192+0000", 2024-09-14T06:16:50.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011711919999999999 2024-09-14T06:16:50.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927215+0000", 2024-09-14T06:16:50.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3677e-05 2024-09-14T06:16:50.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927226+0000", 2024-09-14T06:16:50.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0665e-05 2024-09-14T06:16:50.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927314+0000", 2024-09-14T06:16:50.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8314999999999997e-05 2024-09-14T06:16:50.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927344+0000", 2024-09-14T06:16:50.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9941e-05 2024-09-14T06:16:50.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.930411+0000", 2024-09-14T06:16:50.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030670649999999999 2024-09-14T06:16:50.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.930514+0000", 2024-09-14T06:16:50.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010249 2024-09-14T06:16:50.420 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.420 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.420 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.420 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:18539 3.1e e40/29 3:7e28f783:::rbd_data.1152227be94f.0000000000000000:head v 40'59, mlcod=40'57)", 2024-09-14T06:16:50.421 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:16:50.421 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.0440908619999991, 2024-09-14T06:16:50.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0059097430000000003, 2024-09-14T06:16:50.421 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.421 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.421 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.421 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:16:50.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:16:50.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283043+0000", 2024-09-14T06:16:50.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4499999999999998e-06 2024-09-14T06:16:50.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285878+0000", 2024-09-14T06:16:50.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028348039999999998 2024-09-14T06:16:50.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285908+0000", 2024-09-14T06:16:50.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9756000000000001e-05 2024-09-14T06:16:50.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285922+0000", 2024-09-14T06:16:50.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.43e-05 2024-09-14T06:16:50.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.286030+0000", 2024-09-14T06:16:50.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000107954 2024-09-14T06:16:50.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.286058+0000", 2024-09-14T06:16:50.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7796e-05 2024-09-14T06:16:50.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.288888+0000", 2024-09-14T06:16:50.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028295799999999999 2024-09-14T06:16:50.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.425 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.288951+0000", 2024-09-14T06:16:50.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3102999999999995e-05 2024-09-14T06:16:50.425 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.425 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.425 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.426 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:20514 3.1e e42/29 3:7e502f18:::rbd_data.1152b8cc71.0000000000000003:head v 42'90, mlcod=42'88)", 2024-09-14T06:16:50.426 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:45.814156+0000", 2024-09-14T06:16:50.426 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.5129761139999998, 2024-09-14T06:16:50.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0042868200000000002, 2024-09-14T06:16:50.426 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.426 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.426 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.426 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.426 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.814156+0000", 2024-09-14T06:16:50.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.814156+0000", 2024-09-14T06:16:50.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.814158+0000", 2024-09-14T06:16:50.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6599999999999999e-06 2024-09-14T06:16:50.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.816022+0000", 2024-09-14T06:16:50.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018634179999999999 2024-09-14T06:16:50.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.816056+0000", 2024-09-14T06:16:50.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4445e-05 2024-09-14T06:16:50.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.816073+0000", 2024-09-14T06:16:50.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6759e-05 2024-09-14T06:16:50.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.816157+0000", 2024-09-14T06:16:50.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4109000000000004e-05 2024-09-14T06:16:50.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.816183+0000", 2024-09-14T06:16:50.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6296999999999999e-05 2024-09-14T06:16:50.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.818339+0000", 2024-09-14T06:16:50.430 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0021555879999999999 2024-09-14T06:16:50.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.430 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.430 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.430 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.818442+0000", 2024-09-14T06:16:50.430 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000103544 2024-09-14T06:16:50.430 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.430 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.430 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.430 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.430 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:23383 3.2 e45/29 3:416a9e38:::rbd_data.115256cac537.000000000000000e:head v 45'103, mlcod=45'101)", 2024-09-14T06:16:50.430 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:49.995203+0000", 2024-09-14T06:16:50.431 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.33192886300000002, 2024-09-14T06:16:50.431 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003562169, 2024-09-14T06:16:50.431 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.431 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.431 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:50.431 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.431 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.431 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.995203+0000", 2024-09-14T06:16:50.431 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.431 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.431 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.995203+0000", 2024-09-14T06:16:50.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.995204+0000", 2024-09-14T06:16:50.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5260000000000001e-06 2024-09-14T06:16:50.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.997471+0000", 2024-09-14T06:16:50.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022666959999999999 2024-09-14T06:16:50.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.997511+0000", 2024-09-14T06:16:50.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0138000000000001e-05 2024-09-14T06:16:50.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.997535+0000", 2024-09-14T06:16:50.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3485999999999998e-05 2024-09-14T06:16:50.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.997629+0000", 2024-09-14T06:16:50.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4246999999999994e-05 2024-09-14T06:16:50.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.434 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.997654+0000", 2024-09-14T06:16:50.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4545999999999998e-05 2024-09-14T06:16:50.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.434 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:50.434 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.998717+0000", 2024-09-14T06:16:50.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0010629669999999999 2024-09-14T06:16:50.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.434 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.434 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.998765+0000", 2024-09-14T06:16:50.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8563000000000002e-05 2024-09-14T06:16:50.434 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.434 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.435 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.435 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.435 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.435 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:50.436 INFO:teuthology.orchestra.run.smithi039.stdout:2129 writesame 0x1f599af thru 0x1f67167 data_size 0x1(0xd7b9 bytes) 2024-09-14T06:16:50.437 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:16:50.437 INFO:teuthology.orchestra.run.smithi039.stdout:2131 writesame 0x2890aaa thru 0x289ccbc data_size 0x1(0xc213 bytes) 2024-09-14T06:16:50.437 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:50.435+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:50.438 INFO:teuthology.orchestra.run.smithi039.stdout:2132 write 0x10bd159 thru 0x10c40b4 (0x6f5c bytes) 2024-09-14T06:16:50.463 INFO:teuthology.orchestra.run.smithi039.stdout:2133 writesame 0xc2f9bfd thru 0xc302ce0 data_size 0x16(0x90e4 bytes) 2024-09-14T06:16:50.463 INFO:teuthology.orchestra.run.smithi039.stdout:2134 writesame 0x9fc8887 thru 0x9fc8f6e data_size 0x11(0x6e8 bytes) 2024-09-14T06:16:50.470 INFO:teuthology.orchestra.run.smithi039.stdout:2135 punch from 0x28e483b to 0x28eb3ad, (0x6b72 bytes) 2024-09-14T06:16:50.470 INFO:teuthology.orchestra.run.smithi039.stdout:2136 writesame 0x149c8f5 thru 0x14a4bcf data_size 0x1(0x82db bytes) 2024-09-14T06:16:50.493 INFO:teuthology.orchestra.run.smithi039.stdout:2138 write 0x868be65 thru 0x868f790 (0x392c bytes) 2024-09-14T06:16:50.499 INFO:teuthology.orchestra.run.smithi039.stdout:2144 punch from 0x59fb1a5 to 0x5a053ca, (0xa225 bytes) 2024-09-14T06:16:50.500 INFO:teuthology.orchestra.run.smithi039.stdout:2145 write 0x37594b3 thru 0x3764ff8 (0xbb46 bytes) 2024-09-14T06:16:50.507 INFO:teuthology.orchestra.run.smithi039.stdout:2146 write 0xaa7bf84 thru 0xaa7c635 (0x6b2 bytes) 2024-09-14T06:16:50.513 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:50.513 INFO:teuthology.orchestra.run.smithi039.stdout:2148 write 0x451b86a thru 0x4527e39 (0xc5d0 bytes) 2024-09-14T06:16:50.522 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-09-14T06:16:50.538 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:50.535+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:50.539 INFO:teuthology.orchestra.run.smithi039.stdout:2149 trunc from 0xc302ce1 to 0x3d32d73 2024-09-14T06:16:50.575 INFO:teuthology.orchestra.run.smithi039.stdout:2150 read 0x3bac817 thru 0x3bac9e7 (0x1d1 bytes) 2024-09-14T06:16:50.575 INFO:teuthology.orchestra.run.smithi039.stdout:2151 read 0xebe35f thru 0xecbf51 (0xdbf3 bytes) 2024-09-14T06:16:50.576 INFO:teuthology.orchestra.run.smithi039.stdout:2153 read 0xe423cb thru 0xe4a491 (0x80c7 bytes) 2024-09-14T06:16:50.580 INFO:teuthology.orchestra.run.smithi039.stdout:2154 punch from 0x3377b9f to 0x3387137, (0xf598 bytes) 2024-09-14T06:16:50.606 INFO:teuthology.orchestra.run.smithi039.stdout:2155 write 0xbc56572 thru 0xbc5e9f7 (0x8486 bytes) 2024-09-14T06:16:50.611 INFO:teuthology.orchestra.run.smithi039.stdout:2156 punch from 0x1134eb9 to 0x1142f87, (0xe0ce bytes) 2024-09-14T06:16:50.613 INFO:teuthology.orchestra.run.smithi039.stdout:2157 trunc from 0xbc5e9f8 to 0xd218b68 2024-09-14T06:16:50.614 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:50.615 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:50.615 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:50.615 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:50.631 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:16:50.633 INFO:teuthology.orchestra.run.smithi039.stdout:2160 trunc from 0xd218b68 to 0x84ff731 2024-09-14T06:16:50.638 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:50.639+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:50.655 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:50.655 INFO:teuthology.orchestra.run.smithi039.stdout:2163 punch from 0x59caa05 to 0x59d39b0, (0x8fab bytes) 2024-09-14T06:16:50.655 INFO:teuthology.orchestra.run.smithi039.stdout:2164 read 0x849ee09 thru 0x84a5a8a (0x6c82 bytes) 2024-09-14T06:16:50.655 INFO:teuthology.orchestra.run.smithi039.stdout:2165 read 0x1feda59 thru 0x1ff0c67 (0x320f bytes) 2024-09-14T06:16:50.655 INFO:teuthology.orchestra.run.smithi039.stdout:2166 flatten 2024-09-14T06:16:50.677 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:16:50.677 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:16:50.677 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-14T06:16:50.677 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-14T06:16:50.710 INFO:teuthology.orchestra.run.smithi039.stdout:2167 write 0xa6e3682 thru 0xa6efe89 (0xc808 bytes) 2024-09-14T06:16:50.715 INFO:teuthology.orchestra.run.smithi039.stdout:2168 write 0x1d7bdd9 thru 0x1d843fc (0x8624 bytes) 2024-09-14T06:16:50.719 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:50.719 INFO:teuthology.orchestra.run.smithi039.stdout:2170 writesame 0x3672a9e thru 0x367ce0e data_size 0x3(0xa371 bytes) 2024-09-14T06:16:50.739 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:50.739+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:50.740 INFO:teuthology.orchestra.run.smithi039.stdout:2171 clone 13 order 25 su 262144 sc 3 2024-09-14T06:16:50.761 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:50.761 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:50.761 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:50.762 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:50.762 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:50.776 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:16:50.848 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-14T06:16:50.849 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:50.847+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:50.938 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:50.938 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:50.939 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:50.939 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:50.939 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.939 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11217 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 925696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:50.939 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:50.939 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.945927577999999, 2024-09-14T06:16:50.939 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049306313999999997, 2024-09-14T06:16:50.939 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.939 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.939 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:50.939 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.939 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.939 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.939 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11217 2024-09-14T06:16:50.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.940 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.940 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.940 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.940 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:50.940 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.940 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.940 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.940 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:50.940 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.940 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.940 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.941 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:50.941 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.29e-07 2024-09-14T06:16:50.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.941 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.941 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.941 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991345+0000", 2024-09-14T06:16:50.941 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1480000000000001e-06 2024-09-14T06:16:50.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.941 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.941 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.942 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991349+0000", 2024-09-14T06:16:50.942 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9840000000000003e-06 2024-09-14T06:16:50.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.942 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.942 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.942 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991359+0000", 2024-09-14T06:16:50.942 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4050000000000006e-06 2024-09-14T06:16:50.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.942 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.942 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.942 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.030826+0000", 2024-09-14T06:16:50.942 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.039467214 2024-09-14T06:16:50.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.942 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.943 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.943 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.030894+0000", 2024-09-14T06:16:50.943 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8400999999999998e-05 2024-09-14T06:16:50.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.943 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.943 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.943 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040650+0000", 2024-09-14T06:16:50.943 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0097559329999999996 2024-09-14T06:16:50.943 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.943 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.943 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.943 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.943 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11262 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 81920~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:50.944 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:50.944 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.945741542, 2024-09-14T06:16:50.944 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049588879000000002, 2024-09-14T06:16:50.944 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.944 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.944 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:50.944 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.944 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.944 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.944 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11262 2024-09-14T06:16:50.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.944 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.944 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.944 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.944 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:50.945 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.945 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.945 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.945 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:50.945 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.945 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.945 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.945 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:50.945 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9599999999999999e-07 2024-09-14T06:16:50.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.945 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.945 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.946 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991533+0000", 2024-09-14T06:16:50.946 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.362e-06 2024-09-14T06:16:50.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.946 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.946 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.946 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991539+0000", 2024-09-14T06:16:50.946 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:16:50.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.946 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.946 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.946 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991544+0000", 2024-09-14T06:16:50.946 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:16:50.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.946 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.947 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.947 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040755+0000", 2024-09-14T06:16:50.947 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049210938000000003 2024-09-14T06:16:50.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.947 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.947 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.947 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040833+0000", 2024-09-14T06:16:50.947 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7812999999999995e-05 2024-09-14T06:16:50.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.947 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.947 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.947 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041119+0000", 2024-09-14T06:16:50.947 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0002855 2024-09-14T06:16:50.947 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.947 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.948 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.948 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.948 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11263 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:50.948 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:50.948 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.945718764, 2024-09-14T06:16:50.948 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049944574999999998, 2024-09-14T06:16:50.948 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.948 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.948 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:50.949 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.949 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.949 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.949 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11263 2024-09-14T06:16:50.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.949 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.949 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.949 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:50.949 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.949 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.949 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:50.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.950 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.950 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.950 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:50.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.51e-07 2024-09-14T06:16:50.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.950 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.950 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.950 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991560+0000", 2024-09-14T06:16:50.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6719999999999998e-06 2024-09-14T06:16:50.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.950 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.950 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.950 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991567+0000", 2024-09-14T06:16:50.951 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1470000000000004e-06 2024-09-14T06:16:50.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.951 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.951 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.951 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991572+0000", 2024-09-14T06:16:50.951 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.237e-06 2024-09-14T06:16:50.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.951 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.951 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.951 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041149+0000", 2024-09-14T06:16:50.951 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049576988000000002 2024-09-14T06:16:50.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.951 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.951 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.951 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041277+0000", 2024-09-14T06:16:50.952 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000127687 2024-09-14T06:16:50.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.952 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.952 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.952 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041497+0000", 2024-09-14T06:16:50.952 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022039299999999999 2024-09-14T06:16:50.952 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.952 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.952 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.952 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.952 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11264 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 327680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:50.952 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:50.952 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.945692423000001, 2024-09-14T06:16:50.953 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050248567000000001, 2024-09-14T06:16:50.953 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.953 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.953 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:50.953 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.953 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.953 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.953 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11264 2024-09-14T06:16:50.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.953 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.953 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.953 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.953 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:50.953 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.954 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.954 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.954 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:50.954 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.954 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.954 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.954 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:50.954 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.41e-07 2024-09-14T06:16:50.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.954 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.954 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.954 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991581+0000", 2024-09-14T06:16:50.954 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3740000000000001e-06 2024-09-14T06:16:50.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.955 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.955 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991585+0000", 2024-09-14T06:16:50.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9400000000000004e-06 2024-09-14T06:16:50.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.955 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.955 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991587+0000", 2024-09-14T06:16:50.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.694e-06 2024-09-14T06:16:50.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.955 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.955 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041527+0000", 2024-09-14T06:16:50.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049939349000000001 2024-09-14T06:16:50.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.956 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.956 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.956 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041598+0000", 2024-09-14T06:16:50.956 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1633999999999999e-05 2024-09-14T06:16:50.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.956 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.956 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.956 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041827+0000", 2024-09-14T06:16:50.956 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000229235 2024-09-14T06:16:50.956 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.956 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.956 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.957 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:50.957 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:50.957 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.945679083, 2024-09-14T06:16:50.957 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:16:50.957 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.957 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.957 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:50.957 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.957 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.957 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.957 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:16:50.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.957 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.958 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.958 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:50.958 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.958 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.958 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:50.958 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.958 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.958 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:50.958 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:16:50.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.959 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.959 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.959 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:16:50.959 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:16:50.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.959 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.959 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.959 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:16:50.959 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:16:50.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.960 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.960 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:16:50.960 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:16:50.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.960 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.960 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:16:50.960 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:16:50.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.960 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.960 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:16:50.960 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:16:50.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.961 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.961 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.961 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:16:50.961 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:16:50.961 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.961 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.961 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.961 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.961 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:50.961 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:50.961 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.945666011, 2024-09-14T06:16:50.961 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:16:50.961 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.962 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.962 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:50.962 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.962 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.962 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.962 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:16:50.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.962 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.962 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.962 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:50.962 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.962 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.963 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:50.963 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.963 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.963 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:16:50.963 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:16:50.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.963 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.963 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:16:50.963 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:16:50.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.963 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.964 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:16:50.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:16:50.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.964 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.964 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:16:50.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:16:50.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.964 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.964 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:16:50.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:16:50.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.964 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.965 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:16:50.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:16:50.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.965 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.965 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:16:50.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:16:50.965 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.965 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.965 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.965 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:50.965 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:50.966 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.945648579, 2024-09-14T06:16:50.966 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:16:50.966 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.966 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.966 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:50.966 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.966 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.966 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.966 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:16:50.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.966 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:16:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.968 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.968 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:16:50.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:16:50.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.968 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.968 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:16:50.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:16:50.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.968 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.968 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:16:50.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:16:50.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:16:50.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:16:50.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:16:50.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:16:50.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.970 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.970 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:16:50.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:16:50.970 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.970 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.970 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.970 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:50.970 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:50.970 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.945627474, 2024-09-14T06:16:50.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:16:50.971 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.971 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.971 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:50.971 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.971 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.971 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.971 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:16:50.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.971 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.971 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.971 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:50.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:16:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:16:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:16:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:16:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:16:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:16:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:16:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:16:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:16:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:16:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:16:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:16:50.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:16:50.975 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.975 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.975 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.975 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:50.975 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:50.975 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.945606366, 2024-09-14T06:16:50.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:16:50.975 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.975 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.975 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:50.975 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.975 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.976 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.976 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:16:50.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.976 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:50.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:50.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.977 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.977 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.977 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:50.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:16:50.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.977 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.977 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.977 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:16:50.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:16:50.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:16:50.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:16:50.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:16:50.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:16:50.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.979 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.979 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:16:50.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:16:50.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.979 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.979 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:16:50.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:16:50.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.979 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.979 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:16:50.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:16:50.980 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.980 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.980 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.980 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13590 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 491520~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:50.980 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:50.980 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.799631484000001, 2024-09-14T06:16:50.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042553466999999998, 2024-09-14T06:16:50.980 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.981 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.981 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:50.981 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.981 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.981 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.981 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13590 2024-09-14T06:16:50.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.981 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:50.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:50.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:50.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.34e-07 2024-09-14T06:16:50.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137642+0000", 2024-09-14T06:16:50.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0150000000000002e-06 2024-09-14T06:16:50.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.983 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.983 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137649+0000", 2024-09-14T06:16:50.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4010000000000002e-06 2024-09-14T06:16:50.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.983 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.983 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137653+0000", 2024-09-14T06:16:50.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:16:50.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.983 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.983 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.179985+0000", 2024-09-14T06:16:50.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042331580000000001 2024-09-14T06:16:50.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.984 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180076+0000", 2024-09-14T06:16:50.984 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0796000000000003e-05 2024-09-14T06:16:50.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.984 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180193+0000", 2024-09-14T06:16:50.984 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000117471 2024-09-14T06:16:50.984 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.984 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.984 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.985 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13592 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 606208~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:50.985 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:50.985 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.799606916, 2024-09-14T06:16:50.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042709440000000001, 2024-09-14T06:16:50.985 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.985 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:50.985 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.985 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.985 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.985 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13592 2024-09-14T06:16:50.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.985 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:50.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:50.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137665+0000", 2024-09-14T06:16:50.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4299999999999998e-07 2024-09-14T06:16:50.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137667+0000", 2024-09-14T06:16:50.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5780000000000001e-06 2024-09-14T06:16:50.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137675+0000", 2024-09-14T06:16:50.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0870000000000001e-06 2024-09-14T06:16:50.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137680+0000", 2024-09-14T06:16:50.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4999999999999999e-06 2024-09-14T06:16:50.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180223+0000", 2024-09-14T06:16:50.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042542808000000001 2024-09-14T06:16:50.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180294+0000", 2024-09-14T06:16:50.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1497999999999999e-05 2024-09-14T06:16:50.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180374+0000", 2024-09-14T06:16:50.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9525999999999997e-05 2024-09-14T06:16:50.989 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.989 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.989 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.989 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13594 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 729088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:50.989 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:50.989 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.799583351999999, 2024-09-14T06:16:50.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042830604000000001, 2024-09-14T06:16:50.989 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.990 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.990 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:50.990 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.990 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.990 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.990 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13594 2024-09-14T06:16:50.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.990 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:50.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:50.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:50.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0699999999999998e-07 2024-09-14T06:16:50.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137690+0000", 2024-09-14T06:16:50.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.012e-06 2024-09-14T06:16:50.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137698+0000", 2024-09-14T06:16:50.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4379999999999998e-06 2024-09-14T06:16:50.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137703+0000", 2024-09-14T06:16:50.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1250000000000001e-06 2024-09-14T06:16:50.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180404+0000", 2024-09-14T06:16:50.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042701465000000001 2024-09-14T06:16:50.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180456+0000", 2024-09-14T06:16:50.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1227999999999998e-05 2024-09-14T06:16:50.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180519+0000", 2024-09-14T06:16:50.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-09-14T06:16:50.994 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.994 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.994 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.994 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13595 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 851968~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:50.994 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:50.994 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.799560847, 2024-09-14T06:16:50.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043012433000000003, 2024-09-14T06:16:50.994 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:50.994 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:50.994 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:50.995 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:50.995 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:50.995 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:50.995 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13595 2024-09-14T06:16:50.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.995 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:50.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:50.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:50.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:50.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:50.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:50.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:50.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137711+0000", 2024-09-14T06:16:50.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:16:50.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.997 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:50.997 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137713+0000", 2024-09-14T06:16:50.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5170000000000002e-06 2024-09-14T06:16:50.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.997 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:50.997 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137720+0000", 2024-09-14T06:16:50.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-09-14T06:16:50.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:50.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137726+0000", 2024-09-14T06:16:50.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9580000000000004e-06 2024-09-14T06:16:50.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:50.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180551+0000", 2024-09-14T06:16:50.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042825611999999999 2024-09-14T06:16:50.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:50.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180623+0000", 2024-09-14T06:16:50.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1673000000000003e-05 2024-09-14T06:16:50.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:50.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:50.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180723+0000", 2024-09-14T06:16:50.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.9928999999999998e-05 2024-09-14T06:16:50.999 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.999 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:50.999 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:50.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:50.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.000 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13597 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 974848~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.000 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:51.000 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.799537962, 2024-09-14T06:16:51.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043180442999999999, 2024-09-14T06:16:51.000 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:51.000 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:51.000 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:51.000 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:51.000 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:51.000 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.000 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13597 2024-09-14T06:16:51.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.001 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:51.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.001 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:51.001 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:51.001 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:51.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.001 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:51.001 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:51.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:51.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:51.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137734+0000", 2024-09-14T06:16:51.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4499999999999997e-07 2024-09-14T06:16:51.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:51.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137736+0000", 2024-09-14T06:16:51.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0470000000000001e-06 2024-09-14T06:16:51.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:51.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137742+0000", 2024-09-14T06:16:51.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2269999999999998e-06 2024-09-14T06:16:51.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137747+0000", 2024-09-14T06:16:51.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6670000000000004e-06 2024-09-14T06:16:51.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:51.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180753+0000", 2024-09-14T06:16:51.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043006132000000002 2024-09-14T06:16:51.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:51.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180820+0000", 2024-09-14T06:16:51.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7516999999999997e-05 2024-09-14T06:16:51.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:51.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180914+0000", 2024-09-14T06:16:51.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3407999999999997e-05 2024-09-14T06:16:51.004 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:51.004 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:51.005 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:51.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.005 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13684 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 999424~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.005 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:51.005 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.797305419000001, 2024-09-14T06:16:51.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042643702999999998, 2024-09-14T06:16:51.005 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:51.005 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:51.005 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:51.005 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:51.005 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:51.006 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.006 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13684 2024-09-14T06:16:51.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.006 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:51.006 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.006 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:51.006 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:51.006 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:51.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.006 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.006 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:51.006 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:51.006 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:51.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:51.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139967+0000", 2024-09-14T06:16:51.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1200000000000003e-07 2024-09-14T06:16:51.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:51.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141080+0000", 2024-09-14T06:16:51.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011133029999999999 2024-09-14T06:16:51.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:51.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141084+0000", 2024-09-14T06:16:51.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.544e-06 2024-09-14T06:16:51.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141088+0000", 2024-09-14T06:16:51.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0280000000000002e-06 2024-09-14T06:16:51.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:51.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182457+0000", 2024-09-14T06:16:51.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041368690999999999 2024-09-14T06:16:51.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:51.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182528+0000", 2024-09-14T06:16:51.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0881000000000001e-05 2024-09-14T06:16:51.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.009 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:51.009 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182610+0000", 2024-09-14T06:16:51.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1644000000000006e-05 2024-09-14T06:16:51.009 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:51.009 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:51.010 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:51.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.010 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13721 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 532480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.010 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:51.010 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.796004685, 2024-09-14T06:16:51.010 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042577526999999997, 2024-09-14T06:16:51.011 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:51.011 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:51.011 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:51.011 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:51.011 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:51.011 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.011 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13721 2024-09-14T06:16:51.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.011 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:51.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.011 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:51.011 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:51.011 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:51.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.012 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:51.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:51.012 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:51.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.012 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:51.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:51.012 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2999999999999999e-07 2024-09-14T06:16:51.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.013 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:51.013 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141268+0000", 2024-09-14T06:16:51.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0130000000000001e-06 2024-09-14T06:16:51.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.014 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:51.014 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141271+0000", 2024-09-14T06:16:51.014 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0970000000000002e-06 2024-09-14T06:16:51.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.014 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.014 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.015 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141274+0000", 2024-09-14T06:16:51.015 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5809999999999999e-06 2024-09-14T06:16:51.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.015 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.015 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:51.016 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183522+0000", 2024-09-14T06:16:51.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042248591000000002 2024-09-14T06:16:51.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.016 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.016 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:51.016 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183691+0000", 2024-09-14T06:16:51.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016885399999999999 2024-09-14T06:16:51.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.016 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.016 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:51.016 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183844+0000", 2024-09-14T06:16:51.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015316099999999999 2024-09-14T06:16:51.016 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:51.017 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:51.017 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:51.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.017 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13722 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 655360~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.017 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:51.017 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.795994383, 2024-09-14T06:16:51.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042741610999999999, 2024-09-14T06:16:51.017 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:51.017 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:51.017 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:51.017 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:51.017 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:51.017 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.018 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13722 2024-09-14T06:16:51.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.018 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:51.018 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.018 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:51.018 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:51.018 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:51.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.018 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.018 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:51.018 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:51.018 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:51.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.018 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.018 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:51.019 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:51.019 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:16:51.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.019 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.019 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:51.019 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141278+0000", 2024-09-14T06:16:51.019 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.066e-06 2024-09-14T06:16:51.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.019 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.019 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:51.019 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141282+0000", 2024-09-14T06:16:51.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7100000000000001e-06 2024-09-14T06:16:51.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.020 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141285+0000", 2024-09-14T06:16:51.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.554e-06 2024-09-14T06:16:51.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:51.020 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183824+0000", 2024-09-14T06:16:51.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042539043999999998 2024-09-14T06:16:51.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:51.021 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183906+0000", 2024-09-14T06:16:51.021 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2298000000000004e-05 2024-09-14T06:16:51.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.021 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.021 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:51.021 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184019+0000", 2024-09-14T06:16:51.021 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000112724 2024-09-14T06:16:51.021 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:51.021 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:51.022 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:51.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.022 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13723 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 778240~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.022 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:51.022 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.795983414, 2024-09-14T06:16:51.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042995901000000003, 2024-09-14T06:16:51.022 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:51.022 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:51.022 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:51.022 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:51.022 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:51.022 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.023 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13723 2024-09-14T06:16:51.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.023 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:51.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.023 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:51.023 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:51.023 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:51.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.023 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:51.023 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:51.023 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:51.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.023 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:51.023 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:51.024 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9799999999999999e-07 2024-09-14T06:16:51.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.024 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.024 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:51.024 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141289+0000", 2024-09-14T06:16:51.024 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0589999999999999e-06 2024-09-14T06:16:51.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.024 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.024 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:51.024 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141292+0000", 2024-09-14T06:16:51.024 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.208e-06 2024-09-14T06:16:51.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.024 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.024 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141295+0000", 2024-09-14T06:16:51.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5160000000000001e-06 2024-09-14T06:16:51.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:51.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184069+0000", 2024-09-14T06:16:51.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042774176999999997 2024-09-14T06:16:51.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:51.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184169+0000", 2024-09-14T06:16:51.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000100073 2024-09-14T06:16:51.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:51.026 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184284+0000", 2024-09-14T06:16:51.026 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011457 2024-09-14T06:16:51.026 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:51.026 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:51.026 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:51.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.027 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.027 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13724 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 901120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.027 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:16:51.027 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.79597302, 2024-09-14T06:16:51.027 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043197179000000002, 2024-09-14T06:16:51.028 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:51.028 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:51.028 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:51.028 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:51.028 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:51.029 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.029 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13724 2024-09-14T06:16:51.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.029 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:51.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.029 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:51.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:16:51.030 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:51.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.030 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.030 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:51.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:16:51.030 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:51.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.031 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.031 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:51.031 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141299+0000", 2024-09-14T06:16:51.031 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.03e-07 2024-09-14T06:16:51.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.031 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.032 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:51.032 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141300+0000", 2024-09-14T06:16:51.032 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0389999999999999e-06 2024-09-14T06:16:51.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.032 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.032 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:51.033 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141303+0000", 2024-09-14T06:16:51.033 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0479999999999999e-06 2024-09-14T06:16:51.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.033 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.033 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.033 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141305+0000", 2024-09-14T06:16:51.034 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5739999999999998e-06 2024-09-14T06:16:51.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.034 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:51.034 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184317+0000", 2024-09-14T06:16:51.034 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043011701999999999 2024-09-14T06:16:51.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.035 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:51.035 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184394+0000", 2024-09-14T06:16:51.035 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7324999999999996e-05 2024-09-14T06:16:51.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.035 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.035 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:51.035 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184496+0000", 2024-09-14T06:16:51.036 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010118799999999999 2024-09-14T06:16:51.036 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:51.036 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:51.036 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:51.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.036 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.037 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13726 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 1024000~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.037 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:51.037 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.795962747000001, 2024-09-14T06:16:51.037 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043397185999999997, 2024-09-14T06:16:51.037 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:51.037 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:51.038 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:51.038 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:51.038 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:51.038 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.038 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13726 2024-09-14T06:16:51.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.039 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:51.039 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.039 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:51.039 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:51.039 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:51.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.040 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.040 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:51.040 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:51.040 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:51.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.041 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.041 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:51.041 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:51.041 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:16:51.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.042 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.042 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:51.042 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141310+0000", 2024-09-14T06:16:51.042 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:16:51.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.042 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.042 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:51.042 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141313+0000", 2024-09-14T06:16:51.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0309999999999999e-06 2024-09-14T06:16:51.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.043 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.043 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141315+0000", 2024-09-14T06:16:51.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.52e-06 2024-09-14T06:16:51.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.043 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:51.044 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184527+0000", 2024-09-14T06:16:51.044 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043211855 2024-09-14T06:16:51.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.044 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.044 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:51.044 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184606+0000", 2024-09-14T06:16:51.044 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8287000000000004e-05 2024-09-14T06:16:51.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:51.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:51.045 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:51.045 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184706+0000", 2024-09-14T06:16:51.045 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010021700000000001 2024-09-14T06:16:51.045 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:51.045 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:51.046 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:51.046 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:51.046 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:51.046 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:51.048 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:16:51.049 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:51.047+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:51.062 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-14T06:16:51.155 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:51.155+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:51.181 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:51.181 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:51.181 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:16:51.181 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:51.195 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:16:51.249 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:51.247+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:51.274 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-14T06:16:51.312 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:51.312 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:51.312 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:16:51.313 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:16:51.313 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:51.326 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:16:51.363 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:51.363+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:51.452 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:51.452 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:16:51.452 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:16:51.452 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:16:51.452 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.452 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13229 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.452 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:16:51.452 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.350375551000001, 2024-09-14T06:16:51.452 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081798868999999996, 2024-09-14T06:16:51.452 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.453 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.453 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.453 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.453 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.453 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.453 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13229 2024-09-14T06:16:51.453 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.453 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.453 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.453 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.453 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:16:51.453 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.453 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.454 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.454 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.454 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:16:51.454 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.454 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.454 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.454 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.454 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098710+0000", 2024-09-14T06:16:51.454 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8199999999999999e-07 2024-09-14T06:16:51.454 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.454 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.454 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.454 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098714+0000", 2024-09-14T06:16:51.454 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:16:51.454 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.455 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.455 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.455 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098727+0000", 2024-09-14T06:16:51.455 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2520999999999999e-05 2024-09-14T06:16:51.455 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.455 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.455 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.455 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098749+0000", 2024-09-14T06:16:51.455 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2773999999999999e-05 2024-09-14T06:16:51.455 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.455 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.455 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.455 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160234+0000", 2024-09-14T06:16:51.456 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.061484430999999999 2024-09-14T06:16:51.456 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.456 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.456 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.456 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160281+0000", 2024-09-14T06:16:51.456 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7437e-05 2024-09-14T06:16:51.456 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.456 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.456 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.457 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160569+0000", 2024-09-14T06:16:51.457 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028788700000000002 2024-09-14T06:16:51.457 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.457 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.457 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.457 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163121+0000", 2024-09-14T06:16:51.457 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025514090000000001 2024-09-14T06:16:51.457 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.457 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.457 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.457 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180410+0000", 2024-09-14T06:16:51.457 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.017289098999999999 2024-09-14T06:16:51.457 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.457 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.458 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180472+0000", 2024-09-14T06:16:51.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2547999999999998e-05 2024-09-14T06:16:51.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.458 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.458 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180508+0000", 2024-09-14T06:16:51.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5534000000000002e-05 2024-09-14T06:16:51.458 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.458 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.458 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.458 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.458 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13230 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.459 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:16:51.459 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.350294349, 2024-09-14T06:16:51.459 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081851558000000005, 2024-09-14T06:16:51.459 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.459 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.459 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.459 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.459 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.459 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.459 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13230 2024-09-14T06:16:51.459 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.459 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.460 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.460 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.460 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:16:51.460 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.460 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.460 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.460 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.460 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:16:51.460 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.460 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.460 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.460 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.460 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098791+0000", 2024-09-14T06:16:51.460 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8800000000000001e-07 2024-09-14T06:16:51.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098795+0000", 2024-09-14T06:16:51.461 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.391e-06 2024-09-14T06:16:51.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098804+0000", 2024-09-14T06:16:51.461 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3440000000000001e-06 2024-09-14T06:16:51.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098810+0000", 2024-09-14T06:16:51.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2339999999999999e-06 2024-09-14T06:16:51.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.462 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.462 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160930+0000", 2024-09-14T06:16:51.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062120018999999999 2024-09-14T06:16:51.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.462 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.462 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161017+0000", 2024-09-14T06:16:51.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6868999999999998e-05 2024-09-14T06:16:51.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.462 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.462 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161300+0000", 2024-09-14T06:16:51.463 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028349499999999998 2024-09-14T06:16:51.463 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.463 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.463 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.463 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164612+0000", 2024-09-14T06:16:51.463 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033119680000000002 2024-09-14T06:16:51.463 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.463 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.463 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.463 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180559+0000", 2024-09-14T06:16:51.463 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015946612999999998 2024-09-14T06:16:51.463 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.463 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.464 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.464 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180605+0000", 2024-09-14T06:16:51.464 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5890000000000003e-05 2024-09-14T06:16:51.464 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.464 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.464 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.464 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180642+0000", 2024-09-14T06:16:51.464 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:16:51.464 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.464 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.464 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.464 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.464 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.464 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13231 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.465 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:16:51.465 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.350031935000001, 2024-09-14T06:16:51.465 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081750685000000003, 2024-09-14T06:16:51.465 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.465 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.465 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.465 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.465 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.465 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.465 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13231 2024-09-14T06:16:51.465 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.465 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.465 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.465 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.466 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:16:51.466 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.466 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.466 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.466 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.466 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:16:51.466 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.466 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.466 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.467 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.467 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099053+0000", 2024-09-14T06:16:51.467 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:16:51.467 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.467 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.467 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.467 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099059+0000", 2024-09-14T06:16:51.467 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1789999999999999e-06 2024-09-14T06:16:51.467 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.467 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.468 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.468 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099072+0000", 2024-09-14T06:16:51.468 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3179999999999999e-05 2024-09-14T06:16:51.468 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.468 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.468 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.468 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099080+0000", 2024-09-14T06:16:51.468 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9030000000000004e-06 2024-09-14T06:16:51.468 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.468 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.469 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.469 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161681+0000", 2024-09-14T06:16:51.469 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062601641999999999 2024-09-14T06:16:51.469 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.469 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.469 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.469 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161766+0000", 2024-09-14T06:16:51.469 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4909999999999998e-05 2024-09-14T06:16:51.469 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.469 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.469 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.470 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162109+0000", 2024-09-14T06:16:51.470 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00034264899999999999 2024-09-14T06:16:51.470 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.470 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.470 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.470 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164615+0000", 2024-09-14T06:16:51.470 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025061979999999998 2024-09-14T06:16:51.470 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.470 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.470 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.470 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180718+0000", 2024-09-14T06:16:51.470 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016103240000000001 2024-09-14T06:16:51.471 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.471 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.471 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.471 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180768+0000", 2024-09-14T06:16:51.471 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9781999999999999e-05 2024-09-14T06:16:51.471 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.471 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.471 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.471 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180803+0000", 2024-09-14T06:16:51.471 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5021999999999998e-05 2024-09-14T06:16:51.471 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.471 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.471 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.472 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.472 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13232 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.472 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:16:51.472 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.349990161000001, 2024-09-14T06:16:51.472 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081861981, 2024-09-14T06:16:51.472 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.472 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.472 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.472 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.472 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.472 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.472 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13232 2024-09-14T06:16:51.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.473 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.473 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.473 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.473 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:16:51.473 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.473 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.473 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.473 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.473 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:16:51.473 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.473 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.473 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.473 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.473 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099095+0000", 2024-09-14T06:16:51.474 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:16:51.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.474 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.474 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.474 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099098+0000", 2024-09-14T06:16:51.474 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5420000000000001e-06 2024-09-14T06:16:51.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.474 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.474 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.474 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099106+0000", 2024-09-14T06:16:51.474 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.824e-06 2024-09-14T06:16:51.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.474 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.474 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.475 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099112+0000", 2024-09-14T06:16:51.475 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9789999999999998e-06 2024-09-14T06:16:51.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.475 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.475 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.475 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162576+0000", 2024-09-14T06:16:51.475 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.063463723999999999 2024-09-14T06:16:51.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.475 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.475 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.475 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162627+0000", 2024-09-14T06:16:51.475 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1530000000000003e-05 2024-09-14T06:16:51.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.475 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.476 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.476 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162854+0000", 2024-09-14T06:16:51.476 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000226159 2024-09-14T06:16:51.476 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.476 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.476 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.476 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166090+0000", 2024-09-14T06:16:51.476 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032362319999999999 2024-09-14T06:16:51.476 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.476 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.476 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.476 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180877+0000", 2024-09-14T06:16:51.476 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014786726 2024-09-14T06:16:51.476 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.476 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.477 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.477 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180920+0000", 2024-09-14T06:16:51.477 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3427000000000003e-05 2024-09-14T06:16:51.477 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.477 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.477 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.477 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180956+0000", 2024-09-14T06:16:51.477 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6195999999999997e-05 2024-09-14T06:16:51.478 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.478 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.478 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.478 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.478 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.478 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13233 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.478 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:16:51.478 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.349964898, 2024-09-14T06:16:51.478 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081924860000000002, 2024-09-14T06:16:51.478 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.478 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.478 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.478 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.478 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.478 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.479 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13233 2024-09-14T06:16:51.479 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.479 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.479 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.479 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.479 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:16:51.479 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.479 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.479 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.480 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.480 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:16:51.480 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.480 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.480 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.480 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.480 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099120+0000", 2024-09-14T06:16:51.480 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:16:51.480 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.480 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.480 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.480 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099122+0000", 2024-09-14T06:16:51.480 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:16:51.481 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.481 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.481 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.481 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099126+0000", 2024-09-14T06:16:51.481 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2540000000000003e-06 2024-09-14T06:16:51.481 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.481 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.481 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.481 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099130+0000", 2024-09-14T06:16:51.481 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4060000000000001e-06 2024-09-14T06:16:51.481 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.481 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.481 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.482 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163169+0000", 2024-09-14T06:16:51.482 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064039415000000002 2024-09-14T06:16:51.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.482 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.482 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.482 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163260+0000", 2024-09-14T06:16:51.482 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1038000000000005e-05 2024-09-14T06:16:51.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.482 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.482 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.482 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163640+0000", 2024-09-14T06:16:51.482 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037954699999999999 2024-09-14T06:16:51.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.483 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.483 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.483 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166093+0000", 2024-09-14T06:16:51.483 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024533229999999999 2024-09-14T06:16:51.483 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.483 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.483 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.483 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180993+0000", 2024-09-14T06:16:51.483 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01489997 2024-09-14T06:16:51.483 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.483 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.483 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.483 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181020+0000", 2024-09-14T06:16:51.483 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6806999999999999e-05 2024-09-14T06:16:51.483 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.484 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.484 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.484 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181044+0000", 2024-09-14T06:16:51.484 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4405000000000001e-05 2024-09-14T06:16:51.484 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.484 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.484 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.484 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.484 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.484 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13234 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.484 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:16:51.484 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.349922693, 2024-09-14T06:16:51.484 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082015597999999995, 2024-09-14T06:16:51.484 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.485 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.485 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.485 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.485 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.485 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.485 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13234 2024-09-14T06:16:51.485 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.485 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.485 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.485 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.485 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:16:51.485 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.485 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.485 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:16:51.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:16:51.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0999999999999998e-07 2024-09-14T06:16:51.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099166+0000", 2024-09-14T06:16:51.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:16:51.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.487 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.487 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.487 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099171+0000", 2024-09-14T06:16:51.487 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6949999999999997e-06 2024-09-14T06:16:51.487 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.487 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.487 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.487 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099175+0000", 2024-09-14T06:16:51.487 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9029999999999997e-06 2024-09-14T06:16:51.487 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.487 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.487 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.488 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164063+0000", 2024-09-14T06:16:51.488 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064887608999999999 2024-09-14T06:16:51.488 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.488 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.488 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.488 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164111+0000", 2024-09-14T06:16:51.488 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7648999999999997e-05 2024-09-14T06:16:51.488 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.488 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.488 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.489 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164351+0000", 2024-09-14T06:16:51.489 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024072899999999999 2024-09-14T06:16:51.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.489 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.489 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.489 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167561+0000", 2024-09-14T06:16:51.489 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032094179999999999 2024-09-14T06:16:51.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.489 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.489 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.489 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181093+0000", 2024-09-14T06:16:51.489 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013532245 2024-09-14T06:16:51.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.490 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.490 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181139+0000", 2024-09-14T06:16:51.490 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6483999999999997e-05 2024-09-14T06:16:51.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.490 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.490 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181177+0000", 2024-09-14T06:16:51.490 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7861000000000002e-05 2024-09-14T06:16:51.490 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.491 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.491 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.491 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.491 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13235 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.491 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:16:51.491 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.349815419, 2024-09-14T06:16:51.491 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081988088000000001, 2024-09-14T06:16:51.491 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.491 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.491 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.491 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.491 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.492 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.492 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13235 2024-09-14T06:16:51.492 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.492 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.492 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.492 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.492 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:16:51.492 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.492 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.492 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.492 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.492 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:16:51.492 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.492 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.493 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.493 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.493 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099270+0000", 2024-09-14T06:16:51.493 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7400000000000003e-07 2024-09-14T06:16:51.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.493 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.493 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.493 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099276+0000", 2024-09-14T06:16:51.493 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:16:51.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.493 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.493 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.493 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099284+0000", 2024-09-14T06:16:51.493 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7979999999999995e-06 2024-09-14T06:16:51.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.494 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.494 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.494 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099289+0000", 2024-09-14T06:16:51.494 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:16:51.494 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.494 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.494 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.494 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164664+0000", 2024-09-14T06:16:51.494 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065374901999999999 2024-09-14T06:16:51.494 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.494 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.494 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.494 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164755+0000", 2024-09-14T06:16:51.494 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0790000000000003e-05 2024-09-14T06:16:51.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165143+0000", 2024-09-14T06:16:51.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000387904 2024-09-14T06:16:51.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167564+0000", 2024-09-14T06:16:51.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024208989999999998 2024-09-14T06:16:51.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181212+0000", 2024-09-14T06:16:51.496 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013648664 2024-09-14T06:16:51.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.496 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.496 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181237+0000", 2024-09-14T06:16:51.496 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5177999999999998e-05 2024-09-14T06:16:51.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.496 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.496 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181257+0000", 2024-09-14T06:16:51.496 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9660000000000002e-05 2024-09-14T06:16:51.497 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.497 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.497 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.497 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13236 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.497 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:16:51.497 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.3497086, 2024-09-14T06:16:51.497 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082010939000000005, 2024-09-14T06:16:51.497 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.498 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.498 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.498 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.498 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.498 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.498 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13236 2024-09-14T06:16:51.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.498 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.499 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.499 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:16:51.499 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.499 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.499 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:16:51.499 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.499 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.500 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:16:51.500 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.61e-07 2024-09-14T06:16:51.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.500 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.500 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099382+0000", 2024-09-14T06:16:51.500 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3589999999999999e-06 2024-09-14T06:16:51.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.500 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.500 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099388+0000", 2024-09-14T06:16:51.500 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9619999999999998e-06 2024-09-14T06:16:51.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.501 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.501 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099393+0000", 2024-09-14T06:16:51.501 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3410000000000002e-06 2024-09-14T06:16:51.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.501 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.501 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165553+0000", 2024-09-14T06:16:51.501 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066159604999999996 2024-09-14T06:16:51.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165603+0000", 2024-09-14T06:16:51.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0427999999999999e-05 2024-09-14T06:16:51.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165827+0000", 2024-09-14T06:16:51.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022358299999999999 2024-09-14T06:16:51.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177005+0000", 2024-09-14T06:16:51.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011178044 2024-09-14T06:16:51.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.503 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.503 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181302+0000", 2024-09-14T06:16:51.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042970669999999999 2024-09-14T06:16:51.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.503 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.503 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181349+0000", 2024-09-14T06:16:51.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7287999999999999e-05 2024-09-14T06:16:51.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.504 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.504 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.504 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181387+0000", 2024-09-14T06:16:51.504 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7601000000000001e-05 2024-09-14T06:16:51.504 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.504 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.504 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.504 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.504 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.504 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13237 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.504 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:16:51.505 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.349550452999999, 2024-09-14T06:16:51.505 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081927299999999995, 2024-09-14T06:16:51.505 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.505 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.505 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.505 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.505 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.505 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.505 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13237 2024-09-14T06:16:51.505 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.505 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.505 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.505 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.506 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:16:51.506 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.506 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.506 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.506 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.506 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:16:51.506 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.506 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.506 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.506 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.506 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099535+0000", 2024-09-14T06:16:51.506 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:16:51.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.507 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.507 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.507 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099538+0000", 2024-09-14T06:16:51.507 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:16:51.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.507 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.507 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.507 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099545+0000", 2024-09-14T06:16:51.507 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:16:51.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.507 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.507 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099551+0000", 2024-09-14T06:16:51.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1770000000000003e-06 2024-09-14T06:16:51.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166131+0000", 2024-09-14T06:16:51.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066580499000000001 2024-09-14T06:16:51.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166221+0000", 2024-09-14T06:16:51.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0000000000000006e-05 2024-09-14T06:16:51.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.509 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.509 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166622+0000", 2024-09-14T06:16:51.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040029799999999999 2024-09-14T06:16:51.509 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.509 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.509 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177008+0000", 2024-09-14T06:16:51.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010386772000000001 2024-09-14T06:16:51.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.510 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181420+0000", 2024-09-14T06:16:51.510 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044119190000000003 2024-09-14T06:16:51.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.510 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181443+0000", 2024-09-14T06:16:51.510 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2603999999999999e-05 2024-09-14T06:16:51.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.510 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181461+0000", 2024-09-14T06:16:51.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8405000000000001e-05 2024-09-14T06:16:51.511 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.511 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.511 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.511 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13238 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.511 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:16:51.511 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.349514145000001, 2024-09-14T06:16:51.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082014927000000001, 2024-09-14T06:16:51.511 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.511 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.511 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.511 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.512 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.512 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.512 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13238 2024-09-14T06:16:51.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.512 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.512 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.512 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:16:51.512 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.512 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.512 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:16:51.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.513 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.513 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099571+0000", 2024-09-14T06:16:51.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9699999999999996e-07 2024-09-14T06:16:51.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.513 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.513 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099577+0000", 2024-09-14T06:16:51.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2890000000000003e-06 2024-09-14T06:16:51.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.513 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.514 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099587+0000", 2024-09-14T06:16:51.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0002e-05 2024-09-14T06:16:51.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.514 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.514 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099593+0000", 2024-09-14T06:16:51.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4539999999999998e-06 2024-09-14T06:16:51.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.514 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.514 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167024+0000", 2024-09-14T06:16:51.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067431480000000002 2024-09-14T06:16:51.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.514 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167074+0000", 2024-09-14T06:16:51.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9996e-05 2024-09-14T06:16:51.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167288+0000", 2024-09-14T06:16:51.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021410900000000001 2024-09-14T06:16:51.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177997+0000", 2024-09-14T06:16:51.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010708756 2024-09-14T06:16:51.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.516 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181508+0000", 2024-09-14T06:16:51.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003511363 2024-09-14T06:16:51.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.516 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.516 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181552+0000", 2024-09-14T06:16:51.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3609999999999998e-05 2024-09-14T06:16:51.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.516 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.516 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181585+0000", 2024-09-14T06:16:51.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3271e-05 2024-09-14T06:16:51.516 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.516 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.517 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.517 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.517 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13239 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.517 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:16:51.517 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.349422326999999, 2024-09-14T06:16:51.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081998772999999997, 2024-09-14T06:16:51.517 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.517 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.517 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.517 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.517 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.517 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.517 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13239 2024-09-14T06:16:51.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.518 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.518 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.518 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:16:51.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.518 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.518 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:16:51.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.518 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.518 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099663+0000", 2024-09-14T06:16:51.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6100000000000001e-07 2024-09-14T06:16:51.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099666+0000", 2024-09-14T06:16:51.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.056e-06 2024-09-14T06:16:51.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099672+0000", 2024-09-14T06:16:51.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8120000000000002e-06 2024-09-14T06:16:51.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.520 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.520 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.520 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099676+0000", 2024-09-14T06:16:51.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:16:51.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.520 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.520 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.520 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167602+0000", 2024-09-14T06:16:51.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067925867000000001 2024-09-14T06:16:51.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.520 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.520 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167693+0000", 2024-09-14T06:16:51.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0736000000000007e-05 2024-09-14T06:16:51.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.524 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.524 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168065+0000", 2024-09-14T06:16:51.524 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037245399999999998 2024-09-14T06:16:51.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.524 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178003+0000", 2024-09-14T06:16:51.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0099372360000000003 2024-09-14T06:16:51.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181621+0000", 2024-09-14T06:16:51.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036178489999999998 2024-09-14T06:16:51.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181643+0000", 2024-09-14T06:16:51.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2155e-05 2024-09-14T06:16:51.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181661+0000", 2024-09-14T06:16:51.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8147000000000001e-05 2024-09-14T06:16:51.526 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.526 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.526 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.526 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13240 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.526 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:16:51.526 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.349303234000001, 2024-09-14T06:16:51.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081942607000000001, 2024-09-14T06:16:51.526 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.526 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.526 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.527 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.527 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.527 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.527 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13240 2024-09-14T06:16:51.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.527 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.527 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:16:51.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.527 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:16:51.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099782+0000", 2024-09-14T06:16:51.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3900000000000004e-07 2024-09-14T06:16:51.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099787+0000", 2024-09-14T06:16:51.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6189999999999999e-06 2024-09-14T06:16:51.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099793+0000", 2024-09-14T06:16:51.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.998e-06 2024-09-14T06:16:51.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.529 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.529 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099797+0000", 2024-09-14T06:16:51.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8449999999999996e-06 2024-09-14T06:16:51.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.529 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.529 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168494+0000", 2024-09-14T06:16:51.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.068696500999999993 2024-09-14T06:16:51.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.529 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.530 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168590+0000", 2024-09-14T06:16:51.530 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6733999999999999e-05 2024-09-14T06:16:51.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.530 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.530 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.530 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168973+0000", 2024-09-14T06:16:51.530 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038229399999999998 2024-09-14T06:16:51.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178013+0000", 2024-09-14T06:16:51.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0090397199999999994 2024-09-14T06:16:51.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181683+0000", 2024-09-14T06:16:51.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036705819999999999 2024-09-14T06:16:51.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181705+0000", 2024-09-14T06:16:51.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1551000000000001e-05 2024-09-14T06:16:51.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181724+0000", 2024-09-14T06:16:51.533 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9123999999999999e-05 2024-09-14T06:16:51.533 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.533 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.533 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.533 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.533 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13243 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.533 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:16:51.533 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.348769504, 2024-09-14T06:16:51.533 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081767815999999993, 2024-09-14T06:16:51.533 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.534 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.534 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.534 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.534 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.534 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.534 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13243 2024-09-14T06:16:51.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.534 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.534 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.534 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:16:51.534 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.535 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:16:51.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.535 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:16:51.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1e-07 2024-09-14T06:16:51.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.535 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100317+0000", 2024-09-14T06:16:51.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.092e-06 2024-09-14T06:16:51.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100321+0000", 2024-09-14T06:16:51.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7809999999999999e-06 2024-09-14T06:16:51.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100324+0000", 2024-09-14T06:16:51.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.117e-06 2024-09-14T06:16:51.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171113+0000", 2024-09-14T06:16:51.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.070788816000000004 2024-09-14T06:16:51.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171202+0000", 2024-09-14T06:16:51.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8843000000000003e-05 2024-09-14T06:16:51.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.538 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.538 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171584+0000", 2024-09-14T06:16:51.538 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038231099999999998 2024-09-14T06:16:51.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.538 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.538 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178101+0000", 2024-09-14T06:16:51.538 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0065168329999999997 2024-09-14T06:16:51.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.539 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.539 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.539 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182041+0000", 2024-09-14T06:16:51.539 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039401810000000001 2024-09-14T06:16:51.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.539 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.539 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.539 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182065+0000", 2024-09-14T06:16:51.539 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3623000000000001e-05 2024-09-14T06:16:51.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.539 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182083+0000", 2024-09-14T06:16:51.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7909000000000001e-05 2024-09-14T06:16:51.540 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.540 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.540 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.540 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13244 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.540 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:16:51.540 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.348754683999999, 2024-09-14T06:16:51.541 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081837800000000002, 2024-09-14T06:16:51.541 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.541 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.541 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.541 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.541 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.541 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.541 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13244 2024-09-14T06:16:51.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.541 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.542 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.542 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:16:51.542 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.542 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.542 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:16:51.542 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.542 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.542 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:16:51.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:16:51.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100337+0000", 2024-09-14T06:16:51.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.939e-06 2024-09-14T06:16:51.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100341+0000", 2024-09-14T06:16:51.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2320000000000003e-06 2024-09-14T06:16:51.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.544 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100344+0000", 2024-09-14T06:16:51.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0759999999999999e-06 2024-09-14T06:16:51.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.544 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.544 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171996+0000", 2024-09-14T06:16:51.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.071651919999999994 2024-09-14T06:16:51.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.544 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.544 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172079+0000", 2024-09-14T06:16:51.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2579999999999999e-05 2024-09-14T06:16:51.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.545 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.545 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172457+0000", 2024-09-14T06:16:51.545 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037785100000000001 2024-09-14T06:16:51.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.545 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.545 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178104+0000", 2024-09-14T06:16:51.545 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056471730000000001 2024-09-14T06:16:51.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.545 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.545 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182123+0000", 2024-09-14T06:16:51.545 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040190019999999998 2024-09-14T06:16:51.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.546 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.546 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182147+0000", 2024-09-14T06:16:51.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4241000000000001e-05 2024-09-14T06:16:51.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.546 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.546 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182168+0000", 2024-09-14T06:16:51.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0492999999999998e-05 2024-09-14T06:16:51.546 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.546 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.546 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.547 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.547 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13245 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.547 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:16:51.547 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.348735177, 2024-09-14T06:16:51.547 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081893899000000006, 2024-09-14T06:16:51.547 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.547 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.547 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.547 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.547 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.547 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.547 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13245 2024-09-14T06:16:51.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.548 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.548 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.548 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.548 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:16:51.548 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:16:51.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100350+0000", 2024-09-14T06:16:51.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5499999999999999e-07 2024-09-14T06:16:51.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100353+0000", 2024-09-14T06:16:51.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:16:51.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100357+0000", 2024-09-14T06:16:51.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8500000000000004e-06 2024-09-14T06:16:51.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.551 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100360+0000", 2024-09-14T06:16:51.551 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8700000000000001e-06 2024-09-14T06:16:51.551 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.551 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.551 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.551 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172862+0000", 2024-09-14T06:16:51.552 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.072501992000000001 2024-09-14T06:16:51.552 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.552 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.552 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.552 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172950+0000", 2024-09-14T06:16:51.553 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8988999999999998e-05 2024-09-14T06:16:51.553 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.553 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.553 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.553 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173324+0000", 2024-09-14T06:16:51.553 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037366199999999997 2024-09-14T06:16:51.553 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.553 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.553 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.553 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178107+0000", 2024-09-14T06:16:51.553 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047830520000000003 2024-09-14T06:16:51.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182202+0000", 2024-09-14T06:16:51.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040949280000000003 2024-09-14T06:16:51.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.555 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182226+0000", 2024-09-14T06:16:51.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3535000000000001e-05 2024-09-14T06:16:51.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.555 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.555 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.555 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182243+0000", 2024-09-14T06:16:51.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7445e-05 2024-09-14T06:16:51.555 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.555 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.555 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.556 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13246 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.556 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:16:51.556 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.348720855, 2024-09-14T06:16:51.556 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081962276000000001, 2024-09-14T06:16:51.556 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.556 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.556 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.556 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.556 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.556 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.556 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13246 2024-09-14T06:16:51.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.556 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.557 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.557 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.557 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:16:51.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.557 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.557 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.557 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:16:51.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.557 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.557 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.558 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:16:51.558 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0800000000000001e-07 2024-09-14T06:16:51.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.558 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.558 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100367+0000", 2024-09-14T06:16:51.558 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8439999999999998e-06 2024-09-14T06:16:51.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.558 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.558 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100372+0000", 2024-09-14T06:16:51.558 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4009999999999996e-06 2024-09-14T06:16:51.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.559 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.559 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100378+0000", 2024-09-14T06:16:51.559 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7180000000000001e-06 2024-09-14T06:16:51.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.559 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.559 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.559 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173780+0000", 2024-09-14T06:16:51.559 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.073401991 2024-09-14T06:16:51.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.559 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.559 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.559 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173879+0000", 2024-09-14T06:16:51.559 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9449000000000003e-05 2024-09-14T06:16:51.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.559 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.560 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174240+0000", 2024-09-14T06:16:51.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036118100000000002 2024-09-14T06:16:51.560 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.560 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.560 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178109+0000", 2024-09-14T06:16:51.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038690619999999999 2024-09-14T06:16:51.560 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.560 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.560 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182281+0000", 2024-09-14T06:16:51.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041715900000000002 2024-09-14T06:16:51.560 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.561 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.561 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182306+0000", 2024-09-14T06:16:51.561 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4477e-05 2024-09-14T06:16:51.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.561 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.561 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182326+0000", 2024-09-14T06:16:51.561 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0254999999999998e-05 2024-09-14T06:16:51.561 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.561 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.561 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.562 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13247 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.562 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:16:51.562 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.348623677999999, 2024-09-14T06:16:51.562 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966593000000004, 2024-09-14T06:16:51.562 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.562 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.562 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.562 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.562 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.562 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.562 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13247 2024-09-14T06:16:51.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.562 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.563 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:16:51.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.563 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.563 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:16:51.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.563 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.563 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100462+0000", 2024-09-14T06:16:51.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9800000000000002e-07 2024-09-14T06:16:51.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.564 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.564 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100466+0000", 2024-09-14T06:16:51.564 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4039999999999997e-06 2024-09-14T06:16:51.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.564 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.564 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100475+0000", 2024-09-14T06:16:51.564 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3680000000000008e-06 2024-09-14T06:16:51.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.564 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.564 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100482+0000", 2024-09-14T06:16:51.564 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8630000000000002e-06 2024-09-14T06:16:51.565 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.565 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.565 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.565 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174669+0000", 2024-09-14T06:16:51.565 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074186849999999999 2024-09-14T06:16:51.565 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.565 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.565 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.565 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174758+0000", 2024-09-14T06:16:51.565 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9021e-05 2024-09-14T06:16:51.565 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.565 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.565 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.565 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175137+0000", 2024-09-14T06:16:51.566 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378511 2024-09-14T06:16:51.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.566 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.566 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178158+0000", 2024-09-14T06:16:51.566 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003020956 2024-09-14T06:16:51.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.566 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.566 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182386+0000", 2024-09-14T06:16:51.566 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042278840000000003 2024-09-14T06:16:51.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.566 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.566 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182410+0000", 2024-09-14T06:16:51.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4026e-05 2024-09-14T06:16:51.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182427+0000", 2024-09-14T06:16:51.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7711999999999999e-05 2024-09-14T06:16:51.567 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.567 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.567 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.567 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13248 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.567 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:16:51.567 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.348541597000001, 2024-09-14T06:16:51.568 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966527999999997, 2024-09-14T06:16:51.568 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.568 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.568 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.568 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.568 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.568 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.568 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13248 2024-09-14T06:16:51.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.568 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.568 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.569 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.569 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:16:51.569 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.569 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.569 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:16:51.569 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.569 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.569 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:16:51.570 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:16:51.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.570 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.570 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100548+0000", 2024-09-14T06:16:51.570 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:16:51.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.570 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.570 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100557+0000", 2024-09-14T06:16:51.570 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:16:51.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.570 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.571 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100563+0000", 2024-09-14T06:16:51.571 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2519999999999996e-06 2024-09-14T06:16:51.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.571 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.571 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.571 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175572+0000", 2024-09-14T06:16:51.571 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075008459 2024-09-14T06:16:51.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.571 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.571 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.571 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175656+0000", 2024-09-14T06:16:51.571 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3906999999999995e-05 2024-09-14T06:16:51.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.572 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.572 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176010+0000", 2024-09-14T06:16:51.572 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00035447499999999998 2024-09-14T06:16:51.572 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.572 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.572 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178304+0000", 2024-09-14T06:16:51.572 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0022941379999999998 2024-09-14T06:16:51.572 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.572 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.572 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182467+0000", 2024-09-14T06:16:51.572 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041632020000000004 2024-09-14T06:16:51.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.573 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.573 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.573 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182491+0000", 2024-09-14T06:16:51.573 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3733000000000001e-05 2024-09-14T06:16:51.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.573 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.573 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.573 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182509+0000", 2024-09-14T06:16:51.573 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8113000000000001e-05 2024-09-14T06:16:51.573 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.573 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.573 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.574 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13249 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.574 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:16:51.574 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.348506663, 2024-09-14T06:16:51.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082037225000000005, 2024-09-14T06:16:51.574 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.574 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.574 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.574 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.574 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.574 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.574 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13249 2024-09-14T06:16:51.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.574 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:16:51.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:16:51.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:16:51.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4600000000000005e-07 2024-09-14T06:16:51.576 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.576 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.576 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.576 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100584+0000", 2024-09-14T06:16:51.576 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.947e-06 2024-09-14T06:16:51.576 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.576 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.576 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.576 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100593+0000", 2024-09-14T06:16:51.576 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9760000000000001e-06 2024-09-14T06:16:51.576 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.576 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.576 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.576 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100597+0000", 2024-09-14T06:16:51.577 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9689999999999996e-06 2024-09-14T06:16:51.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.577 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.577 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.577 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176461+0000", 2024-09-14T06:16:51.577 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075864132000000001 2024-09-14T06:16:51.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.577 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.577 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.577 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176516+0000", 2024-09-14T06:16:51.577 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4416999999999998e-05 2024-09-14T06:16:51.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.577 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.577 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.578 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176734+0000", 2024-09-14T06:16:51.578 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021859399999999999 2024-09-14T06:16:51.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.578 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.578 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.578 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178308+0000", 2024-09-14T06:16:51.578 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015733660000000001 2024-09-14T06:16:51.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.579 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.579 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.579 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182573+0000", 2024-09-14T06:16:51.579 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042649200000000002 2024-09-14T06:16:51.579 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.579 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.579 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.579 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182597+0000", 2024-09-14T06:16:51.579 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4355e-05 2024-09-14T06:16:51.579 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.579 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.580 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.580 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182615+0000", 2024-09-14T06:16:51.580 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8003e-05 2024-09-14T06:16:51.580 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.580 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.580 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.580 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.580 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.580 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13250 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:51.580 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:16:51.581 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.348342058, 2024-09-14T06:16:51.581 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081983890000000004, 2024-09-14T06:16:51.581 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:51.581 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:51.581 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:51.581 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:51.581 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:51.581 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:51.581 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13250 2024-09-14T06:16:51.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.581 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:51.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.581 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:51.581 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:16:51.582 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.582 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.582 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.582 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:51.582 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:16:51.582 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:51.582 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.582 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.582 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:51.582 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100743+0000", 2024-09-14T06:16:51.582 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:16:51.582 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.582 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.583 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:51.583 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100753+0000", 2024-09-14T06:16:51.583 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9979999999999998e-06 2024-09-14T06:16:51.583 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.583 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.583 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:51.583 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100772+0000", 2024-09-14T06:16:51.583 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8664e-05 2024-09-14T06:16:51.583 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.583 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.583 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:51.583 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100779+0000", 2024-09-14T06:16:51.583 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.046e-06 2024-09-14T06:16:51.583 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.584 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:51.584 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177049+0000", 2024-09-14T06:16:51.584 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076270819000000004 2024-09-14T06:16:51.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.584 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:51.584 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177140+0000", 2024-09-14T06:16:51.584 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0746000000000002e-05 2024-09-14T06:16:51.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.584 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:51.584 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177519+0000", 2024-09-14T06:16:51.584 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378656 2024-09-14T06:16:51.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.585 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:51.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178349+0000", 2024-09-14T06:16:51.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00083061599999999995 2024-09-14T06:16:51.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.585 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:51.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182679+0000", 2024-09-14T06:16:51.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043298379999999999 2024-09-14T06:16:51.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.585 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:51.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182705+0000", 2024-09-14T06:16:51.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5602999999999999e-05 2024-09-14T06:16:51.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:51.586 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:51.586 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:51.586 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182726+0000", 2024-09-14T06:16:51.586 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1297e-05 2024-09-14T06:16:51.586 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.586 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.586 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.586 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:51.586 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:51.586 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:51.587 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:51.450+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:51.587 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:16:51.589 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-14T06:16:51.631 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:51.630+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:51.689 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:51.687+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:51.761 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:51.761 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:51.762 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:16:51.762 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:51.775 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:16:51.790 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:51.787+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:51.819 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-14T06:16:51.825 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-14T06:16:51.859 INFO:teuthology.orchestra.run.smithi039.stdout:2172 punch from 0x3f54f7f to 0x3f57955, (0x29d6 bytes) 2024-09-14T06:16:51.860 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:16:51.873 INFO:teuthology.orchestra.run.smithi039.stdout:2173 trunc from 0xa6efe8a to 0x4b5fb3c 2024-09-14T06:16:51.891 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:51.891+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:51.913 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:51.914 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:51.914 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:16:51.914 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:16:51.914 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:51.934 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:16:51.942 INFO:teuthology.orchestra.run.smithi039.stdout:2178 write 0x5d09f85 thru 0x5d13b0b (0x9b87 bytes) 2024-09-14T06:16:51.956 INFO:teuthology.orchestra.run.smithi039.stdout:2179 write 0x9682e02 thru 0x9688070 (0x526f bytes) 2024-09-14T06:16:51.960 INFO:teuthology.orchestra.run.smithi039.stdout:2180 read 0x925bfe2 thru 0x9268668 (0xc687 bytes) 2024-09-14T06:16:51.960 INFO:teuthology.orchestra.run.smithi039.stdout:2181 writesame 0x819ef49 thru 0x819f26f data_size 0x1(0x327 bytes) 2024-09-14T06:16:51.962 INFO:teuthology.orchestra.run.smithi039.stdout:2183 read 0x76065f2 thru 0x7606e7d (0x88c bytes) 2024-09-14T06:16:51.962 INFO:teuthology.orchestra.run.smithi039.stdout:2184 punch from 0x903ab2c to 0x903d80e, (0x2ce2 bytes) 2024-09-14T06:16:51.962 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:51.962 INFO:teuthology.orchestra.run.smithi039.stdout:2188 punch from 0x8e0b01d to 0x8e1a508, (0xf4eb bytes) 2024-09-14T06:16:51.963 INFO:teuthology.orchestra.run.smithi039.stdout:2190 read 0xb7ab36 thru 0xb883f3 (0xd8be bytes) 2024-09-14T06:16:51.963 INFO:teuthology.orchestra.run.smithi039.stdout:2191 write 0x3145fe3 thru 0x314601b (0x39 bytes) 2024-09-14T06:16:51.964 INFO:teuthology.orchestra.run.smithi039.stdout:2193 write 0x17b2a5e thru 0x17b4ecb (0x246e bytes) 2024-09-14T06:16:51.966 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:51.979 INFO:teuthology.orchestra.run.smithi039.stdout:2195 write 0xe0d605d thru 0xe0dee6e (0x8e12 bytes) 2024-09-14T06:16:51.983 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:51.983 INFO:teuthology.orchestra.run.smithi039.stdout:2197 read 0x16004b0 thru 0x160243e (0x1f8f bytes) 2024-09-14T06:16:51.983 INFO:teuthology.orchestra.run.smithi039.stdout:2198 read 0x6ed8785 thru 0x6ee00ee (0x796a bytes) 2024-09-14T06:16:51.983 INFO:teuthology.orchestra.run.smithi039.stdout:2199 writesame 0x1178758 thru 0x1187073 data_size 0x6(0xe91c bytes) 2024-09-14T06:16:51.987 INFO:teuthology.orchestra.run.smithi039.stdout:2200 writesame 0x1127b5c thru 0x112e347 data_size 0xc(0x67ec bytes) 2024-09-14T06:16:51.989 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:51.992 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:51.991+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:51.995 INFO:teuthology.orchestra.run.smithi039.stdout:2202 write 0xe354163 thru 0xe3558df (0x177d bytes) 2024-09-14T06:16:51.998 INFO:teuthology.orchestra.run.smithi039.stdout:2203 read 0xaf51990 thru 0xaf5214f (0x7c0 bytes) 2024-09-14T06:16:51.998 INFO:teuthology.orchestra.run.smithi039.stdout:2204 write 0x1b59318 thru 0x1b61c18 (0x8901 bytes) 2024-09-14T06:16:52.007 INFO:teuthology.orchestra.run.smithi039.stdout:2206 writesame 0xeabfa3d thru 0xeac7d0a data_size 0x3(0x82ce bytes) 2024-09-14T06:16:52.009 INFO:teuthology.orchestra.run.smithi039.stdout:2207 write 0x37f6167 thru 0x37fb95f (0x57f9 bytes) 2024-09-14T06:16:52.016 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-14T06:16:52.023 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:52.023 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:52.024 INFO:teuthology.orchestra.run.smithi039.stdout:2208 trunc from 0xeac7d0b to 0x52442fc 2024-09-14T06:16:52.040 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:52.040 INFO:teuthology.orchestra.run.smithi039.stdout:2211 punch from 0x115311f to 0x1160ef3, (0xddd4 bytes) 2024-09-14T06:16:52.040 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:52.040 INFO:teuthology.orchestra.run.smithi039.stdout:2213 read 0x4386f94 thru 0x438d655 (0x66c2 bytes) 2024-09-14T06:16:52.040 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:52.043 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:52.043 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:52.044 INFO:teuthology.orchestra.run.smithi039.stdout:2216 trunc from 0x52442fc to 0x2873a01 2024-09-14T06:16:52.062 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:52.062 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:52.064 INFO:teuthology.orchestra.run.smithi039.stdout:2217 writesame 0x41b0eb2 thru 0x41b671b data_size 0x1(0x586a bytes) 2024-09-14T06:16:52.067 INFO:teuthology.orchestra.run.smithi039.stdout:2218 trunc from 0x41b671c to 0x2698769 2024-09-14T06:16:52.075 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:52.075 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:16:52.075 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:16:52.075 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:16:52.075 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.075 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16116 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 851968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.075 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654230+0000", 2024-09-14T06:16:52.075 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.421236208, 2024-09-14T06:16:52.076 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.033897548, 2024-09-14T06:16:52.076 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.076 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.076 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.076 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.076 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.076 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.076 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16116 2024-09-14T06:16:52.076 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.076 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.076 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.076 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.076 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654230+0000", 2024-09-14T06:16:52.077 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.077 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.077 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.077 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.077 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654230+0000", 2024-09-14T06:16:52.077 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.077 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.077 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.077 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.077 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654231+0000", 2024-09-14T06:16:52.077 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2699999999999999e-07 2024-09-14T06:16:52.077 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.077 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.078 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.078 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654233+0000", 2024-09-14T06:16:52.078 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6259999999999999e-06 2024-09-14T06:16:52.078 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.078 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.078 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.078 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654243+0000", 2024-09-14T06:16:52.078 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5969999999999998e-06 2024-09-14T06:16:52.078 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.078 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.078 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.078 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654250+0000", 2024-09-14T06:16:52.078 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.007e-06 2024-09-14T06:16:52.078 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.079 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.079 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.079 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.687631+0000", 2024-09-14T06:16:52.079 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.033381704999999998 2024-09-14T06:16:52.079 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.079 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.079 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.079 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.687852+0000", 2024-09-14T06:16:52.079 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022096099999999999 2024-09-14T06:16:52.079 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.079 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.079 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.079 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688127+0000", 2024-09-14T06:16:52.079 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00027492499999999997 2024-09-14T06:16:52.080 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.080 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.080 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.080 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.080 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.080 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16117 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.080 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654259+0000", 2024-09-14T06:16:52.080 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.421206765000001, 2024-09-14T06:16:52.080 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.034316067999999998, 2024-09-14T06:16:52.080 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.080 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.080 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.080 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.080 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.081 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.081 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16117 2024-09-14T06:16:52.081 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.081 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.081 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.081 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.081 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654259+0000", 2024-09-14T06:16:52.081 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.081 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.081 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.081 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.081 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654259+0000", 2024-09-14T06:16:52.082 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.082 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.082 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.082 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654260+0000", 2024-09-14T06:16:52.082 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1699999999999997e-07 2024-09-14T06:16:52.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.082 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.082 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.082 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654263+0000", 2024-09-14T06:16:52.082 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6029999999999999e-06 2024-09-14T06:16:52.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.082 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.082 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.083 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654272+0000", 2024-09-14T06:16:52.083 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1600000000000004e-06 2024-09-14T06:16:52.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.083 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.083 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.083 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654279+0000", 2024-09-14T06:16:52.083 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9469999999999997e-06 2024-09-14T06:16:52.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.083 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.083 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.083 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688248+0000", 2024-09-14T06:16:52.083 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.033969223999999999 2024-09-14T06:16:52.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.083 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.084 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.084 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688387+0000", 2024-09-14T06:16:52.084 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013955200000000001 2024-09-14T06:16:52.084 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.084 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.084 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.084 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688575+0000", 2024-09-14T06:16:52.084 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000187865 2024-09-14T06:16:52.085 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.086 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.086 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.086 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.086 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.087 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16118 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.087 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654288+0000", 2024-09-14T06:16:52.087 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.421178379000001, 2024-09-14T06:16:52.087 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.034884971000000001, 2024-09-14T06:16:52.087 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.088 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.088 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.088 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.088 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.088 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.088 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16118 2024-09-14T06:16:52.089 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.089 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.089 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.089 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.089 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654288+0000", 2024-09-14T06:16:52.089 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.089 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.090 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.090 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.090 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654288+0000", 2024-09-14T06:16:52.090 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.090 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.090 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.090 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.090 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654288+0000", 2024-09-14T06:16:52.090 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1399999999999996e-07 2024-09-14T06:16:52.091 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.091 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.091 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.091 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654291+0000", 2024-09-14T06:16:52.091 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4700000000000001e-06 2024-09-14T06:16:52.091 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.092 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.092 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.092 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654300+0000", 2024-09-14T06:16:52.092 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:16:52.092 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.092 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.093 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.093 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654307+0000", 2024-09-14T06:16:52.093 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1430000000000001e-06 2024-09-14T06:16:52.093 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.093 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.093 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.093 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688682+0000", 2024-09-14T06:16:52.094 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.034375304000000002 2024-09-14T06:16:52.094 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.094 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.094 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.094 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688897+0000", 2024-09-14T06:16:52.094 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021480499999999999 2024-09-14T06:16:52.094 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.094 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.094 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.094 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689173+0000", 2024-09-14T06:16:52.094 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00027551499999999999 2024-09-14T06:16:52.095 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.095 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.095 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.095 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.095 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.095 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16119 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.095 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654316+0000", 2024-09-14T06:16:52.095 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.421149915999999, 2024-09-14T06:16:52.095 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035292513999999997, 2024-09-14T06:16:52.095 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.095 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.095 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.095 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.095 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.096 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.096 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16119 2024-09-14T06:16:52.096 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.096 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.096 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.096 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.096 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654316+0000", 2024-09-14T06:16:52.096 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.096 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.096 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.096 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.096 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654316+0000", 2024-09-14T06:16:52.096 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.097 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.097 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.097 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.097 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654317+0000", 2024-09-14T06:16:52.097 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.37e-07 2024-09-14T06:16:52.097 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.097 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.097 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.097 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654319+0000", 2024-09-14T06:16:52.097 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6469999999999998e-06 2024-09-14T06:16:52.097 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.097 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.097 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.098 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654332+0000", 2024-09-14T06:16:52.098 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2330000000000001e-05 2024-09-14T06:16:52.098 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.098 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.098 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.098 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654339+0000", 2024-09-14T06:16:52.098 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2130000000000002e-06 2024-09-14T06:16:52.098 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.098 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.098 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.098 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689282+0000", 2024-09-14T06:16:52.098 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.034942742999999998 2024-09-14T06:16:52.098 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.098 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.098 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.099 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689424+0000", 2024-09-14T06:16:52.099 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014175900000000001 2024-09-14T06:16:52.099 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.099 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.099 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.099 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689609+0000", 2024-09-14T06:16:52.099 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00018508500000000001 2024-09-14T06:16:52.099 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.099 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.100 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.100 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.100 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.100 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16120 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.100 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654348+0000", 2024-09-14T06:16:52.100 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.421117668999999, 2024-09-14T06:16:52.100 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035856590000000001, 2024-09-14T06:16:52.100 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.100 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.100 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.100 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.100 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.101 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.101 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16120 2024-09-14T06:16:52.101 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.101 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.101 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.101 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.101 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654348+0000", 2024-09-14T06:16:52.101 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.101 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.101 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.101 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.101 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654348+0000", 2024-09-14T06:16:52.101 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.101 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.102 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.102 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.102 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654349+0000", 2024-09-14T06:16:52.102 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2699999999999999e-07 2024-09-14T06:16:52.102 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.102 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.102 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.102 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654352+0000", 2024-09-14T06:16:52.102 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6400000000000001e-06 2024-09-14T06:16:52.102 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.102 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.102 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.102 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654363+0000", 2024-09-14T06:16:52.103 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1715999999999999e-05 2024-09-14T06:16:52.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.103 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.103 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.103 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654371+0000", 2024-09-14T06:16:52.103 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5769999999999996e-06 2024-09-14T06:16:52.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.103 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.103 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.103 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689718+0000", 2024-09-14T06:16:52.104 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035347376999999999 2024-09-14T06:16:52.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.104 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.104 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.104 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689941+0000", 2024-09-14T06:16:52.104 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022233499999999999 2024-09-14T06:16:52.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.104 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.104 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.104 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.690205+0000", 2024-09-14T06:16:52.104 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00026421799999999998 2024-09-14T06:16:52.105 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.105 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.105 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.105 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.105 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.105 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16121 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 3145728~7288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.105 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654381+0000", 2024-09-14T06:16:52.105 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.421085352, 2024-09-14T06:16:52.105 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.036696549000000002, 2024-09-14T06:16:52.105 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.105 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.105 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.106 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.106 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.106 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.106 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16121 2024-09-14T06:16:52.106 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.106 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.106 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.106 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.106 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654381+0000", 2024-09-14T06:16:52.106 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.106 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.106 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.106 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.106 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654381+0000", 2024-09-14T06:16:52.107 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.107 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.107 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.107 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.107 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654381+0000", 2024-09-14T06:16:52.107 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9999999999999997e-07 2024-09-14T06:16:52.107 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.107 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.107 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.107 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654384+0000", 2024-09-14T06:16:52.107 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8499999999999998e-06 2024-09-14T06:16:52.107 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.107 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.107 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.108 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654404+0000", 2024-09-14T06:16:52.108 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9355999999999999e-05 2024-09-14T06:16:52.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.108 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.108 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.108 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654417+0000", 2024-09-14T06:16:52.108 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.295e-05 2024-09-14T06:16:52.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.108 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.108 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.108 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.690327+0000", 2024-09-14T06:16:52.108 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035910899000000003 2024-09-14T06:16:52.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.108 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.109 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.109 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.690557+0000", 2024-09-14T06:16:52.109 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022935499999999999 2024-09-14T06:16:52.109 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.109 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.109 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.109 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.691077+0000", 2024-09-14T06:16:52.109 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00052043900000000004 2024-09-14T06:16:52.109 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.109 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.109 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.109 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.109 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.109 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16122 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.110 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654428+0000", 2024-09-14T06:16:52.110 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.421038006, 2024-09-14T06:16:52.110 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.037792064, 2024-09-14T06:16:52.110 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.110 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.110 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.110 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.110 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.110 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.110 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16122 2024-09-14T06:16:52.110 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.110 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.110 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.110 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.111 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654428+0000", 2024-09-14T06:16:52.111 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.111 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.111 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.111 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654428+0000", 2024-09-14T06:16:52.111 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.111 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.111 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.111 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654429+0000", 2024-09-14T06:16:52.111 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8999999999999995e-07 2024-09-14T06:16:52.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.111 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.112 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.112 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654432+0000", 2024-09-14T06:16:52.112 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0130000000000002e-06 2024-09-14T06:16:52.112 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.112 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.112 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.112 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654442+0000", 2024-09-14T06:16:52.112 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9199999999999999e-06 2024-09-14T06:16:52.112 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.112 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.112 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.113 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654449+0000", 2024-09-14T06:16:52.113 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:16:52.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.113 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.113 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.113 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.691192+0000", 2024-09-14T06:16:52.113 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.036742958999999999 2024-09-14T06:16:52.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.113 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.113 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.114 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.691445+0000", 2024-09-14T06:16:52.114 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000253608 2024-09-14T06:16:52.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.114 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.114 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692220+0000", 2024-09-14T06:16:52.114 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00077489400000000004 2024-09-14T06:16:52.114 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.114 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.114 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.114 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16123 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.114 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654458+0000", 2024-09-14T06:16:52.115 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.421008017, 2024-09-14T06:16:52.115 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038257820999999997, 2024-09-14T06:16:52.115 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.115 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.115 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.115 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.115 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.115 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.115 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16123 2024-09-14T06:16:52.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.116 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.116 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.116 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.116 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654458+0000", 2024-09-14T06:16:52.116 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.116 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.116 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.116 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654458+0000", 2024-09-14T06:16:52.116 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.116 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.117 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.117 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654459+0000", 2024-09-14T06:16:52.117 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0699999999999996e-07 2024-09-14T06:16:52.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.117 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.117 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.117 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654461+0000", 2024-09-14T06:16:52.117 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.633e-06 2024-09-14T06:16:52.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.117 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.117 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.117 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654470+0000", 2024-09-14T06:16:52.117 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0440000000000001e-06 2024-09-14T06:16:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654496+0000", 2024-09-14T06:16:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5279999999999999e-05 2024-09-14T06:16:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692393+0000", 2024-09-14T06:16:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.037897532999999997 2024-09-14T06:16:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.119 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692526+0000", 2024-09-14T06:16:52.119 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013326899999999999 2024-09-14T06:16:52.119 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.119 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.119 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.119 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692716+0000", 2024-09-14T06:16:52.119 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00018935499999999999 2024-09-14T06:16:52.119 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.119 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.119 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.119 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.120 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.120 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16124 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.120 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654506+0000", 2024-09-14T06:16:52.120 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.420959679999999, 2024-09-14T06:16:52.120 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038503725000000003, 2024-09-14T06:16:52.120 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.120 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.120 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.120 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.120 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.120 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.120 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16124 2024-09-14T06:16:52.120 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.121 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.121 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.121 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654506+0000", 2024-09-14T06:16:52.121 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.121 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.121 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654506+0000", 2024-09-14T06:16:52.121 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.122 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.122 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654507+0000", 2024-09-14T06:16:52.122 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1299999999999999e-07 2024-09-14T06:16:52.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.122 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.122 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.123 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654518+0000", 2024-09-14T06:16:52.123 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0696999999999999e-05 2024-09-14T06:16:52.123 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.123 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.123 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.123 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654531+0000", 2024-09-14T06:16:52.124 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3596e-05 2024-09-14T06:16:52.124 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.124 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.124 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.124 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654539+0000", 2024-09-14T06:16:52.124 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6170000000000001e-06 2024-09-14T06:16:52.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.125 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.125 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.125 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692766+0000", 2024-09-14T06:16:52.125 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038227401000000001 2024-09-14T06:16:52.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.125 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.126 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.126 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692869+0000", 2024-09-14T06:16:52.126 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000102216 2024-09-14T06:16:52.126 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.126 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.126 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.126 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693010+0000", 2024-09-14T06:16:52.126 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000141385 2024-09-14T06:16:52.127 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.127 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.127 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.127 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.127 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.127 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16125 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.128 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654549+0000", 2024-09-14T06:16:52.128 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.420916524000001, 2024-09-14T06:16:52.128 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038734667, 2024-09-14T06:16:52.128 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.128 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.128 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.129 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.129 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.129 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.129 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16125 2024-09-14T06:16:52.129 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.130 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.130 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.130 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.130 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654549+0000", 2024-09-14T06:16:52.130 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.130 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.130 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.130 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654549+0000", 2024-09-14T06:16:52.130 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.130 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.131 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.131 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654550+0000", 2024-09-14T06:16:52.131 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6700000000000003e-07 2024-09-14T06:16:52.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.131 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.131 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.131 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654562+0000", 2024-09-14T06:16:52.131 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1926000000000001e-05 2024-09-14T06:16:52.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.131 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.131 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.131 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654575+0000", 2024-09-14T06:16:52.131 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3217000000000001e-05 2024-09-14T06:16:52.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.132 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.132 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.132 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654583+0000", 2024-09-14T06:16:52.132 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3660000000000003e-06 2024-09-14T06:16:52.132 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.132 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.132 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.132 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693055+0000", 2024-09-14T06:16:52.132 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038471752999999997 2024-09-14T06:16:52.132 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.132 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.132 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.132 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693152+0000", 2024-09-14T06:16:52.133 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7548999999999997e-05 2024-09-14T06:16:52.133 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.133 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.133 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.133 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693284+0000", 2024-09-14T06:16:52.133 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000132089 2024-09-14T06:16:52.133 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.133 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.133 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.133 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.133 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.133 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16126 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 5439488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.133 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654595+0000", 2024-09-14T06:16:52.134 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.420870820999999, 2024-09-14T06:16:52.136 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038961922000000003, 2024-09-14T06:16:52.136 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.136 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.136 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.136 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.136 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.136 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.136 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16126 2024-09-14T06:16:52.137 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.137 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.137 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.137 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.137 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654595+0000", 2024-09-14T06:16:52.137 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.137 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.137 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.137 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.137 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654595+0000", 2024-09-14T06:16:52.137 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.137 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.137 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.138 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.138 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654596+0000", 2024-09-14T06:16:52.138 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7700000000000004e-07 2024-09-14T06:16:52.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.138 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.138 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.138 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654600+0000", 2024-09-14T06:16:52.138 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.216e-06 2024-09-14T06:16:52.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.138 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.138 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.138 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654610+0000", 2024-09-14T06:16:52.138 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7070000000000004e-06 2024-09-14T06:16:52.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.139 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.139 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.139 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654617+0000", 2024-09-14T06:16:52.139 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2200000000000003e-06 2024-09-14T06:16:52.139 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.139 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.139 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.139 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693327+0000", 2024-09-14T06:16:52.139 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038709427999999997 2024-09-14T06:16:52.139 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.139 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.139 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.139 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693426+0000", 2024-09-14T06:16:52.140 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9401999999999995e-05 2024-09-14T06:16:52.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.140 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.140 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.140 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693557+0000", 2024-09-14T06:16:52.140 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013117200000000001 2024-09-14T06:16:52.140 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.140 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.140 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.140 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.140 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16127 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 5898240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.140 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654627+0000", 2024-09-14T06:16:52.140 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.420839387999999, 2024-09-14T06:16:52.141 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.040234709, 2024-09-14T06:16:52.141 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.141 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.141 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.141 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.141 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.141 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.141 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16127 2024-09-14T06:16:52.141 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.141 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.141 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.141 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.141 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654627+0000", 2024-09-14T06:16:52.141 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.141 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.142 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.142 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.142 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654627+0000", 2024-09-14T06:16:52.142 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.142 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.142 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.142 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.142 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654627+0000", 2024-09-14T06:16:52.142 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4700000000000001e-07 2024-09-14T06:16:52.142 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.142 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.142 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.142 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654630+0000", 2024-09-14T06:16:52.142 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8600000000000001e-06 2024-09-14T06:16:52.142 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.142 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.143 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.143 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654639+0000", 2024-09-14T06:16:52.143 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2599999999999994e-06 2024-09-14T06:16:52.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.143 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.143 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.143 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654647+0000", 2024-09-14T06:16:52.143 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2269999999999996e-06 2024-09-14T06:16:52.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.143 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.143 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.143 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693851+0000", 2024-09-14T06:16:52.143 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.039203937000000001 2024-09-14T06:16:52.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.143 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.144 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.144 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.694465+0000", 2024-09-14T06:16:52.144 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00061404799999999996 2024-09-14T06:16:52.144 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.144 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.144 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.144 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.694861+0000", 2024-09-14T06:16:52.144 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00039662999999999998 2024-09-14T06:16:52.144 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.144 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.144 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.144 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.144 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.144 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16128 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.144 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654656+0000", 2024-09-14T06:16:52.144 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.420809925, 2024-09-14T06:16:52.145 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.040797799000000003, 2024-09-14T06:16:52.145 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.145 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.145 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.145 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.145 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.145 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.145 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16128 2024-09-14T06:16:52.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.145 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.145 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.145 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.145 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654656+0000", 2024-09-14T06:16:52.145 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.145 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.146 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.146 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654656+0000", 2024-09-14T06:16:52.146 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.146 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.146 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.146 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.146 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654657+0000", 2024-09-14T06:16:52.146 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2699999999999999e-07 2024-09-14T06:16:52.146 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.146 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.146 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.146 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654659+0000", 2024-09-14T06:16:52.146 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6400000000000001e-06 2024-09-14T06:16:52.146 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.146 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.147 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.147 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654669+0000", 2024-09-14T06:16:52.147 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1730000000000001e-06 2024-09-14T06:16:52.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.147 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.147 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.147 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654685+0000", 2024-09-14T06:16:52.147 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6237000000000001e-05 2024-09-14T06:16:52.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.147 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.147 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.147 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.694952+0000", 2024-09-14T06:16:52.147 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.040267078999999997 2024-09-14T06:16:52.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.147 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.147 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.148 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695174+0000", 2024-09-14T06:16:52.148 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000221745 2024-09-14T06:16:52.148 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.148 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.148 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.148 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695454+0000", 2024-09-14T06:16:52.148 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000280198 2024-09-14T06:16:52.148 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.148 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.148 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.148 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.148 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.148 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16129 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.148 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654697+0000", 2024-09-14T06:16:52.148 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.420768829, 2024-09-14T06:16:52.149 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.041164486, 2024-09-14T06:16:52.149 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.149 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.149 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.149 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.149 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.149 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.149 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16129 2024-09-14T06:16:52.149 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.149 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.149 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.149 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.149 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654697+0000", 2024-09-14T06:16:52.149 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.149 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654697+0000", 2024-09-14T06:16:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654698+0000", 2024-09-14T06:16:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:16:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654701+0000", 2024-09-14T06:16:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.756e-06 2024-09-14T06:16:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.150 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.151 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.151 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654710+0000", 2024-09-14T06:16:52.151 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7669999999999998e-06 2024-09-14T06:16:52.151 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.151 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.151 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.151 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654718+0000", 2024-09-14T06:16:52.151 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.097e-06 2024-09-14T06:16:52.151 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.151 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.151 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.151 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695552+0000", 2024-09-14T06:16:52.151 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.040834148000000001 2024-09-14T06:16:52.151 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.151 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.152 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.152 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695690+0000", 2024-09-14T06:16:52.152 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000138479 2024-09-14T06:16:52.152 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.152 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.152 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.152 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695862+0000", 2024-09-14T06:16:52.152 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017150499999999999 2024-09-14T06:16:52.152 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.152 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.152 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.152 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.152 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.152 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16130 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.152 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654727+0000", 2024-09-14T06:16:52.152 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.420739021999999, 2024-09-14T06:16:52.153 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.041755107999999999, 2024-09-14T06:16:52.153 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.153 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.153 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.153 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.153 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.153 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.153 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16130 2024-09-14T06:16:52.153 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.153 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.153 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.153 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.153 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654727+0000", 2024-09-14T06:16:52.153 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.153 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.153 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654727+0000", 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654728+0000", 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6300000000000004e-07 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654730+0000", 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7439999999999999e-06 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.154 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654742+0000", 2024-09-14T06:16:52.155 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1036000000000001e-05 2024-09-14T06:16:52.155 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.155 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.155 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.155 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654748+0000", 2024-09-14T06:16:52.155 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9199999999999998e-06 2024-09-14T06:16:52.155 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.155 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.155 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.155 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695969+0000", 2024-09-14T06:16:52.155 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.041220672 2024-09-14T06:16:52.155 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.155 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.155 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.155 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696192+0000", 2024-09-14T06:16:52.156 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022279799999999999 2024-09-14T06:16:52.156 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.156 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.156 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.156 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696482+0000", 2024-09-14T06:16:52.156 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00029017499999999999 2024-09-14T06:16:52.156 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.156 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.156 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.156 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.156 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.156 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16131 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.156 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654758+0000", 2024-09-14T06:16:52.156 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.420708182, 2024-09-14T06:16:52.156 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.042069361999999999, 2024-09-14T06:16:52.157 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.157 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.157 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.157 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.157 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.157 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.157 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16131 2024-09-14T06:16:52.157 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.157 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.157 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.157 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.157 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654758+0000", 2024-09-14T06:16:52.157 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.157 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.157 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.158 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.158 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654758+0000", 2024-09-14T06:16:52.158 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.158 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.158 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.158 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654759+0000", 2024-09-14T06:16:52.158 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3e-07 2024-09-14T06:16:52.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.158 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.158 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.158 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654761+0000", 2024-09-14T06:16:52.158 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7599999999999998e-06 2024-09-14T06:16:52.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.158 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.158 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.159 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654773+0000", 2024-09-14T06:16:52.159 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1216999999999999e-05 2024-09-14T06:16:52.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.159 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.159 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.159 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654780+0000", 2024-09-14T06:16:52.159 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3529999999999997e-06 2024-09-14T06:16:52.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.159 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.159 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.159 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696524+0000", 2024-09-14T06:16:52.159 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.041744594000000003 2024-09-14T06:16:52.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.160 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.160 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.160 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696652+0000", 2024-09-14T06:16:52.160 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000127866 2024-09-14T06:16:52.160 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.160 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.160 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.160 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696827+0000", 2024-09-14T06:16:52.160 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000174842 2024-09-14T06:16:52.160 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.160 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.160 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.160 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.160 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.161 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16132 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.161 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654840+0000", 2024-09-14T06:16:52.161 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.420625653, 2024-09-14T06:16:52.161 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.042362179999999999, 2024-09-14T06:16:52.161 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.161 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.161 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.161 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.161 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.161 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.161 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16132 2024-09-14T06:16:52.161 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.161 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.161 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.161 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.162 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654840+0000", 2024-09-14T06:16:52.162 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.162 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.162 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.162 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.162 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654840+0000", 2024-09-14T06:16:52.162 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.162 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.162 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.162 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.162 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654841+0000", 2024-09-14T06:16:52.162 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9300000000000006e-07 2024-09-14T06:16:52.162 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.162 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.163 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.163 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654845+0000", 2024-09-14T06:16:52.163 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.18e-06 2024-09-14T06:16:52.163 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.163 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.163 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.163 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654867+0000", 2024-09-14T06:16:52.163 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.27e-05 2024-09-14T06:16:52.163 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.163 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.163 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.163 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654892+0000", 2024-09-14T06:16:52.163 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4284e-05 2024-09-14T06:16:52.163 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.163 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.164 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.164 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696907+0000", 2024-09-14T06:16:52.164 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.042015957999999999 2024-09-14T06:16:52.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.164 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.164 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.164 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.697032+0000", 2024-09-14T06:16:52.164 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012415899999999999 2024-09-14T06:16:52.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.164 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.164 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.164 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.697203+0000", 2024-09-14T06:16:52.164 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017090599999999999 2024-09-14T06:16:52.164 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.165 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.165 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.165 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.165 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.165 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16181 3.0 3:067f695c:::rbd_data.1152fc09e7e6.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:52.165 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.757965+0000", 2024-09-14T06:16:52.165 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.317501329000001, 2024-09-14T06:16:52.165 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065959498000000005, 2024-09-14T06:16:52.165 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.165 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.165 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:52.165 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.165 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.165 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.166 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16181 2024-09-14T06:16:52.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.166 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.166 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.166 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.166 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.757965+0000", 2024-09-14T06:16:52.166 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.166 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.166 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.166 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.757965+0000", 2024-09-14T06:16:52.166 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.166 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.166 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.167 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.757967+0000", 2024-09-14T06:16:52.167 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6869999999999999e-06 2024-09-14T06:16:52.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.167 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.167 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.167 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.757984+0000", 2024-09-14T06:16:52.167 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.658e-05 2024-09-14T06:16:52.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.167 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.167 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.167 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.758025+0000", 2024-09-14T06:16:52.167 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1158999999999997e-05 2024-09-14T06:16:52.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.167 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.167 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.168 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.758049+0000", 2024-09-14T06:16:52.168 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3442999999999999e-05 2024-09-14T06:16:52.168 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.168 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.168 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.168 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.758200+0000", 2024-09-14T06:16:52.168 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015118900000000001 2024-09-14T06:16:52.168 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.168 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.168 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.168 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.758340+0000", 2024-09-14T06:16:52.168 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000140722 2024-09-14T06:16:52.168 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.168 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.168 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 7", 2024-09-14T06:16:52.168 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.769990+0000", 2024-09-14T06:16:52.169 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011649286999999999 2024-09-14T06:16:52.169 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.169 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.169 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:52.169 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.823608+0000", 2024-09-14T06:16:52.169 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.053617797000000002 2024-09-14T06:16:52.169 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.169 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.169 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:52.169 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.823740+0000", 2024-09-14T06:16:52.169 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000132132 2024-09-14T06:16:52.169 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.169 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.169 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:52.169 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.823843+0000", 2024-09-14T06:16:52.170 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000103712 2024-09-14T06:16:52.170 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.170 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.170 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.170 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.823924+0000", 2024-09-14T06:16:52.170 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0790000000000004e-05 2024-09-14T06:16:52.170 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.170 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.170 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.170 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.170 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.170 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:18155 3.17 3:e91637d8:::rbd_data.115232f4ac2b.0000000000000000:a [sparse-read 5767168~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T06:16:52.170 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:41.255599+0000", 2024-09-14T06:16:52.170 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.819867257, 2024-09-14T06:16:52.171 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035557431, 2024-09-14T06:16:52.171 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.171 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.171 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.171 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.171 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.171 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.171 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 18155 2024-09-14T06:16:52.171 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.171 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.171 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.171 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.171 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255599+0000", 2024-09-14T06:16:52.171 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.171 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.171 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.172 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.172 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255599+0000", 2024-09-14T06:16:52.172 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.172 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.172 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.172 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.172 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255600+0000", 2024-09-14T06:16:52.172 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.37e-06 2024-09-14T06:16:52.172 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.172 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.172 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.172 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255624+0000", 2024-09-14T06:16:52.172 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3938e-05 2024-09-14T06:16:52.172 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.173 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.173 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.173 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255679+0000", 2024-09-14T06:16:52.173 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4768000000000001e-05 2024-09-14T06:16:52.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.173 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.173 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.173 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255699+0000", 2024-09-14T06:16:52.173 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0058e-05 2024-09-14T06:16:52.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.173 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.173 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.173 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.289431+0000", 2024-09-14T06:16:52.173 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.03373259 2024-09-14T06:16:52.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.174 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.174 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.174 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.289640+0000", 2024-09-14T06:16:52.174 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000208464 2024-09-14T06:16:52.174 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.174 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.174 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.174 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.291156+0000", 2024-09-14T06:16:52.174 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001516243 2024-09-14T06:16:52.174 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.174 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.174 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.174 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.175 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.175 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:18156 3.17 3:e91637d8:::rbd_data.115232f4ac2b.0000000000000000:a [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T06:16:52.175 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:41.255741+0000", 2024-09-14T06:16:52.175 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 10.819725482999999, 2024-09-14T06:16:52.175 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035889028000000003, 2024-09-14T06:16:52.175 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:52.175 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:52.175 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:52.175 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:52.175 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:52.175 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:52.175 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 18156 2024-09-14T06:16:52.175 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.175 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:52.176 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.176 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:52.176 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255741+0000", 2024-09-14T06:16:52.176 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.176 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.176 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.176 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:52.176 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255741+0000", 2024-09-14T06:16:52.176 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:52.176 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.176 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.176 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:52.176 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255742+0000", 2024-09-14T06:16:52.176 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3659999999999999e-06 2024-09-14T06:16:52.177 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.177 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.177 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:52.177 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255760+0000", 2024-09-14T06:16:52.177 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8031999999999999e-05 2024-09-14T06:16:52.177 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.177 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.177 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:52.177 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255784+0000", 2024-09-14T06:16:52.177 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4454999999999999e-05 2024-09-14T06:16:52.177 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.177 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.177 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:52.178 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255803+0000", 2024-09-14T06:16:52.178 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8541000000000001e-05 2024-09-14T06:16:52.178 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.178 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.178 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:52.178 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.291231+0000", 2024-09-14T06:16:52.178 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035428098999999998 2024-09-14T06:16:52.178 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.178 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.178 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:52.178 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.291420+0000", 2024-09-14T06:16:52.178 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000188786 2024-09-14T06:16:52.178 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:52.178 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:52.178 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:52.178 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.291630+0000", 2024-09-14T06:16:52.179 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00020974899999999999 2024-09-14T06:16:52.179 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.179 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.179 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.179 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:52.179 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:52.179 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:52.181 INFO:teuthology.orchestra.run.smithi039.stdout:2219 read 0xcb15b2 thru 0xcb1cbd (0x70c bytes) 2024-09-14T06:16:52.181 INFO:teuthology.orchestra.run.smithi039.stdout:2220 read 0x8fa2f4 thru 0x90476c (0xa479 bytes) 2024-09-14T06:16:52.181 INFO:teuthology.orchestra.run.smithi039.stdout:2222 read 0x474e2b thru 0x47f2d0 (0xa4a6 bytes) 2024-09-14T06:16:52.181 INFO:teuthology.orchestra.run.smithi039.stdout:2223 read 0x20f8110 thru 0x20f8b46 (0xa37 bytes) 2024-09-14T06:16:52.181 INFO:teuthology.orchestra.run.smithi039.stdout:2225 write 0x5a8c01 thru 0x5ae1c8 (0x55c8 bytes) 2024-09-14T06:16:52.182 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:52.182 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:16:52.182 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:52.178+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:52.183 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:52.183 INFO:teuthology.orchestra.run.smithi039.stdout:2226 read 0x20e6458 thru 0x20efed3 (0x9a7c bytes) 2024-09-14T06:16:52.183 INFO:teuthology.orchestra.run.smithi039.stdout:2227 punch from 0x2339840 to 0x233f672, (0x5e32 bytes) 2024-09-14T06:16:52.183 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:52.183 INFO:teuthology.orchestra.run.smithi039.stdout:2229 writesame 0xde4f855 thru 0xde53011 data_size 0x1(0x37bd bytes) 2024-09-14T06:16:52.183 INFO:teuthology.orchestra.run.smithi039.stdout:2230 writesame 0x3c41871 thru 0x3c4fe31 data_size 0x1(0xe5c1 bytes) 2024-09-14T06:16:52.184 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:52.184 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:52.184 INFO:teuthology.orchestra.run.smithi039.stdout:2234 punch from 0x6554afe to 0x65623e0, (0xd8e2 bytes) 2024-09-14T06:16:52.184 INFO:teuthology.orchestra.run.smithi039.stdout:2235 write 0xb08fb4 thru 0xb172b1 (0xe2fe bytes) 2024-09-14T06:16:52.184 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:52.185 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:52.185 INFO:teuthology.orchestra.run.smithi039.stdout:2236 trunc from 0xde53012 to 0x6f6584c 2024-09-14T06:16:52.186 INFO:teuthology.orchestra.run.smithi039.stdout:2238 read 0x49e8a65 thru 0x49efa84 (0x7020 bytes) 2024-09-14T06:16:52.186 INFO:teuthology.orchestra.run.smithi039.stdout:2239 read 0x16060da thru 0x1615c08 (0xfb2f bytes) 2024-09-14T06:16:52.186 INFO:teuthology.orchestra.run.smithi039.stdout:2241 read 0x2c5e84f thru 0x2c5f1b1 (0x963 bytes) 2024-09-14T06:16:52.186 INFO:teuthology.orchestra.run.smithi039.stdout:2242 write 0x1ebf44e thru 0x1ec7d49 (0x88fc bytes) 2024-09-14T06:16:52.186 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:52.186 INFO:teuthology.orchestra.run.smithi039.stdout:2245 read 0x5e4f552 thru 0x5e5370f (0x41be bytes) 2024-09-14T06:16:52.186 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:52.186 INFO:teuthology.orchestra.run.smithi039.stdout:2247 read 0x6c81c2c thru 0x6c8abe4 (0x8fb9 bytes) 2024-09-14T06:16:52.186 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:52.186 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:52.186 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:52.187 INFO:teuthology.orchestra.run.smithi039.stdout:2249 trunc from 0x6f6584c to 0xc101257 2024-09-14T06:16:52.187 INFO:teuthology.orchestra.run.smithi039.stdout:2250 write 0xdd8b65 thru 0xdda87d (0x1d19 bytes) 2024-09-14T06:16:52.187 INFO:teuthology.orchestra.run.smithi039.stdout:2251 clone 14 order 20 su 32768 sc 13 2024-09-14T06:16:52.187 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:52.187 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:52.210 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-14T06:16:52.299 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:52.299+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:52.346 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:52.346 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:52.360 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:16:52.382 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:52.379+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:52.477 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:52.478 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:52.483 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:52.483+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:52.493 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:16:52.554 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:16:52.599 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:52.599+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:52.621 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:52.621 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:52.636 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:16:52.671 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone13 from 0xc101257 (overlap 0x2698769) to 0xb94239 2024-09-14T06:16:52.685 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:52.682+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:52.743 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-14T06:16:52.768 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:52.768 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:52.782 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:16:52.786 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:52.783+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:52.887 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:52.887+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:52.954 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:52.955 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:52.969 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:16:52.988 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:52.987+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:53.089 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:53.086+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:53.090 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:53.091 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:53.095 INFO:teuthology.orchestra.run.smithi039.stdout:2252 read 0x302f20 thru 0x308b4e (0x5c2f bytes) 2024-09-14T06:16:53.103 INFO:teuthology.orchestra.run.smithi039.stdout:2253 write 0x943e70c thru 0x94477a7 (0x909c bytes) 2024-09-14T06:16:53.106 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:16:53.110 INFO:teuthology.orchestra.run.smithi039.stdout:2255 read 0x1d77591 thru 0x1d84823 (0xd293 bytes) 2024-09-14T06:16:53.120 INFO:teuthology.orchestra.run.smithi039.stdout:2256 read 0x7d21ab1 thru 0x7d2a116 (0x8666 bytes) 2024-09-14T06:16:53.120 INFO:teuthology.orchestra.run.smithi039.stdout:2257 writesame 0xb244b5b thru 0xb247660 data_size 0x1(0x2b06 bytes) 2024-09-14T06:16:53.142 INFO:teuthology.orchestra.run.smithi039.stdout:2260 trunc from 0xc101257 to 0x80bd9a7 2024-09-14T06:16:53.169 INFO:teuthology.orchestra.run.smithi039.stdout:2261 read 0x80a707a thru 0x80b007a (0x9001 bytes) 2024-09-14T06:16:53.176 INFO:teuthology.orchestra.run.smithi039.stdout:2264 trunc from 0x80bd9a7 to 0x4486065 2024-09-14T06:16:53.190 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:53.190+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:53.257 INFO:teuthology.orchestra.run.smithi039.stdout:2265 writesame 0x86eae42 thru 0x86f09f6 data_size 0x1(0x5bb5 bytes) 2024-09-14T06:16:53.258 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:53.259 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:53.266 INFO:teuthology.orchestra.run.smithi039.stdout:2266 writesame 0xaaa6efc thru 0xaaaac94 data_size 0x1(0x3d99 bytes) 2024-09-14T06:16:53.266 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:53.276 INFO:teuthology.orchestra.run.smithi039.stdout:2270 trunc from 0xaaaac95 to 0x3247af5 2024-09-14T06:16:53.278 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:16:53.319 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:53.319+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:53.385 INFO:teuthology.orchestra.run.smithi039.stdout:2271 write 0xeb3ebd8 thru 0xeb41875 (0x2c9e bytes) 2024-09-14T06:16:53.391 INFO:teuthology.orchestra.run.smithi039.stdout:2272 punch from 0xdf173ba to 0xdf19db6, (0x29fc bytes) 2024-09-14T06:16:53.391 INFO:teuthology.orchestra.run.smithi039.stdout:2273 write 0xa36086e thru 0xa369975 (0x9108 bytes) 2024-09-14T06:16:53.391 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:53.390+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:53.396 INFO:teuthology.orchestra.run.smithi039.stdout:2274 read 0x2277b82 thru 0x227c1ec (0x466b bytes) 2024-09-14T06:16:53.397 INFO:teuthology.orchestra.run.smithi039.stdout:2275 write 0xc20f744 thru 0xc2139d8 (0x4295 bytes) 2024-09-14T06:16:53.404 INFO:teuthology.orchestra.run.smithi039.stdout:2276 write 0x15e3700 thru 0x15effab (0xc8ac bytes) 2024-09-14T06:16:53.426 INFO:teuthology.orchestra.run.smithi039.stdout:2277 write 0x5c6a3d4 thru 0x5c71046 (0x6c73 bytes) 2024-09-14T06:16:53.430 INFO:teuthology.orchestra.run.smithi039.stdout:2279 read 0x3261386 thru 0x3262a8f (0x170a bytes) 2024-09-14T06:16:53.431 INFO:teuthology.orchestra.run.smithi039.stdout:2280 read 0x4c76e71 thru 0x4c80266 (0x93f6 bytes) 2024-09-14T06:16:53.438 INFO:teuthology.orchestra.run.smithi039.stdout:2281 trunc from 0xeb41876 to 0x83ea04d 2024-09-14T06:16:53.451 INFO:teuthology.orchestra.run.smithi039.stdout:2282 write 0x196a932 thru 0x196e5d0 (0x3c9f bytes) 2024-09-14T06:16:53.458 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:53.458 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:53.461 INFO:teuthology.orchestra.run.smithi039.stdout:2283 read 0x4103abf thru 0x41128af (0xedf1 bytes) 2024-09-14T06:16:53.473 INFO:teuthology.orchestra.run.smithi039.stdout:2285 trunc from 0x83ea04d to 0x193b59 2024-09-14T06:16:53.478 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:16:53.523 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:53.523+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:53.564 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-09-14T06:16:53.580 INFO:teuthology.orchestra.run.smithi039.stdout:2286 punch from 0x184e91 to 0x18d13b, (0x82aa bytes) 2024-09-14T06:16:53.580 INFO:teuthology.orchestra.run.smithi039.stdout:2287 punch from 0x11c9fb to 0x1264fc, (0x9b01 bytes) 2024-09-14T06:16:53.580 INFO:teuthology.orchestra.run.smithi039.stdout:2289 read 0x71e4 thru 0x11554 (0xa371 bytes) 2024-09-14T06:16:53.584 INFO:teuthology.orchestra.run.smithi039.stdout:2290 read 0x108864 thru 0x10f6ec (0x6e89 bytes) 2024-09-14T06:16:53.585 INFO:teuthology.orchestra.run.smithi039.stdout:2292 read 0x2ea0b thru 0x364ed (0x7ae3 bytes) 2024-09-14T06:16:53.589 INFO:teuthology.orchestra.run.smithi039.stdout:2293 punch from 0xb75fe to 0xb9b2e, (0x2530 bytes) 2024-09-14T06:16:53.590 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:53.590 INFO:teuthology.orchestra.run.smithi039.stdout:2297 read 0x9c95f thru 0xa8217 (0xb8b9 bytes) 2024-09-14T06:16:53.590 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:53.594 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:53.590+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:53.601 INFO:teuthology.orchestra.run.smithi039.stdout:2299 write 0x46373c8 thru 0x4644bc3 (0xd7fc bytes) 2024-09-14T06:16:53.608 INFO:teuthology.orchestra.run.smithi039.stdout:2301 trunc from 0x4644bc4 to 0x4be4c1b 2024-09-14T06:16:53.613 INFO:teuthology.orchestra.run.smithi039.stdout:2302 punch from 0x3330d0c to 0x3337fc1, (0x72b5 bytes) 2024-09-14T06:16:53.614 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:53.614 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:53.614 INFO:teuthology.orchestra.run.smithi039.stdout:2307 read 0x35c5bec thru 0x35c6b55 (0xf6a bytes) 2024-09-14T06:16:53.622 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:53.622 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:53.625 INFO:teuthology.orchestra.run.smithi039.stdout:2308 trunc from 0x4be4c1b to 0xe8f9c97 2024-09-14T06:16:53.630 INFO:teuthology.orchestra.run.smithi039.stdout:2309 read 0x77dcf91 thru 0x77e0f0b (0x3f7b bytes) 2024-09-14T06:16:53.631 INFO:teuthology.orchestra.run.smithi039.stdout:2310 write 0x833a5f1 thru 0x83462bf (0xbccf bytes) 2024-09-14T06:16:53.635 INFO:teuthology.orchestra.run.smithi039.stdout:2311 write 0xdfbd636 thru 0xdfc698a (0x9355 bytes) 2024-09-14T06:16:53.641 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:16:53.641 INFO:teuthology.orchestra.run.smithi039.stdout:2312 writesame 0x64f90ec thru 0x65077fa data_size 0x1(0xe70f bytes) 2024-09-14T06:16:53.660 INFO:teuthology.orchestra.run.smithi039.stdout:2313 punch from 0x503e4eb to 0x5040b14, (0x2629 bytes) 2024-09-14T06:16:53.660 INFO:teuthology.orchestra.run.smithi039.stdout:2314 read 0xd28f074 thru 0xd29032a (0x12b7 bytes) 2024-09-14T06:16:53.660 INFO:teuthology.orchestra.run.smithi039.stdout:2315 read 0xe0668fb thru 0xe0686b8 (0x1dbe bytes) 2024-09-14T06:16:53.660 INFO:teuthology.orchestra.run.smithi039.stdout:2316 punch from 0x864561c to 0x864dd54, (0x8738 bytes) 2024-09-14T06:16:53.660 INFO:teuthology.orchestra.run.smithi039.stdout:2317 read 0x63fea6f thru 0x63ff691 (0xc23 bytes) 2024-09-14T06:16:53.660 INFO:teuthology.orchestra.run.smithi039.stdout:2319 read 0x222c668 thru 0x222eed1 (0x286a bytes) 2024-09-14T06:16:53.660 INFO:teuthology.orchestra.run.smithi039.stdout:2321 punch from 0x6dbd157 to 0x6dc45d7, (0x7480 bytes) 2024-09-14T06:16:53.660 INFO:teuthology.orchestra.run.smithi039.stdout:2322 write 0xaca9949 thru 0xacb2e77 (0x952f bytes) 2024-09-14T06:16:53.666 INFO:teuthology.orchestra.run.smithi039.stdout:2323 write 0x8b6495c thru 0x8b6cae8 (0x818d bytes) 2024-09-14T06:16:53.671 INFO:teuthology.orchestra.run.smithi039.stdout:2324 write 0xbb00ac7 thru 0xbb05f4e (0x5488 bytes) 2024-09-14T06:16:53.679 INFO:teuthology.orchestra.run.smithi039.stdout:2325 trunc from 0xe8f9c97 to 0xc0ff8bf 2024-09-14T06:16:53.694 INFO:teuthology.orchestra.run.smithi039.stdout:2326 write 0x16c43b0 thru 0x16c795a (0x35ab bytes) 2024-09-14T06:16:53.694 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:53.694+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:53.697 INFO:teuthology.orchestra.run.smithi039.stdout:2332 writesame 0x15126b5 thru 0x1519bac data_size 0x4c(0x74f8 bytes) 2024-09-14T06:16:53.702 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:53.703 INFO:teuthology.orchestra.run.smithi039.stdout:2335 punch from 0x9bfb538 to 0x9c02502, (0x6fca bytes) 2024-09-14T06:16:53.703 INFO:teuthology.orchestra.run.smithi039.stdout:2336 read 0xbd95b7d thru 0xbd9a25b (0x46df bytes) 2024-09-14T06:16:53.703 INFO:teuthology.orchestra.run.smithi039.stdout:2337 writesame 0xb844a8e thru 0xb846c63 data_size 0x2(0x21d6 bytes) 2024-09-14T06:16:53.710 INFO:teuthology.orchestra.run.smithi039.stdout:2338 write 0x58efefa thru 0x58f1411 (0x1518 bytes) 2024-09-14T06:16:53.715 INFO:teuthology.orchestra.run.smithi039.stdout:2339 read 0x61b5812 thru 0x61bf33a (0x9b29 bytes) 2024-09-14T06:16:53.716 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:53.716 INFO:teuthology.orchestra.run.smithi039.stdout:2341 writesame 0xf29628 thru 0xf32d9e data_size 0x5(0x9777 bytes) 2024-09-14T06:16:53.726 INFO:teuthology.orchestra.run.smithi039.stdout:2342 trunc from 0xc0ff8bf to 0xed05fd4 2024-09-14T06:16:53.731 INFO:teuthology.orchestra.run.smithi039.stdout:2343 read 0xa8a6b67 thru 0xa8b2830 (0xbcca bytes) 2024-09-14T06:16:53.732 INFO:teuthology.orchestra.run.smithi039.stdout:2344 write 0xcf2480f thru 0xcf328e0 (0xe0d2 bytes) 2024-09-14T06:16:53.739 INFO:teuthology.orchestra.run.smithi039.stdout:2346 punch from 0xb12b2af to 0xb12d082, (0x1dd3 bytes) 2024-09-14T06:16:53.739 INFO:teuthology.orchestra.run.smithi039.stdout:2347 read 0x6da22ff thru 0x6da419e (0x1ea0 bytes) 2024-09-14T06:16:53.758 INFO:teuthology.orchestra.run.smithi039.stdout:2349 trunc from 0xed05fd4 to 0xea101a 2024-09-14T06:16:53.791 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:53.791 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:53.796 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:53.794+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:53.796 INFO:teuthology.orchestra.run.smithi039.stdout:2350 writesame 0xce988a6 thru 0xcea7770 data_size 0x15(0xeecb bytes) 2024-09-14T06:16:53.805 INFO:teuthology.orchestra.run.smithi039.stdout:2351 write 0xe8550a3 thru 0xe8638b0 (0xe80e bytes) 2024-09-14T06:16:53.808 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:16:53.812 INFO:teuthology.orchestra.run.smithi039.stdout:2352 read 0x2f9cce thru 0x2fdb0f (0x3e42 bytes) 2024-09-14T06:16:53.812 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:53.813 INFO:teuthology.orchestra.run.smithi039.stdout:2355 writesame 0x6796d3f thru 0x67a3d27 data_size 0x5(0xcfe9 bytes) 2024-09-14T06:16:53.823 INFO:teuthology.orchestra.run.smithi039.stdout:2356 read 0x2750245 thru 0x275c7f4 (0xc5b0 bytes) 2024-09-14T06:16:53.823 INFO:teuthology.orchestra.run.smithi039.stdout:2359 write 0x7cd0ef2 thru 0x7cd670a (0x5819 bytes) 2024-09-14T06:16:53.827 INFO:teuthology.orchestra.run.smithi039.stdout:2360 writesame 0x16b1e6e thru 0x16bec26 data_size 0x3(0xcdb9 bytes) 2024-09-14T06:16:53.833 INFO:teuthology.orchestra.run.smithi039.stdout:2361 punch from 0x5aa4ef4 to 0x5aac970, (0x7a7c bytes) 2024-09-14T06:16:53.833 INFO:teuthology.orchestra.run.smithi039.stdout:2362 write 0x7c7c8bc thru 0x7c85bb8 (0x92fd bytes) 2024-09-14T06:16:53.837 INFO:teuthology.orchestra.run.smithi039.stdout:2364 read 0x79ae7f2 thru 0x79b283e (0x404d bytes) 2024-09-14T06:16:53.837 INFO:teuthology.orchestra.run.smithi039.stdout:2368 write 0x2507718 thru 0x25141b9 (0xcaa2 bytes) 2024-09-14T06:16:53.845 INFO:teuthology.orchestra.run.smithi039.stdout:2369 punch from 0xc13cd06 to 0xc1411c0, (0x44ba bytes) 2024-09-14T06:16:53.845 INFO:teuthology.orchestra.run.smithi039.stdout:2373 writesame 0x30dd26b thru 0x30e13c5 data_size 0x27(0x415b bytes) 2024-09-14T06:16:53.850 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:53.850 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:53.850 INFO:teuthology.orchestra.run.smithi039.stdout:2376 write 0xe5fd68b thru 0xe602070 (0x49e6 bytes) 2024-09-14T06:16:53.856 INFO:teuthology.orchestra.run.smithi039.stdout:2378 writesame 0x1b9e028 thru 0x1b9f9d7 data_size 0x18(0x19b0 bytes) 2024-09-14T06:16:53.859 INFO:teuthology.orchestra.run.smithi039.stdout:2380 punch from 0x14836e9 to 0x148e04a, (0xa961 bytes) 2024-09-14T06:16:53.859 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:53.860 INFO:teuthology.orchestra.run.smithi039.stdout:2383 trunc from 0xe8638b1 to 0xe79c980 2024-09-14T06:16:53.869 INFO:teuthology.orchestra.run.smithi039.stdout:2385 write 0xe5597d thru 0xe5ecb3 (0x9337 bytes) 2024-09-14T06:16:53.875 INFO:teuthology.orchestra.run.smithi039.stdout:2386 punch from 0xe63b3ff to 0xe649e8c, (0xea8d bytes) 2024-09-14T06:16:53.875 INFO:teuthology.orchestra.run.smithi039.stdout:2388 read 0x571cc94 thru 0x572b6c8 (0xea35 bytes) 2024-09-14T06:16:53.875 INFO:teuthology.orchestra.run.smithi039.stdout:2389 read 0x57fc455 thru 0x57feb3a (0x26e6 bytes) 2024-09-14T06:16:53.875 INFO:teuthology.orchestra.run.smithi039.stdout:2390 punch from 0xd4ba96c to 0xd4c91a6, (0xe83a bytes) 2024-09-14T06:16:53.875 INFO:teuthology.orchestra.run.smithi039.stdout:2391 write 0x23d0f94 thru 0x23da6dd (0x974a bytes) 2024-09-14T06:16:53.877 INFO:teuthology.orchestra.run.smithi039.stdout:2393 writesame 0xa4fc431 thru 0xa5096c8 data_size 0x5c(0xd298 bytes) 2024-09-14T06:16:53.884 INFO:teuthology.orchestra.run.smithi039.stdout:2395 read 0xd4ccf45 thru 0xd4dc594 (0xf650 bytes) 2024-09-14T06:16:53.891 INFO:teuthology.orchestra.run.smithi039.stdout:2396 trunc from 0xe79c980 to 0x8404f6e 2024-09-14T06:16:53.896 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:53.894+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:53.906 INFO:teuthology.orchestra.run.smithi039.stdout:2397 write 0x64a5d3b thru 0x64b161c (0xb8e2 bytes) 2024-09-14T06:16:53.913 INFO:teuthology.orchestra.run.smithi039.stdout:2398 punch from 0x82ab5a7 to 0x82aee9b, (0x38f4 bytes) 2024-09-14T06:16:53.925 INFO:teuthology.orchestra.run.smithi039.stdout:2400 writesame 0xd11510b thru 0xd11547a data_size 0x14(0x370 bytes) 2024-09-14T06:16:53.934 INFO:teuthology.orchestra.run.smithi039.stdout:2401 trunc from 0xd11547b to 0x7485377 2024-09-14T06:16:53.947 INFO:teuthology.orchestra.run.smithi039.stdout:2402 trunc from 0x7485377 to 0x6938df1 2024-09-14T06:16:53.948 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:53.948 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:53.953 INFO:teuthology.orchestra.run.smithi039.stdout:2406 read 0x25a3d5c thru 0x25b1f98 (0xe23d bytes) 2024-09-14T06:16:53.953 INFO:teuthology.orchestra.run.smithi039.stdout:2407 read 0x20fc5cd thru 0x2100ba8 (0x45dc bytes) 2024-09-14T06:16:53.954 INFO:teuthology.orchestra.run.smithi039.stdout:2408 write 0x6168d4c thru 0x616c348 (0x35fd bytes) 2024-09-14T06:16:53.956 INFO:teuthology.orchestra.run.smithi039.stdout:2410 read 0x588dbac thru 0x58930bb (0x5510 bytes) 2024-09-14T06:16:53.956 INFO:teuthology.orchestra.run.smithi039.stdout:2411 punch from 0x4eae9cb to 0x4eb1272, (0x28a7 bytes) 2024-09-14T06:16:53.956 INFO:teuthology.orchestra.run.smithi039.stdout:2412 write 0x1fb9704 thru 0x1fc07d1 (0x70ce bytes) 2024-09-14T06:16:53.958 INFO:teuthology.orchestra.run.smithi039.stdout:2413 punch from 0xa61cd4 to 0xa7127d, (0xf5a9 bytes) 2024-09-14T06:16:53.958 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:53.958 INFO:teuthology.orchestra.run.smithi039.stdout:2415 read 0x195b978 thru 0x196b7d1 (0xfe5a bytes) 2024-09-14T06:16:53.958 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:53.958 INFO:teuthology.orchestra.run.smithi039.stdout:2417 write 0x63b74fb thru 0x63b923c (0x1d42 bytes) 2024-09-14T06:16:53.964 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:16:53.965 INFO:teuthology.orchestra.run.smithi039.stdout:2418 read 0x568630b thru 0x5695d9b (0xfa91 bytes) 2024-09-14T06:16:53.965 INFO:teuthology.orchestra.run.smithi039.stdout:2419 write 0x5dbdd0 thru 0x5de54c (0x277d bytes) 2024-09-14T06:16:53.966 INFO:teuthology.orchestra.run.smithi039.stdout:2420 read 0x65a05e2 thru 0x65a9062 (0x8a81 bytes) 2024-09-14T06:16:53.966 INFO:teuthology.orchestra.run.smithi039.stdout:2421 read 0x4336478 thru 0x4344fe5 (0xeb6e bytes) 2024-09-14T06:16:53.966 INFO:teuthology.orchestra.run.smithi039.stdout:2422 read 0x7ae4f9 thru 0x7b89aa (0xa4b2 bytes) 2024-09-14T06:16:53.974 INFO:teuthology.orchestra.run.smithi039.stdout:2423 trunc from 0x6938df1 to 0xcb950a4 2024-09-14T06:16:53.986 INFO:teuthology.orchestra.run.smithi039.stdout:2424 write 0xcdd1655 thru 0xcdd26fc (0x10a8 bytes) 2024-09-14T06:16:53.991 INFO:teuthology.orchestra.run.smithi039.stdout:2425 read 0xb0aa82a thru 0xb0addb5 (0x358c bytes) 2024-09-14T06:16:53.992 INFO:teuthology.orchestra.run.smithi039.stdout:2426 write 0x85abe2a thru 0x85b9ea5 (0xe07c bytes) 2024-09-14T06:16:53.997 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:53.995+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:53.997 INFO:teuthology.orchestra.run.smithi039.stdout:2427 write 0x778ea4d thru 0x77952f8 (0x68ac bytes) 2024-09-14T06:16:54.003 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:54.003 INFO:teuthology.orchestra.run.smithi039.stdout:2435 writesame 0x312eecc thru 0x3135029 data_size 0x67(0x615e bytes) 2024-09-14T06:16:54.014 INFO:teuthology.orchestra.run.smithi039.stdout:2436 write 0xd150843 thru 0xd15e52f (0xdced bytes) 2024-09-14T06:16:54.020 INFO:teuthology.orchestra.run.smithi039.stdout:2438 read 0x453cd0 thru 0x4542e7 (0x618 bytes) 2024-09-14T06:16:54.020 INFO:teuthology.orchestra.run.smithi039.stdout:2441 punch from 0xc01bedd to 0xc023b16, (0x7c39 bytes) 2024-09-14T06:16:54.021 INFO:teuthology.orchestra.run.smithi039.stdout:2442 read 0xae58b72 thru 0xae5a0e6 (0x1575 bytes) 2024-09-14T06:16:54.021 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:54.021 INFO:teuthology.orchestra.run.smithi039.stdout:2444 write 0x46b027f thru 0x46b31cd (0x2f4f bytes) 2024-09-14T06:16:54.024 INFO:teuthology.orchestra.run.smithi039.stdout:2445 write 0x3ebd257 thru 0x3ecc32b (0xf0d5 bytes) 2024-09-14T06:16:54.032 INFO:teuthology.orchestra.run.smithi039.stdout:2446 read 0x1e32240 thru 0x1e3803f (0x5e00 bytes) 2024-09-14T06:16:54.032 INFO:teuthology.orchestra.run.smithi039.stdout:2447 write 0x36ec423 thru 0x36fb6e7 (0xf2c5 bytes) 2024-09-14T06:16:54.039 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:54.039 INFO:teuthology.orchestra.run.smithi039.stdout:2450 read 0x8023e8e thru 0x8029090 (0x5203 bytes) 2024-09-14T06:16:54.039 INFO:teuthology.orchestra.run.smithi039.stdout:2451 writesame 0x459a4d7 thru 0x45a84b9 data_size 0x5(0xdfe3 bytes) 2024-09-14T06:16:54.050 INFO:teuthology.orchestra.run.smithi039.stdout:2452 punch from 0x443ae42 to 0x4445ff5, (0xb1b3 bytes) 2024-09-14T06:16:54.050 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:54.054 INFO:teuthology.orchestra.run.smithi039.stdout:2454 trunc from 0xd15e530 to 0x993b945 2024-09-14T06:16:54.067 INFO:teuthology.orchestra.run.smithi039.stdout:2455 read 0xd694b5 thru 0xd6cfc1 (0x3b0d bytes) 2024-09-14T06:16:54.071 INFO:teuthology.orchestra.run.smithi039.stdout:2457 trunc from 0x993b945 to 0xcd484fc 2024-09-14T06:16:54.078 INFO:teuthology.orchestra.run.smithi039.stdout:2458 write 0x178fe4e thru 0x179667b (0x682e bytes) 2024-09-14T06:16:54.134 INFO:teuthology.orchestra.run.smithi039.stdout:2466 writesame 0x2470408 thru 0x2473246 data_size 0x1(0x2e3f bytes) 2024-09-14T06:16:54.135 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:54.135 INFO:teuthology.orchestra.run.smithi039.stdout:2470 read 0xa8e9075 thru 0xa8f2c8a (0x9c16 bytes) 2024-09-14T06:16:54.135 INFO:teuthology.orchestra.run.smithi039.stdout:2471 read 0x59c5c94 thru 0x59c9ce9 (0x4056 bytes) 2024-09-14T06:16:54.135 INFO:teuthology.orchestra.run.smithi039.stdout:2472 write 0x915bcde thru 0x916b1bc (0xf4df bytes) 2024-09-14T06:16:54.135 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:54.135 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:54.136 INFO:teuthology.orchestra.run.smithi039.stdout:2473 read 0x679cfc1 thru 0x67a21a2 (0x51e2 bytes) 2024-09-14T06:16:54.136 INFO:teuthology.orchestra.run.smithi039.stdout:2474 punch from 0x131790e to 0x1324321, (0xca13 bytes) 2024-09-14T06:16:54.136 INFO:teuthology.orchestra.run.smithi039.stdout:2476 trunc from 0xcd484fc to 0xec62c25 2024-09-14T06:16:54.137 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:16:54.138 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:54.138 INFO:teuthology.orchestra.run.smithi039.stdout:2480 writesame 0x1a77f0 thru 0x1b163d data_size 0x17(0x9e4e bytes) 2024-09-14T06:16:54.138 INFO:teuthology.orchestra.run.smithi039.stdout:2481 write 0xac6588 thru 0xac6685 (0xfe bytes) 2024-09-14T06:16:54.138 INFO:teuthology.orchestra.run.smithi039.stdout:2482 write 0xe36f723 thru 0xe37e330 (0xec0e bytes) 2024-09-14T06:16:54.138 INFO:teuthology.orchestra.run.smithi039.stdout:2483 writesame 0x1f3dd51 thru 0x1f3fe60 data_size 0x2e(0x2110 bytes) 2024-09-14T06:16:54.138 INFO:teuthology.orchestra.run.smithi039.stdout:2484 read 0xdabd788 thru 0xdac0a51 (0x32ca bytes) 2024-09-14T06:16:54.138 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:54.134+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:54.153 INFO:teuthology.orchestra.run.smithi039.stdout:2485 trunc from 0xec62c25 to 0x1e0da11 2024-09-14T06:16:54.190 INFO:teuthology.orchestra.run.smithi039.stdout:2486 write 0x4e61a17 thru 0x4e63040 (0x162a bytes) 2024-09-14T06:16:54.200 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:16:54.213 INFO:teuthology.orchestra.run.smithi039.stdout:2487 write 0xe398424 thru 0xe3a7e84 (0xfa61 bytes) 2024-09-14T06:16:54.235 INFO:teuthology.orchestra.run.smithi039.stdout:2488 trunc from 0xe3a7e85 to 0x2184d04 2024-09-14T06:16:54.235 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:54.234+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:54.252 INFO:teuthology.orchestra.run.smithi039.stdout:2489 read 0x6a9d02 thru 0x6ab7da (0x1ad9 bytes) 2024-09-14T06:16:54.252 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:54.277 INFO:teuthology.orchestra.run.smithi039.stdout:2492 writesame 0xed156d3 thru 0xed178d6 data_size 0xe(0x2204 bytes) 2024-09-14T06:16:54.277 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:54.277 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:54.277 INFO:teuthology.orchestra.run.smithi039.stdout:2495 writesame 0xaa7b3a6 thru 0xaa8ac21 data_size 0xab(0xf87c bytes) 2024-09-14T06:16:54.298 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:54.299 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:54.299 INFO:teuthology.orchestra.run.smithi039.stdout:2496 trunc from 0xed178d7 to 0x36cd2b7 2024-09-14T06:16:54.315 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:54.317 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:16:54.317 INFO:teuthology.orchestra.run.smithi039.stdout:2499 trunc from 0x36cd2b7 to 0x2db7e5b 2024-09-14T06:16:54.337 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:54.334+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:54.338 INFO:teuthology.orchestra.run.smithi039.stdout:2500 writesame 0xa2a02f3 thru 0xa2a3ca6 data_size 0x6(0x39b4 bytes) 2024-09-14T06:16:54.338 INFO:teuthology.orchestra.run.smithi039.stdout:2501 write 0xe300a2 thru 0xe3d22a (0xd189 bytes) 2024-09-14T06:16:54.340 INFO:teuthology.orchestra.run.smithi039.stdout:2502 write 0x8b3f2d1 thru 0x8b3ff04 (0xc34 bytes) 2024-09-14T06:16:54.345 INFO:teuthology.orchestra.run.smithi039.stdout:2503 writesame 0x2595f1d thru 0x259cc0f data_size 0x9(0x6cf3 bytes) 2024-09-14T06:16:54.352 INFO:teuthology.orchestra.run.smithi039.stdout:2504 write 0x2d1d4bb thru 0x2d24e08 (0x794e bytes) 2024-09-14T06:16:54.361 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:54.361 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:54.370 INFO:teuthology.orchestra.run.smithi039.stdout:2505 trunc from 0xa2a3ca7 to 0x8c55eb 2024-09-14T06:16:54.383 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:54.384 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:54.403 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:54.404 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:54.412 INFO:teuthology.orchestra.run.smithi039.stdout:2506 trunc from 0x8c55eb to 0xe231650 2024-09-14T06:16:54.418 INFO:teuthology.orchestra.run.smithi039.stdout:2507 write 0x6f0b498 thru 0x6f126bc (0x7225 bytes) 2024-09-14T06:16:54.425 INFO:teuthology.orchestra.run.smithi039.stdout:2508 write 0x8a82c94 thru 0x8a904cd (0xd83a bytes) 2024-09-14T06:16:54.426 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:54.426 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:54.437 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:54.435+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:54.450 INFO:teuthology.orchestra.run.smithi039.stdout:2510 trunc from 0xe231650 to 0x19bf64f 2024-09-14T06:16:54.451 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:54.451 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:54.474 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:54.474 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:54.482 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:54.482 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:54.483 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:54.483 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:54.483 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:54.486 INFO:teuthology.orchestra.run.smithi039.stdout:2511 write 0xd02df6e thru 0xd0355b9 (0x764c bytes) 2024-09-14T06:16:54.498 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:54.499 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:54.499 INFO:teuthology.orchestra.run.smithi039.stdout:2512 trunc from 0xd0355ba to 0x8bf8890 2024-09-14T06:16:54.501 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:16:54.521 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:54.521 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:54.523 INFO:teuthology.orchestra.run.smithi039.stdout:2513 write 0xb868316 thru 0xb8695c3 (0x12ae bytes) 2024-09-14T06:16:54.528 INFO:teuthology.orchestra.run.smithi039.stdout:2514 write 0xb519e20 thru 0xb522c35 (0x8e16 bytes) 2024-09-14T06:16:54.538 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:54.538+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:54.545 INFO:teuthology.orchestra.run.smithi039.stdout:2515 writesame 0xdce59b8 thru 0xdce6c7e data_size 0x1(0x12c7 bytes) 2024-09-14T06:16:54.545 INFO:teuthology.orchestra.run.smithi039.stdout:2517 read 0x7afe35b thru 0x7b09e24 (0xbaca bytes) 2024-09-14T06:16:54.545 INFO:teuthology.orchestra.run.smithi039.stdout:2519 read 0x28bdf6a thru 0x28cb068 (0xd0ff bytes) 2024-09-14T06:16:54.545 INFO:teuthology.orchestra.run.smithi039.stdout:2520 write 0x3c7b95f thru 0x3c7cda2 (0x1444 bytes) 2024-09-14T06:16:54.549 INFO:teuthology.orchestra.run.smithi039.stdout:2522 write 0x336ce60 thru 0x337bd8d (0xef2e bytes) 2024-09-14T06:16:54.556 INFO:teuthology.orchestra.run.smithi039.stdout:2524 read 0x958e3bf thru 0x9594c5a (0x689c bytes) 2024-09-14T06:16:54.556 INFO:teuthology.orchestra.run.smithi039.stdout:2525 punch from 0x804b363 to 0x80540f2, (0x8d8f bytes) 2024-09-14T06:16:54.556 INFO:teuthology.orchestra.run.smithi039.stdout:2527 read 0x5512c17 thru 0x5513c48 (0x1032 bytes) 2024-09-14T06:16:54.556 INFO:teuthology.orchestra.run.smithi039.stdout:2529 read 0x8889d9d thru 0x888a2a9 (0x50d bytes) 2024-09-14T06:16:54.556 INFO:teuthology.orchestra.run.smithi039.stdout:2531 write 0x8bc0e28 thru 0x8bcae39 (0xa012 bytes) 2024-09-14T06:16:54.561 INFO:teuthology.orchestra.run.smithi039.stdout:2532 read 0xcd2a40b thru 0xcd36ad7 (0xc6cd bytes) 2024-09-14T06:16:54.561 INFO:teuthology.orchestra.run.smithi039.stdout:2533 punch from 0x3b49560 to 0x3b50832, (0x72d2 bytes) 2024-09-14T06:16:54.561 INFO:teuthology.orchestra.run.smithi039.stdout:2534 write 0x17f74f7 thru 0x17f9b4b (0x2655 bytes) 2024-09-14T06:16:54.566 INFO:teuthology.orchestra.run.smithi039.stdout:2535 write 0x825a182 thru 0x82675af (0xd42e bytes) 2024-09-14T06:16:54.572 INFO:teuthology.orchestra.run.smithi039.stdout:2536 writesame 0xb51d8b6 thru 0xb52d1c3 data_size 0x47(0xf90e bytes) 2024-09-14T06:16:54.577 INFO:teuthology.orchestra.run.smithi039.stdout:2538 read 0x66d9b59 thru 0x66df18f (0x5637 bytes) 2024-09-14T06:16:54.577 INFO:teuthology.orchestra.run.smithi039.stdout:2539 write 0x31186cd thru 0x3120837 (0x816b bytes) 2024-09-14T06:16:54.582 INFO:teuthology.orchestra.run.smithi039.stdout:2540 punch from 0x9722fda to 0x9732cf5, (0xfd1b bytes) 2024-09-14T06:16:54.582 INFO:teuthology.orchestra.run.smithi039.stdout:2543 punch from 0x73b878c to 0x73c0a4b, (0x82bf bytes) 2024-09-14T06:16:54.582 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:54.582 INFO:teuthology.orchestra.run.smithi039.stdout:2545 read 0xd9d1402 thru 0xd9e0111 (0xed10 bytes) 2024-09-14T06:16:54.583 INFO:teuthology.orchestra.run.smithi039.stdout:2546 trunc from 0xdce6c7f to 0xd2e6e1f 2024-09-14T06:16:54.589 INFO:teuthology.orchestra.run.smithi039.stdout:2548 write 0xd11418f thru 0xd11c3ee (0x8260 bytes) 2024-09-14T06:16:54.601 INFO:teuthology.orchestra.run.smithi039.stdout:2549 writesame 0xe0d05eb thru 0xe0d4c2f data_size 0x1(0x4645 bytes) 2024-09-14T06:16:54.601 INFO:teuthology.orchestra.run.smithi039.stdout:2550 write 0xa27982f thru 0xa288a5f (0xf231 bytes) 2024-09-14T06:16:54.608 INFO:teuthology.orchestra.run.smithi039.stdout:2552 read 0x9089c72 thru 0x90984cb (0xe85a bytes) 2024-09-14T06:16:54.609 INFO:teuthology.orchestra.run.smithi039.stdout:2553 writesame 0xc01ef4d thru 0xc027f00 data_size 0x22(0x8fb4 bytes) 2024-09-14T06:16:54.613 INFO:teuthology.orchestra.run.smithi039.stdout:2554 read 0x4627868 thru 0x463631c (0xeab5 bytes) 2024-09-14T06:16:54.619 INFO:teuthology.orchestra.run.smithi039.stdout:2555 trunc from 0xe0d4c30 to 0xa0c29a5 2024-09-14T06:16:54.636 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:54.636 INFO:teuthology.orchestra.run.smithi039.stdout:2558 punch from 0x44e9ec8 to 0x44eddba, (0x3ef2 bytes) 2024-09-14T06:16:54.639 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:54.638+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:54.644 INFO:teuthology.orchestra.run.smithi039.stdout:2559 writesame 0xadc2b27 thru 0xadcbaf4 data_size 0x2(0x8fce bytes) 2024-09-14T06:16:54.644 INFO:teuthology.orchestra.run.smithi039.stdout:2560 writesame 0x93b9b1c thru 0x93c9970 data_size 0x21(0xfe55 bytes) 2024-09-14T06:16:54.650 INFO:teuthology.orchestra.run.smithi039.stdout:2561 punch from 0x86072b to 0x86485b, (0x4130 bytes) 2024-09-14T06:16:54.650 INFO:teuthology.orchestra.run.smithi039.stdout:2563 write 0xaa4b8ac thru 0xaa50c6a (0x53bf bytes) 2024-09-14T06:16:54.655 INFO:teuthology.orchestra.run.smithi039.stdout:2564 read 0x388e438 thru 0x38939a7 (0x5570 bytes) 2024-09-14T06:16:54.655 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:54.656 INFO:teuthology.orchestra.run.smithi039.stdout:2568 read 0x876cd9d thru 0x87794ec (0xc750 bytes) 2024-09-14T06:16:54.656 INFO:teuthology.orchestra.run.smithi039.stdout:2569 read 0x7029246 thru 0x702be77 (0x2c32 bytes) 2024-09-14T06:16:54.656 INFO:teuthology.orchestra.run.smithi039.stdout:2570 punch from 0x4d18959 to 0x4d1a076, (0x171d bytes) 2024-09-14T06:16:54.659 INFO:teuthology.orchestra.run.smithi039.stdout:2571 trunc from 0xadcbaf5 to 0xd9ac71e 2024-09-14T06:16:54.665 INFO:teuthology.orchestra.run.smithi039.stdout:2573 write 0x80675b0 thru 0x8074c68 (0xd6b9 bytes) 2024-09-14T06:16:54.669 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:54.670 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:54.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:54.670 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:54.670 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.670 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.5 30/29 [PushOp(3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head, version: 28'5, data_included: [0~212992,360448~311296], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head@28'5, size: 1032192, copy_subset: [(0, 1032192)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1032192, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:16:54.670 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:54.670 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.392300194000001, 2024-09-14T06:16:54.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041630249999999999, 2024-09-14T06:16:54.670 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.670 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.670 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:54.670 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.670 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:54.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:54.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275888+0000", 2024-09-14T06:16:54.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.367e-06 2024-09-14T06:16:54.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.672 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.672 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277098+0000", 2024-09-14T06:16:54.672 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001210661 2024-09-14T06:16:54.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.672 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.672 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277122+0000", 2024-09-14T06:16:54.672 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3413000000000001e-05 2024-09-14T06:16:54.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.672 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.672 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277141+0000", 2024-09-14T06:16:54.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9133000000000001e-05 2024-09-14T06:16:54.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277364+0000", 2024-09-14T06:16:54.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000223365 2024-09-14T06:16:54.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277405+0000", 2024-09-14T06:16:54.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1053000000000001e-05 2024-09-14T06:16:54.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.674 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.674 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.280049+0000", 2024-09-14T06:16:54.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026440330000000001 2024-09-14T06:16:54.674 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.674 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.674 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.674 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3868 3.16 3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head [stat,truncate 1998848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:54.674 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:54.674 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.165704823999999, 2024-09-14T06:16:54.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034445919999999998, 2024-09-14T06:16:54.674 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.675 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.675 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.675 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:54.675 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:54.675 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:54.675 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3868 2024-09-14T06:16:54.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.675 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.675 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.675 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.675 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:54.675 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.675 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.676 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.676 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:54.676 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.676 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.676 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.676 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:54.676 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1900000000000002e-07 2024-09-14T06:16:54.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.676 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.676 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.676 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502501+0000", 2024-09-14T06:16:54.676 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8340999999999999e-05 2024-09-14T06:16:54.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.677 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.677 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502512+0000", 2024-09-14T06:16:54.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1347e-05 2024-09-14T06:16:54.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.677 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.677 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502518+0000", 2024-09-14T06:16:54.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.947e-06 2024-09-14T06:16:54.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.677 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502570+0000", 2024-09-14T06:16:54.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1858999999999999e-05 2024-09-14T06:16:54.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.678 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502633+0000", 2024-09-14T06:16:54.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2974999999999999e-05 2024-09-14T06:16:54.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.678 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:16:54.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502795+0000", 2024-09-14T06:16:54.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016262899999999999 2024-09-14T06:16:54.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:54.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505179+0000", 2024-09-14T06:16:54.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002383844 2024-09-14T06:16:54.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:54.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505676+0000", 2024-09-14T06:16:54.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00049634899999999999 2024-09-14T06:16:54.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505816+0000", 2024-09-14T06:16:54.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00014026300000000001 2024-09-14T06:16:54.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.681 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.681 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505926+0000", 2024-09-14T06:16:54.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000110319 2024-09-14T06:16:54.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.681 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.682 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3916 3.16 3:6ab8b1d8:::rbd_data.1152bd92e04f.0000000000000002:head [stat,truncate 1590169] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:54.682 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:54.682 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.088985393000002, 2024-09-14T06:16:54.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022958078999999999, 2024-09-14T06:16:54.682 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.682 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.682 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.682 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:54.682 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:54.682 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:54.682 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3916 2024-09-14T06:16:54.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.682 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:54.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:54.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579203+0000", 2024-09-14T06:16:54.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.748e-06 2024-09-14T06:16:54.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579208+0000", 2024-09-14T06:16:54.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:16:54.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579223+0000", 2024-09-14T06:16:54.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.508e-05 2024-09-14T06:16:54.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579232+0000", 2024-09-14T06:16:54.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9719999999999998e-06 2024-09-14T06:16:54.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579307+0000", 2024-09-14T06:16:54.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5028000000000001e-05 2024-09-14T06:16:54.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579412+0000", 2024-09-14T06:16:54.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010525400000000001 2024-09-14T06:16:54.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:16:54.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579535+0000", 2024-09-14T06:16:54.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122726 2024-09-14T06:16:54.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:54.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.581415+0000", 2024-09-14T06:16:54.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018800360000000001 2024-09-14T06:16:54.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:54.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602017+0000", 2024-09-14T06:16:54.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.020602425000000001 2024-09-14T06:16:54.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602096+0000", 2024-09-14T06:16:54.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8968999999999996e-05 2024-09-14T06:16:54.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602159+0000", 2024-09-14T06:16:54.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2824999999999996e-05 2024-09-14T06:16:54.687 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.687 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.688 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.688 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4873 3.e 3:76656e66:::rbd_data.1152bd92e04f.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=2b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:54.688 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:54.688 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.474538853999999, 2024-09-14T06:16:54.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048796120000000002, 2024-09-14T06:16:54.688 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.688 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.688 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.688 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:54.688 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:54.688 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:54.689 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4873 2024-09-14T06:16:54.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.689 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:54.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:54.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193651+0000", 2024-09-14T06:16:54.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:16:54.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193663+0000", 2024-09-14T06:16:54.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2047e-05 2024-09-14T06:16:54.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193692+0000", 2024-09-14T06:16:54.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9346e-05 2024-09-14T06:16:54.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193708+0000", 2024-09-14T06:16:54.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6123000000000001e-05 2024-09-14T06:16:54.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193798+0000", 2024-09-14T06:16:54.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9909999999999998e-05 2024-09-14T06:16:54.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193863+0000", 2024-09-14T06:16:54.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4583000000000002e-05 2024-09-14T06:16:54.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:54.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.194717+0000", 2024-09-14T06:16:54.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00085473299999999999 2024-09-14T06:16:54.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:54.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198400+0000", 2024-09-14T06:16:54.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036830890000000001 2024-09-14T06:16:54.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.694 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:54.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198461+0000", 2024-09-14T06:16:54.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0844000000000002e-05 2024-09-14T06:16:54.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.694 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198496+0000", 2024-09-14T06:16:54.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4820000000000002e-05 2024-09-14T06:16:54.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.694 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198527+0000", 2024-09-14T06:16:54.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1229999999999997e-05 2024-09-14T06:16:54.695 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.695 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.695 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.695 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:6925 3.5 3:a15e9af1:::rbd_data.1152f87cbac6.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:16:54.695 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:54.695 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.553407316000001, 2024-09-14T06:16:54.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0049096490000000003, 2024-09-14T06:16:54.696 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.696 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.696 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.696 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:54.696 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:54.696 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:54.696 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 6925 2024-09-14T06:16:54.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.696 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:54.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:54.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114780+0000", 2024-09-14T06:16:54.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.09e-07 2024-09-14T06:16:54.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114784+0000", 2024-09-14T06:16:54.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4669999999999998e-06 2024-09-14T06:16:54.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114799+0000", 2024-09-14T06:16:54.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4491000000000001e-05 2024-09-14T06:16:54.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114810+0000", 2024-09-14T06:16:54.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1243000000000001e-05 2024-09-14T06:16:54.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115515+0000", 2024-09-14T06:16:54.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00070521899999999998 2024-09-14T06:16:54.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115610+0000", 2024-09-14T06:16:54.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4260999999999998e-05 2024-09-14T06:16:54.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.700 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:54.700 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115742+0000", 2024-09-14T06:16:54.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000132414 2024-09-14T06:16:54.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.700 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:54.700 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.116650+0000", 2024-09-14T06:16:54.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00090814900000000002 2024-09-14T06:16:54.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:54.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119568+0000", 2024-09-14T06:16:54.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029178440000000002 2024-09-14T06:16:54.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119636+0000", 2024-09-14T06:16:54.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8365e-05 2024-09-14T06:16:54.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119689+0000", 2024-09-14T06:16:54.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2386999999999998e-05 2024-09-14T06:16:54.702 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.702 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.702 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.702 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:7151 3.e 3:7793b17d:::rbd_data.1152f87cbac6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:16:54.702 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:54.703 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.338589526, 2024-09-14T06:16:54.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.029556783, 2024-09-14T06:16:54.703 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.703 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.703 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.703 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:54.703 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:54.703 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:54.703 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 7151 2024-09-14T06:16:54.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.703 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:54.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.704 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:54.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.704 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329599+0000", 2024-09-14T06:16:54.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5270000000000001e-06 2024-09-14T06:16:54.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329610+0000", 2024-09-14T06:16:54.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0431000000000001e-05 2024-09-14T06:16:54.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329634+0000", 2024-09-14T06:16:54.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3617000000000001e-05 2024-09-14T06:16:54.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329644+0000", 2024-09-14T06:16:54.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.095e-05 2024-09-14T06:16:54.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329712+0000", 2024-09-14T06:16:54.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7067e-05 2024-09-14T06:16:54.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329827+0000", 2024-09-14T06:16:54.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011498 2024-09-14T06:16:54.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:54.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.334271+0000", 2024-09-14T06:16:54.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044445839999999997 2024-09-14T06:16:54.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:54.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.358967+0000", 2024-09-14T06:16:54.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024696356999999999 2024-09-14T06:16:54.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:54.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359027+0000", 2024-09-14T06:16:54.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0006000000000001e-05 2024-09-14T06:16:54.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359099+0000", 2024-09-14T06:16:54.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1052999999999999e-05 2024-09-14T06:16:54.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359154+0000", 2024-09-14T06:16:54.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5210999999999997e-05 2024-09-14T06:16:54.709 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.709 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.709 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.709 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:7151 3.e e32/29 ondisk, result = 0)", 2024-09-14T06:16:54.709 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:54.709 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.318645092000001, 2024-09-14T06:16:54.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0096251340000000005, 2024-09-14T06:16:54.709 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.710 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.710 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:54.710 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:54.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:54.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349543+0000", 2024-09-14T06:16:54.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:16:54.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349552+0000", 2024-09-14T06:16:54.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1039999999999996e-06 2024-09-14T06:16:54.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349579+0000", 2024-09-14T06:16:54.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6449999999999999e-05 2024-09-14T06:16:54.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349593+0000", 2024-09-14T06:16:54.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4099e-05 2024-09-14T06:16:54.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359003+0000", 2024-09-14T06:16:54.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0094096119999999995 2024-09-14T06:16:54.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359014+0000", 2024-09-14T06:16:54.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0871e-05 2024-09-14T06:16:54.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359167+0000", 2024-09-14T06:16:54.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015300700000000001 2024-09-14T06:16:54.714 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.714 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.714 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.714 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15516 3.1 e36/29 3:860a8b0d:::rbd_data.115223d7013d.00000000000000af:head v 36'26, mlcod=36'24)", 2024-09-14T06:16:54.714 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:54.714 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 18.410890741999999, 2024-09-14T06:16:54.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035459770000000001, 2024-09-14T06:16:54.715 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.715 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.715 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.715 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:54.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:54.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257297+0000", 2024-09-14T06:16:54.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4900000000000004e-07 2024-09-14T06:16:54.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257304+0000", 2024-09-14T06:16:54.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:16:54.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257321+0000", 2024-09-14T06:16:54.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6311e-05 2024-09-14T06:16:54.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257329+0000", 2024-09-14T06:16:54.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0399999999999993e-06 2024-09-14T06:16:54.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257414+0000", 2024-09-14T06:16:54.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4846999999999996e-05 2024-09-14T06:16:54.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257436+0000", 2024-09-14T06:16:54.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2048999999999998e-05 2024-09-14T06:16:54.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.260796+0000", 2024-09-14T06:16:54.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033607559999999999 2024-09-14T06:16:54.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.260842+0000", 2024-09-14T06:16:54.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5226999999999999e-05 2024-09-14T06:16:54.719 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.719 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.719 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.720 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15613 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'121, mlcod=36'116)", 2024-09-14T06:16:54.720 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:54.720 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 18.375282237, 2024-09-14T06:16:54.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053619389999999996, 2024-09-14T06:16:54.720 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.720 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.720 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.720 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:54.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:54.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292905+0000", 2024-09-14T06:16:54.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6800000000000009e-07 2024-09-14T06:16:54.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292915+0000", 2024-09-14T06:16:54.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:16:54.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292927+0000", 2024-09-14T06:16:54.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-09-14T06:16:54.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292935+0000", 2024-09-14T06:16:54.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2459999999999997e-06 2024-09-14T06:16:54.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292992+0000", 2024-09-14T06:16:54.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6873999999999998e-05 2024-09-14T06:16:54.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293016+0000", 2024-09-14T06:16:54.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4037e-05 2024-09-14T06:16:54.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298217+0000", 2024-09-14T06:16:54.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052008710000000001 2024-09-14T06:16:54.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298266+0000", 2024-09-14T06:16:54.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9691000000000002e-05 2024-09-14T06:16:54.724 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.724 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.724 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.724 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15615 3.11 e36/29 3:8d07d636:::rbd_data.115223d7013d.0000000000000002:head v 36'122, mlcod=36'116)", 2024-09-14T06:16:54.724 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:54.725 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 18.374385877999998, 2024-09-14T06:16:54.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053069229999999998, 2024-09-14T06:16:54.725 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.725 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.725 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.725 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:54.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:54.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:54.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5799999999999998e-07 2024-09-14T06:16:54.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294003+0000", 2024-09-14T06:16:54.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020149600000000001 2024-09-14T06:16:54.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294016+0000", 2024-09-14T06:16:54.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3424999999999999e-05 2024-09-14T06:16:54.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294024+0000", 2024-09-14T06:16:54.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5240000000000003e-06 2024-09-14T06:16:54.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294673+0000", 2024-09-14T06:16:54.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064906700000000003 2024-09-14T06:16:54.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294698+0000", 2024-09-14T06:16:54.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5301999999999999e-05 2024-09-14T06:16:54.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299064+0000", 2024-09-14T06:16:54.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043652170000000002 2024-09-14T06:16:54.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299108+0000", 2024-09-14T06:16:54.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4033999999999999e-05 2024-09-14T06:16:54.729 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.729 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.729 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.729 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15619 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'124, mlcod=36'119)", 2024-09-14T06:16:54.729 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:54.729 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 18.370331929999999, 2024-09-14T06:16:54.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061545389999999997, 2024-09-14T06:16:54.729 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.730 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.730 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.730 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:54.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:54.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:54.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3399999999999998e-07 2024-09-14T06:16:54.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297861+0000", 2024-09-14T06:16:54.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7640000000000002e-06 2024-09-14T06:16:54.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297876+0000", 2024-09-14T06:16:54.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4881e-05 2024-09-14T06:16:54.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297884+0000", 2024-09-14T06:16:54.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.588e-06 2024-09-14T06:16:54.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298294+0000", 2024-09-14T06:16:54.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00041011200000000002 2024-09-14T06:16:54.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298317+0000", 2024-09-14T06:16:54.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3417999999999998e-05 2024-09-14T06:16:54.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303972+0000", 2024-09-14T06:16:54.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056548680000000004 2024-09-14T06:16:54.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304009+0000", 2024-09-14T06:16:54.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7073999999999998e-05 2024-09-14T06:16:54.734 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.734 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.734 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.734 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15621 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'125, mlcod=36'119)", 2024-09-14T06:16:54.734 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:54.734 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 18.368820913, 2024-09-14T06:16:54.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0069496169999999999, 2024-09-14T06:16:54.734 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.734 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.735 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.735 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:54.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:54.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:54.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2200000000000003e-07 2024-09-14T06:16:54.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299372+0000", 2024-09-14T06:16:54.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3850000000000001e-06 2024-09-14T06:16:54.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299387+0000", 2024-09-14T06:16:54.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4905e-05 2024-09-14T06:16:54.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299393+0000", 2024-09-14T06:16:54.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9090000000000003e-06 2024-09-14T06:16:54.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303591+0000", 2024-09-14T06:16:54.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041976640000000003 2024-09-14T06:16:54.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303619+0000", 2024-09-14T06:16:54.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7651e-05 2024-09-14T06:16:54.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306287+0000", 2024-09-14T06:16:54.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026686689999999998 2024-09-14T06:16:54.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306315+0000", 2024-09-14T06:16:54.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7712000000000002e-05 2024-09-14T06:16:54.738 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.738 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.738 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.739 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15623 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'126, mlcod=36'119)", 2024-09-14T06:16:54.739 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:54.739 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 18.365514099999999, 2024-09-14T06:16:54.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003658859, 2024-09-14T06:16:54.739 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.739 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.739 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.739 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:54.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:54.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302673+0000", 2024-09-14T06:16:54.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4199999999999995e-07 2024-09-14T06:16:54.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302679+0000", 2024-09-14T06:16:54.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7599999999999999e-06 2024-09-14T06:16:54.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302689+0000", 2024-09-14T06:16:54.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0335e-05 2024-09-14T06:16:54.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302697+0000", 2024-09-14T06:16:54.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3420000000000002e-06 2024-09-14T06:16:54.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304047+0000", 2024-09-14T06:16:54.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0013503530000000001 2024-09-14T06:16:54.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304059+0000", 2024-09-14T06:16:54.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2285e-05 2024-09-14T06:16:54.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306318+0000", 2024-09-14T06:16:54.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022586580000000002 2024-09-14T06:16:54.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306331+0000", 2024-09-14T06:16:54.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3383999999999999e-05 2024-09-14T06:16:54.744 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.744 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.744 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.744 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15626 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'127, mlcod=36'120)", 2024-09-14T06:16:54.744 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:54.744 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 18.365149705, 2024-09-14T06:16:54.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038671249999999999, 2024-09-14T06:16:54.744 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.744 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.744 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.745 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:54.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:54.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:54.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0100000000000005e-07 2024-09-14T06:16:54.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303043+0000", 2024-09-14T06:16:54.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.293e-06 2024-09-14T06:16:54.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303072+0000", 2024-09-14T06:16:54.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9135000000000001e-05 2024-09-14T06:16:54.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303081+0000", 2024-09-14T06:16:54.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7080000000000003e-06 2024-09-14T06:16:54.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.305965+0000", 2024-09-14T06:16:54.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028841769999999999 2024-09-14T06:16:54.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.305993+0000", 2024-09-14T06:16:54.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8121000000000001e-05 2024-09-14T06:16:54.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306861+0000", 2024-09-14T06:16:54.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00086771099999999996 2024-09-14T06:16:54.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306904+0000", 2024-09-14T06:16:54.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3378999999999999e-05 2024-09-14T06:16:54.748 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.748 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.748 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.749 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:16521 3.e e36/34 3:77c8e3e5:::rbd_data.1152fc09e7e6.0000000000000003:head v 36'83, mlcod=36'81)", 2024-09-14T06:16:54.749 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:54.749 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 17.742167612999999, 2024-09-14T06:16:54.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044948899999999996, 2024-09-14T06:16:54.749 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.749 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.749 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.749 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:54.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:54.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926020+0000", 2024-09-14T06:16:54.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.545e-06 2024-09-14T06:16:54.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927192+0000", 2024-09-14T06:16:54.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011711919999999999 2024-09-14T06:16:54.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927215+0000", 2024-09-14T06:16:54.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3677e-05 2024-09-14T06:16:54.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927226+0000", 2024-09-14T06:16:54.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0665e-05 2024-09-14T06:16:54.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927314+0000", 2024-09-14T06:16:54.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8314999999999997e-05 2024-09-14T06:16:54.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927344+0000", 2024-09-14T06:16:54.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9941e-05 2024-09-14T06:16:54.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.930411+0000", 2024-09-14T06:16:54.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030670649999999999 2024-09-14T06:16:54.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.753 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.930514+0000", 2024-09-14T06:16:54.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010249 2024-09-14T06:16:54.753 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.753 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.753 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.753 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:18539 3.1e e40/29 3:7e28f783:::rbd_data.1152227be94f.0000000000000000:head v 40'59, mlcod=40'57)", 2024-09-14T06:16:54.753 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:16:54.753 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.385145562, 2024-09-14T06:16:54.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0059097430000000003, 2024-09-14T06:16:54.754 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.754 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.754 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.754 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:16:54.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:16:54.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283043+0000", 2024-09-14T06:16:54.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4499999999999998e-06 2024-09-14T06:16:54.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285878+0000", 2024-09-14T06:16:54.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028348039999999998 2024-09-14T06:16:54.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285908+0000", 2024-09-14T06:16:54.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9756000000000001e-05 2024-09-14T06:16:54.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285922+0000", 2024-09-14T06:16:54.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.43e-05 2024-09-14T06:16:54.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.286030+0000", 2024-09-14T06:16:54.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000107954 2024-09-14T06:16:54.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.286058+0000", 2024-09-14T06:16:54.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7796e-05 2024-09-14T06:16:54.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.288888+0000", 2024-09-14T06:16:54.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028295799999999999 2024-09-14T06:16:54.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.758 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.288951+0000", 2024-09-14T06:16:54.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3102999999999995e-05 2024-09-14T06:16:54.758 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.758 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.758 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.758 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:20514 3.1e e42/29 3:7e502f18:::rbd_data.1152b8cc71.0000000000000003:head v 42'90, mlcod=42'88)", 2024-09-14T06:16:54.758 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:45.814156+0000", 2024-09-14T06:16:54.758 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.8540308139999997, 2024-09-14T06:16:54.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0042868200000000002, 2024-09-14T06:16:54.758 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.759 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.759 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.759 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.814156+0000", 2024-09-14T06:16:54.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.814156+0000", 2024-09-14T06:16:54.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.814158+0000", 2024-09-14T06:16:54.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6599999999999999e-06 2024-09-14T06:16:54.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.816022+0000", 2024-09-14T06:16:54.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018634179999999999 2024-09-14T06:16:54.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.816056+0000", 2024-09-14T06:16:54.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4445e-05 2024-09-14T06:16:54.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.816073+0000", 2024-09-14T06:16:54.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6759e-05 2024-09-14T06:16:54.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.816157+0000", 2024-09-14T06:16:54.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4109000000000004e-05 2024-09-14T06:16:54.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.816183+0000", 2024-09-14T06:16:54.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6296999999999999e-05 2024-09-14T06:16:54.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.818339+0000", 2024-09-14T06:16:54.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0021555879999999999 2024-09-14T06:16:54.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.818442+0000", 2024-09-14T06:16:54.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000103544 2024-09-14T06:16:54.763 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.763 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.763 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.763 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:23383 3.2 e45/29 3:416a9e38:::rbd_data.115256cac537.000000000000000e:head v 45'103, mlcod=45'101)", 2024-09-14T06:16:54.763 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:49.995203+0000", 2024-09-14T06:16:54.763 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.6729835629999998, 2024-09-14T06:16:54.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003562169, 2024-09-14T06:16:54.764 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.764 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.764 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.764 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.995203+0000", 2024-09-14T06:16:54.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.995203+0000", 2024-09-14T06:16:54.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.995204+0000", 2024-09-14T06:16:54.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5260000000000001e-06 2024-09-14T06:16:54.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.997471+0000", 2024-09-14T06:16:54.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022666959999999999 2024-09-14T06:16:54.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.997511+0000", 2024-09-14T06:16:54.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0138000000000001e-05 2024-09-14T06:16:54.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.997535+0000", 2024-09-14T06:16:54.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3485999999999998e-05 2024-09-14T06:16:54.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.997629+0000", 2024-09-14T06:16:54.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4246999999999994e-05 2024-09-14T06:16:54.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.997654+0000", 2024-09-14T06:16:54.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4545999999999998e-05 2024-09-14T06:16:54.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.998717+0000", 2024-09-14T06:16:54.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0010629669999999999 2024-09-14T06:16:54.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.998765+0000", 2024-09-14T06:16:54.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8563000000000002e-05 2024-09-14T06:16:54.768 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.768 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.768 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.769 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:23716 3.5 e46/34 3:a78aeb3d:::rbd_data.115256cac537.0000000000000006:head v 46'54, mlcod=45'52)", 2024-09-14T06:16:54.769 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:50.679271+0000", 2024-09-14T06:16:54.769 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.9889158999999998, 2024-09-14T06:16:54.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035774600000000002, 2024-09-14T06:16:54.769 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.769 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.769 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.769 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.679271+0000", 2024-09-14T06:16:54.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.679271+0000", 2024-09-14T06:16:54.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.679271+0000", 2024-09-14T06:16:54.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8299999999999996e-07 2024-09-14T06:16:54.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.681001+0000", 2024-09-14T06:16:54.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001730075 2024-09-14T06:16:54.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.681013+0000", 2024-09-14T06:16:54.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1821000000000001e-05 2024-09-14T06:16:54.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.681021+0000", 2024-09-14T06:16:54.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1300000000000001e-06 2024-09-14T06:16:54.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.681094+0000", 2024-09-14T06:16:54.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2173000000000002e-05 2024-09-14T06:16:54.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.681119+0000", 2024-09-14T06:16:54.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5595000000000001e-05 2024-09-14T06:16:54.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.682775+0000", 2024-09-14T06:16:54.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0016556419999999999 2024-09-14T06:16:54.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.682848+0000", 2024-09-14T06:16:54.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3240999999999999e-05 2024-09-14T06:16:54.773 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.773 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.773 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.774 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:24229 3.5 e47/34 3:a30fd30b:::rbd_data.1152db28e106.0000000000000001:head v 47'55, mlcod=45'53)", 2024-09-14T06:16:54.774 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:16:54.774 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 2.7581753149999999, 2024-09-14T06:16:54.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0092924199999999992, 2024-09-14T06:16:54.774 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:54.774 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:54.774 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:54.774 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:54.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:54.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:16:54.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:54.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:16:54.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:54.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:54.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910013+0000", 2024-09-14T06:16:54.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.812e-06 2024-09-14T06:16:54.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:54.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915374+0000", 2024-09-14T06:16:54.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053609240000000004 2024-09-14T06:16:54.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:54.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915395+0000", 2024-09-14T06:16:54.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1118e-05 2024-09-14T06:16:54.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:54.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915406+0000", 2024-09-14T06:16:54.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1303e-05 2024-09-14T06:16:54.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:54.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915490+0000", 2024-09-14T06:16:54.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4017000000000006e-05 2024-09-14T06:16:54.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:54.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915515+0000", 2024-09-14T06:16:54.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4712e-05 2024-09-14T06:16:54.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:54.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919240+0000", 2024-09-14T06:16:54.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037247690000000002 2024-09-14T06:16:54.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:54.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:54.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:54.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919304+0000", 2024-09-14T06:16:54.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3764999999999997e-05 2024-09-14T06:16:54.779 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.779 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.779 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.779 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:54.779 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:54.779 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:54.780 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:54.780 INFO:teuthology.orchestra.run.smithi039.stdout:2576 writesame 0x470cbc3 thru 0x4716711 data_size 0x1d(0x9b4f bytes) 2024-09-14T06:16:54.780 INFO:teuthology.orchestra.run.smithi039.stdout:2577 writesame 0xbdaf320 thru 0xbdbe97b data_size 0x2(0xf65c bytes) 2024-09-14T06:16:54.781 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:16:54.783 INFO:teuthology.orchestra.run.smithi039.stdout:2578 read 0x8823cb2 thru 0x882af77 (0x72c6 bytes) 2024-09-14T06:16:54.783 INFO:teuthology.orchestra.run.smithi039.stdout:2579 trunc from 0xd9ac71e to 0x709c102 2024-09-14T06:16:54.783 INFO:teuthology.orchestra.run.smithi039.stdout:2580 read 0x6aeb965 thru 0x6af5f58 (0xa5f4 bytes) 2024-09-14T06:16:54.783 INFO:teuthology.orchestra.run.smithi039.stdout:2581 read 0x5a96534 thru 0x5aa51f1 (0xecbe bytes) 2024-09-14T06:16:54.783 INFO:teuthology.orchestra.run.smithi039.stdout:2582 read 0x6da3904 thru 0x6da4ab5 (0x11b2 bytes) 2024-09-14T06:16:54.783 INFO:teuthology.orchestra.run.smithi039.stdout:2584 read 0xcdf561 thru 0xce6bd0 (0x7670 bytes) 2024-09-14T06:16:54.783 INFO:teuthology.orchestra.run.smithi039.stdout:2585 trunc from 0x709c102 to 0xe1383c1 2024-09-14T06:16:54.783 INFO:teuthology.orchestra.run.smithi039.stdout:2587 write 0x993abf9 thru 0x994aa2c (0xfe34 bytes) 2024-09-14T06:16:54.783 INFO:teuthology.orchestra.run.smithi039.stdout:2588 write 0x75ed237 thru 0x75ee932 (0x16fc bytes) 2024-09-14T06:16:54.783 INFO:teuthology.orchestra.run.smithi039.stdout:2589 write 0xa6f0283 thru 0xa6f7217 (0x6f95 bytes) 2024-09-14T06:16:54.784 INFO:teuthology.orchestra.run.smithi039.stdout:2591 punch from 0x2176c01 to 0x21778f6, (0xcf5 bytes) 2024-09-14T06:16:54.784 INFO:teuthology.orchestra.run.smithi039.stdout:2592 trunc from 0xe1383c1 to 0x6913b09 2024-09-14T06:16:54.784 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:54.778+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:54.795 INFO:teuthology.orchestra.run.smithi039.stdout:2594 trunc from 0x6913b09 to 0x80ad1de 2024-09-14T06:16:54.831 INFO:teuthology.orchestra.run.smithi039.stdout:2596 write 0xebdfa63 thru 0xebec4a8 (0xca46 bytes) 2024-09-14T06:16:54.839 INFO:teuthology.orchestra.run.smithi039.stdout:2597 write 0xe0fbcc8 thru 0xe108734 (0xca6d bytes) 2024-09-14T06:16:54.847 INFO:teuthology.orchestra.run.smithi039.stdout:2598 write 0xddff1c0 thru 0xde06f65 (0x7da6 bytes) 2024-09-14T06:16:54.855 INFO:teuthology.orchestra.run.smithi039.stdout:2600 write 0x4a542ae thru 0x4a617b7 (0xd50a bytes) 2024-09-14T06:16:54.861 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:54.862 INFO:teuthology.orchestra.run.smithi039.stdout:2602 write 0xac49a47 thru 0xac541dd (0xa797 bytes) 2024-09-14T06:16:54.867 INFO:teuthology.orchestra.run.smithi039.stdout:2603 punch from 0x9c363d to 0x9c4693, (0x1056 bytes) 2024-09-14T06:16:54.881 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:54.878+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:54.883 INFO:teuthology.orchestra.run.smithi039.stdout:2604 trunc from 0xebec4a9 to 0x4d6ef41 2024-09-14T06:16:54.902 INFO:teuthology.orchestra.run.smithi039.stdout:2605 read 0x44c8436 thru 0x44d4688 (0xc253 bytes) 2024-09-14T06:16:54.914 INFO:teuthology.orchestra.run.smithi039.stdout:2606 trunc from 0x4d6ef41 to 0xc51202e 2024-09-14T06:16:54.922 INFO:teuthology.orchestra.run.smithi039.stdout:2609 write 0xc4c1846 thru 0xc4c74fc (0x5cb7 bytes) 2024-09-14T06:16:54.926 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:54.926 INFO:teuthology.orchestra.run.smithi039.stdout:2611 read 0x6b05d8b thru 0x6b0eb49 (0x8dbf bytes) 2024-09-14T06:16:54.926 INFO:teuthology.orchestra.run.smithi039.stdout:2613 write 0x939b40f thru 0x939f51a (0x410c bytes) 2024-09-14T06:16:54.934 INFO:teuthology.orchestra.run.smithi039.stdout:2614 trunc from 0xc51202e to 0xe67ffbb 2024-09-14T06:16:54.942 INFO:teuthology.orchestra.run.smithi039.stdout:2615 write 0x28f726f thru 0x28f7733 (0x4c5 bytes) 2024-09-14T06:16:54.945 INFO:teuthology.orchestra.run.smithi039.stdout:2616 punch from 0x4bf0890 to 0x4bfc209, (0xb979 bytes) 2024-09-14T06:16:54.948 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:54.948 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:54.949 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:54.949 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:54.960 INFO:teuthology.orchestra.run.smithi039.stdout:2618 trunc from 0xe67ffbb to 0x6674c5b 2024-09-14T06:16:54.972 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:16:54.977 INFO:teuthology.orchestra.run.smithi039.stdout:2619 write 0x9b667e thru 0x9bb44e (0x4dd1 bytes) 2024-09-14T06:16:54.992 INFO:teuthology.orchestra.run.smithi039.stdout:2620 write 0x7112bc5 thru 0x7119e46 (0x7282 bytes) 2024-09-14T06:16:54.993 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:54.987+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:54.996 INFO:teuthology.orchestra.run.smithi039.stdout:2621 read 0xb349a8 thru 0xb41ab5 (0xd10e bytes) 2024-09-14T06:16:55.013 INFO:teuthology.orchestra.run.smithi039.stdout:2622 write 0xcd25a4a thru 0xcd2c0fb (0x66b2 bytes) 2024-09-14T06:16:55.019 INFO:teuthology.orchestra.run.smithi039.stdout:2623 write 0x5578333 thru 0x557f76f (0x743d bytes) 2024-09-14T06:16:55.024 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.033 INFO:teuthology.orchestra.run.smithi039.stdout:2626 trunc from 0xcd2c0fc to 0x73d1fa4 2024-09-14T06:16:55.049 INFO:teuthology.orchestra.run.smithi039.stdout:2630 write 0x56e86a1 thru 0x56e8c47 (0x5a7 bytes) 2024-09-14T06:16:55.054 INFO:teuthology.orchestra.run.smithi039.stdout:2632 flatten 2024-09-14T06:16:55.070 INFO:teuthology.orchestra.run.smithi039.stdout:2633 trunc from 0x73d1fa4 to 0x91bfd3 2024-09-14T06:16:55.082 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:55.079+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:55.115 INFO:teuthology.orchestra.run.smithi039.stdout:2635 trunc from 0x91bfd3 to 0xd3e691f 2024-09-14T06:16:55.121 INFO:teuthology.orchestra.run.smithi039.stdout:2636 write 0x68e7bd3 thru 0x68e9e44 (0x2272 bytes) 2024-09-14T06:16:55.125 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:55.125 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:55.125 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:55.125 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:55.125 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:55.127 INFO:teuthology.orchestra.run.smithi039.stdout:2638 read 0x4f8d321 thru 0x4f9a210 (0xcef0 bytes) 2024-09-14T06:16:55.127 INFO:teuthology.orchestra.run.smithi039.stdout:2639 write 0x4477d71 thru 0x447867a (0x90a bytes) 2024-09-14T06:16:55.133 INFO:teuthology.orchestra.run.smithi039.stdout:2640 write 0x743a183 thru 0x743f4b0 (0x532e bytes) 2024-09-14T06:16:55.143 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:16:55.151 INFO:teuthology.orchestra.run.smithi039.stdout:2641 trunc from 0xd3e691f to 0x64d9c78 2024-09-14T06:16:55.166 INFO:teuthology.orchestra.run.smithi039.stdout:2642 read 0x1034348 thru 0x1037f3f (0x3bf8 bytes) 2024-09-14T06:16:55.167 INFO:teuthology.orchestra.run.smithi039.stdout:2643 read 0x25b6001 thru 0x25b795b (0x195b bytes) 2024-09-14T06:16:55.167 INFO:teuthology.orchestra.run.smithi039.stdout:2644 punch from 0x56fa008 to 0x57096a7, (0xf69f bytes) 2024-09-14T06:16:55.167 INFO:teuthology.orchestra.run.smithi039.stdout:2645 writesame 0x3a86fa5 thru 0x3a8d8d0 data_size 0x4(0x692c bytes) 2024-09-14T06:16:55.183 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:55.182+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:55.184 INFO:teuthology.orchestra.run.smithi039.stdout:2646 writesame 0x74335fb thru 0x743ce38 data_size 0xd(0x983e bytes) 2024-09-14T06:16:55.184 INFO:teuthology.orchestra.run.smithi039.stdout:2647 write 0x2df2e69 thru 0x2e00677 (0xd80f bytes) 2024-09-14T06:16:55.191 INFO:teuthology.orchestra.run.smithi039.stdout:2650 write 0x14410fd thru 0x14417f2 (0x6f6 bytes) 2024-09-14T06:16:55.197 INFO:teuthology.orchestra.run.smithi039.stdout:2651 read 0x6b0b00b thru 0x6b17c4c (0xcc42 bytes) 2024-09-14T06:16:55.214 INFO:teuthology.orchestra.run.smithi039.stdout:2652 write 0xdf8df3a thru 0xdf9c311 (0xe3d8 bytes) 2024-09-14T06:16:55.223 INFO:teuthology.orchestra.run.smithi039.stdout:2653 punch from 0x4d341aa to 0x4d3871f, (0x4575 bytes) 2024-09-14T06:16:55.223 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.223 INFO:teuthology.orchestra.run.smithi039.stdout:2658 write 0xc63c417 thru 0xc640dd8 (0x49c2 bytes) 2024-09-14T06:16:55.230 INFO:teuthology.orchestra.run.smithi039.stdout:2659 read 0xa3a0900 thru 0xa3acfb7 (0xc6b8 bytes) 2024-09-14T06:16:55.230 INFO:teuthology.orchestra.run.smithi039.stdout:2660 punch from 0x4d1bc93 to 0x4d21655, (0x59c2 bytes) 2024-09-14T06:16:55.230 INFO:teuthology.orchestra.run.smithi039.stdout:2661 writesame 0xa2af271 thru 0xa2b1e7e data_size 0x1(0x2c0e bytes) 2024-09-14T06:16:55.248 INFO:teuthology.orchestra.run.smithi039.stdout:2662 trunc from 0xdf9c312 to 0xc5ad974 2024-09-14T06:16:55.266 INFO:teuthology.orchestra.run.smithi039.stdout:2664 writesame 0x926cad5 thru 0x92749a1 data_size 0xd(0x7ecd bytes) 2024-09-14T06:16:55.274 INFO:teuthology.orchestra.run.smithi039.stdout:2665 read 0x42d9176 thru 0x42e2c30 (0x9abb bytes) 2024-09-14T06:16:55.284 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:55.282+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:55.287 INFO:teuthology.orchestra.run.smithi039.stdout:2666 writesame 0xe9cc1c1 thru 0xe9dbe3f data_size 0x1(0xfc7f bytes) 2024-09-14T06:16:55.287 INFO:teuthology.orchestra.run.smithi039.stdout:2667 writesame 0x9f7d99e thru 0x9f8cf77 data_size 0x1(0xf5da bytes) 2024-09-14T06:16:55.289 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:55.289 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:55.289 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:55.289 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:55.289 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.289 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11217 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 925696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.289 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.296163212, 2024-09-14T06:16:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049306313999999997, 2024-09-14T06:16:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11217 2024-09-14T06:16:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.29e-07 2024-09-14T06:16:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991345+0000", 2024-09-14T06:16:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1480000000000001e-06 2024-09-14T06:16:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991349+0000", 2024-09-14T06:16:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9840000000000003e-06 2024-09-14T06:16:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991359+0000", 2024-09-14T06:16:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4050000000000006e-06 2024-09-14T06:16:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.030826+0000", 2024-09-14T06:16:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.039467214 2024-09-14T06:16:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.030894+0000", 2024-09-14T06:16:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8400999999999998e-05 2024-09-14T06:16:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040650+0000", 2024-09-14T06:16:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0097559329999999996 2024-09-14T06:16:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11262 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 81920~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.295977176000001, 2024-09-14T06:16:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049588879000000002, 2024-09-14T06:16:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11262 2024-09-14T06:16:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9599999999999999e-07 2024-09-14T06:16:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991533+0000", 2024-09-14T06:16:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.362e-06 2024-09-14T06:16:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991539+0000", 2024-09-14T06:16:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:16:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991544+0000", 2024-09-14T06:16:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:16:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040755+0000", 2024-09-14T06:16:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049210938000000003 2024-09-14T06:16:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040833+0000", 2024-09-14T06:16:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7812999999999995e-05 2024-09-14T06:16:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041119+0000", 2024-09-14T06:16:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0002855 2024-09-14T06:16:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11263 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.295954397999999, 2024-09-14T06:16:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049944574999999998, 2024-09-14T06:16:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11263 2024-09-14T06:16:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.51e-07 2024-09-14T06:16:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991560+0000", 2024-09-14T06:16:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6719999999999998e-06 2024-09-14T06:16:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991567+0000", 2024-09-14T06:16:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1470000000000004e-06 2024-09-14T06:16:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991572+0000", 2024-09-14T06:16:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.237e-06 2024-09-14T06:16:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041149+0000", 2024-09-14T06:16:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049576988000000002 2024-09-14T06:16:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041277+0000", 2024-09-14T06:16:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000127687 2024-09-14T06:16:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041497+0000", 2024-09-14T06:16:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022039299999999999 2024-09-14T06:16:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11264 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 327680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.295928057000001, 2024-09-14T06:16:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050248567000000001, 2024-09-14T06:16:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11264 2024-09-14T06:16:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.41e-07 2024-09-14T06:16:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991581+0000", 2024-09-14T06:16:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3740000000000001e-06 2024-09-14T06:16:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991585+0000", 2024-09-14T06:16:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9400000000000004e-06 2024-09-14T06:16:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991587+0000", 2024-09-14T06:16:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.694e-06 2024-09-14T06:16:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041527+0000", 2024-09-14T06:16:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049939349000000001 2024-09-14T06:16:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041598+0000", 2024-09-14T06:16:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1633999999999999e-05 2024-09-14T06:16:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041827+0000", 2024-09-14T06:16:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000229235 2024-09-14T06:16:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.295914716999999, 2024-09-14T06:16:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:16:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:16:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:16:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:16:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:16:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:16:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:16:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:16:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:16:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:16:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:16:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:16:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:16:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:16:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:16:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.295901645000001, 2024-09-14T06:16:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:16:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:16:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:16:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:16:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:16:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:16:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:16:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:16:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:16:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:16:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:16:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:16:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:16:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:16:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:16:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:16:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.295884213000001, 2024-09-14T06:16:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:16:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:16:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:16:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:16:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:16:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:16:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:16:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:16:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:16:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:16:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:16:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:16:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:16:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:16:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:16:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.295863107999999, 2024-09-14T06:16:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:16:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:16:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:16:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:16:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:16:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:16:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:16:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:16:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:16:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:16:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:16:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:16:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:16:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:16:55.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:16:55.326 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.326 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.326 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.295842, 2024-09-14T06:16:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:16:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:16:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:16:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:16:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:16:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:16:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:16:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:16:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:16:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:16:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:16:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:16:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:16:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:16:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:16:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13590 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 491520~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.149867118, 2024-09-14T06:16:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042553466999999998, 2024-09-14T06:16:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13590 2024-09-14T06:16:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.34e-07 2024-09-14T06:16:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137642+0000", 2024-09-14T06:16:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0150000000000002e-06 2024-09-14T06:16:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137649+0000", 2024-09-14T06:16:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4010000000000002e-06 2024-09-14T06:16:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137653+0000", 2024-09-14T06:16:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:16:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.179985+0000", 2024-09-14T06:16:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042331580000000001 2024-09-14T06:16:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180076+0000", 2024-09-14T06:16:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0796000000000003e-05 2024-09-14T06:16:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180193+0000", 2024-09-14T06:16:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000117471 2024-09-14T06:16:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13592 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 606208~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.149842549999999, 2024-09-14T06:16:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042709440000000001, 2024-09-14T06:16:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13592 2024-09-14T06:16:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137665+0000", 2024-09-14T06:16:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4299999999999998e-07 2024-09-14T06:16:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137667+0000", 2024-09-14T06:16:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5780000000000001e-06 2024-09-14T06:16:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137675+0000", 2024-09-14T06:16:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0870000000000001e-06 2024-09-14T06:16:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137680+0000", 2024-09-14T06:16:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4999999999999999e-06 2024-09-14T06:16:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180223+0000", 2024-09-14T06:16:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042542808000000001 2024-09-14T06:16:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180294+0000", 2024-09-14T06:16:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1497999999999999e-05 2024-09-14T06:16:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180374+0000", 2024-09-14T06:16:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9525999999999997e-05 2024-09-14T06:16:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13594 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 729088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.149818986, 2024-09-14T06:16:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042830604000000001, 2024-09-14T06:16:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13594 2024-09-14T06:16:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0699999999999998e-07 2024-09-14T06:16:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137690+0000", 2024-09-14T06:16:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.012e-06 2024-09-14T06:16:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137698+0000", 2024-09-14T06:16:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4379999999999998e-06 2024-09-14T06:16:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137703+0000", 2024-09-14T06:16:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1250000000000001e-06 2024-09-14T06:16:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180404+0000", 2024-09-14T06:16:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042701465000000001 2024-09-14T06:16:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180456+0000", 2024-09-14T06:16:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1227999999999998e-05 2024-09-14T06:16:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180519+0000", 2024-09-14T06:16:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-09-14T06:16:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13595 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 851968~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.149796480999999, 2024-09-14T06:16:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043012433000000003, 2024-09-14T06:16:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13595 2024-09-14T06:16:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137711+0000", 2024-09-14T06:16:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:16:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137713+0000", 2024-09-14T06:16:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5170000000000002e-06 2024-09-14T06:16:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137720+0000", 2024-09-14T06:16:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-09-14T06:16:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137726+0000", 2024-09-14T06:16:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9580000000000004e-06 2024-09-14T06:16:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180551+0000", 2024-09-14T06:16:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042825611999999999 2024-09-14T06:16:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180623+0000", 2024-09-14T06:16:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1673000000000003e-05 2024-09-14T06:16:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180723+0000", 2024-09-14T06:16:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.9928999999999998e-05 2024-09-14T06:16:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13597 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 974848~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.149773595999999, 2024-09-14T06:16:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043180442999999999, 2024-09-14T06:16:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13597 2024-09-14T06:16:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137734+0000", 2024-09-14T06:16:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4499999999999997e-07 2024-09-14T06:16:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137736+0000", 2024-09-14T06:16:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0470000000000001e-06 2024-09-14T06:16:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137742+0000", 2024-09-14T06:16:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2269999999999998e-06 2024-09-14T06:16:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137747+0000", 2024-09-14T06:16:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6670000000000004e-06 2024-09-14T06:16:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180753+0000", 2024-09-14T06:16:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043006132000000002 2024-09-14T06:16:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180820+0000", 2024-09-14T06:16:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7516999999999997e-05 2024-09-14T06:16:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180914+0000", 2024-09-14T06:16:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3407999999999997e-05 2024-09-14T06:16:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13684 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 999424~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.147541053000001, 2024-09-14T06:16:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042643702999999998, 2024-09-14T06:16:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13684 2024-09-14T06:16:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139967+0000", 2024-09-14T06:16:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1200000000000003e-07 2024-09-14T06:16:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141080+0000", 2024-09-14T06:16:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011133029999999999 2024-09-14T06:16:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141084+0000", 2024-09-14T06:16:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.544e-06 2024-09-14T06:16:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141088+0000", 2024-09-14T06:16:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0280000000000002e-06 2024-09-14T06:16:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182457+0000", 2024-09-14T06:16:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041368690999999999 2024-09-14T06:16:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182528+0000", 2024-09-14T06:16:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0881000000000001e-05 2024-09-14T06:16:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182610+0000", 2024-09-14T06:16:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1644000000000006e-05 2024-09-14T06:16:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13721 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 532480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.146240319, 2024-09-14T06:16:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042577526999999997, 2024-09-14T06:16:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13721 2024-09-14T06:16:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2999999999999999e-07 2024-09-14T06:16:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141268+0000", 2024-09-14T06:16:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0130000000000001e-06 2024-09-14T06:16:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141271+0000", 2024-09-14T06:16:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0970000000000002e-06 2024-09-14T06:16:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141274+0000", 2024-09-14T06:16:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5809999999999999e-06 2024-09-14T06:16:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183522+0000", 2024-09-14T06:16:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042248591000000002 2024-09-14T06:16:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183691+0000", 2024-09-14T06:16:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016885399999999999 2024-09-14T06:16:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183844+0000", 2024-09-14T06:16:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015316099999999999 2024-09-14T06:16:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13722 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 655360~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.146230017000001, 2024-09-14T06:16:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042741610999999999, 2024-09-14T06:16:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13722 2024-09-14T06:16:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:16:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141278+0000", 2024-09-14T06:16:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.066e-06 2024-09-14T06:16:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141282+0000", 2024-09-14T06:16:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7100000000000001e-06 2024-09-14T06:16:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141285+0000", 2024-09-14T06:16:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.554e-06 2024-09-14T06:16:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183824+0000", 2024-09-14T06:16:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042539043999999998 2024-09-14T06:16:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183906+0000", 2024-09-14T06:16:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2298000000000004e-05 2024-09-14T06:16:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184019+0000", 2024-09-14T06:16:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000112724 2024-09-14T06:16:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13723 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 778240~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.146219047999999, 2024-09-14T06:16:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042995901000000003, 2024-09-14T06:16:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13723 2024-09-14T06:16:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9799999999999999e-07 2024-09-14T06:16:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141289+0000", 2024-09-14T06:16:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0589999999999999e-06 2024-09-14T06:16:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141292+0000", 2024-09-14T06:16:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.208e-06 2024-09-14T06:16:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141295+0000", 2024-09-14T06:16:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5160000000000001e-06 2024-09-14T06:16:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184069+0000", 2024-09-14T06:16:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042774176999999997 2024-09-14T06:16:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184169+0000", 2024-09-14T06:16:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000100073 2024-09-14T06:16:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184284+0000", 2024-09-14T06:16:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011457 2024-09-14T06:16:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13724 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 901120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:16:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.146208653999999, 2024-09-14T06:16:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043197179000000002, 2024-09-14T06:16:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13724 2024-09-14T06:16:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:16:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:16:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141299+0000", 2024-09-14T06:16:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.03e-07 2024-09-14T06:16:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141300+0000", 2024-09-14T06:16:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0389999999999999e-06 2024-09-14T06:16:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141303+0000", 2024-09-14T06:16:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0479999999999999e-06 2024-09-14T06:16:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141305+0000", 2024-09-14T06:16:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5739999999999998e-06 2024-09-14T06:16:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184317+0000", 2024-09-14T06:16:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043011701999999999 2024-09-14T06:16:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184394+0000", 2024-09-14T06:16:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7324999999999996e-05 2024-09-14T06:16:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184496+0000", 2024-09-14T06:16:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010118799999999999 2024-09-14T06:16:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13726 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 1024000~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.146198381000001, 2024-09-14T06:16:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043397185999999997, 2024-09-14T06:16:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13726 2024-09-14T06:16:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:16:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:16:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141310+0000", 2024-09-14T06:16:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:16:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141313+0000", 2024-09-14T06:16:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0309999999999999e-06 2024-09-14T06:16:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141315+0000", 2024-09-14T06:16:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.52e-06 2024-09-14T06:16:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184527+0000", 2024-09-14T06:16:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043211855 2024-09-14T06:16:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184606+0000", 2024-09-14T06:16:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8287000000000004e-05 2024-09-14T06:16:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184706+0000", 2024-09-14T06:16:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010021700000000001 2024-09-14T06:16:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.385 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:55.385 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:55.385 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:55.386 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:16:55.388 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.388 INFO:teuthology.orchestra.run.smithi039.stdout:2669 write 0xa5ed315 thru 0xa5edb34 (0x820 bytes) 2024-09-14T06:16:55.388 INFO:teuthology.orchestra.run.smithi039.stdout:2670 punch from 0xa7a6aaf to 0xa7a8f7e, (0x24cf bytes) 2024-09-14T06:16:55.388 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.388 INFO:teuthology.orchestra.run.smithi039.stdout:2672 write 0xc28444e thru 0xc2861a5 (0x1d58 bytes) 2024-09-14T06:16:55.388 INFO:teuthology.orchestra.run.smithi039.stdout:2673 write 0x49ada98 thru 0x49b6d7b (0x92e4 bytes) 2024-09-14T06:16:55.388 INFO:teuthology.orchestra.run.smithi039.stdout:2674 write 0x9f84248 thru 0x9f93ce0 (0xfa99 bytes) 2024-09-14T06:16:55.389 INFO:teuthology.orchestra.run.smithi039.stdout:2675 punch from 0x18a45f4 to 0x18ac2be, (0x7cca bytes) 2024-09-14T06:16:55.389 INFO:teuthology.orchestra.run.smithi039.stdout:2676 read 0x2e9ab89 thru 0x2ea6b74 (0xbfec bytes) 2024-09-14T06:16:55.389 INFO:teuthology.orchestra.run.smithi039.stdout:2677 read 0x2affb98 thru 0x2b081b0 (0x8619 bytes) 2024-09-14T06:16:55.389 INFO:teuthology.orchestra.run.smithi039.stdout:2678 punch from 0x53039b5 to 0x53109f1, (0xd03c bytes) 2024-09-14T06:16:55.389 INFO:teuthology.orchestra.run.smithi039.stdout:2679 punch from 0x3d98d57 to 0x3da1082, (0x832b bytes) 2024-09-14T06:16:55.389 INFO:teuthology.orchestra.run.smithi039.stdout:2680 read 0x195f490 thru 0x19687e5 (0x9356 bytes) 2024-09-14T06:16:55.389 INFO:teuthology.orchestra.run.smithi039.stdout:2683 writesame 0x9f251de thru 0x9f2a7df data_size 0x2(0x5602 bytes) 2024-09-14T06:16:55.389 INFO:teuthology.orchestra.run.smithi039.stdout:2684 writesame 0x5977474 thru 0x597e6c3 data_size 0x76(0x7250 bytes) 2024-09-14T06:16:55.389 INFO:teuthology.orchestra.run.smithi039.stdout:2685 trunc from 0xe9dbe40 to 0xab5e491 2024-09-14T06:16:55.389 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:55.383+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:55.394 INFO:teuthology.orchestra.run.smithi039.stdout:2686 writesame 0x162f388 thru 0x1632869 data_size 0x7(0x34e2 bytes) 2024-09-14T06:16:55.403 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.424 INFO:teuthology.orchestra.run.smithi039.stdout:2688 writesame 0xd52e117 thru 0xd53d854 data_size 0x2a(0xf73e bytes) 2024-09-14T06:16:55.424 INFO:teuthology.orchestra.run.smithi039.stdout:2689 punch from 0x2956b87 to 0x295e7e2, (0x7c5b bytes) 2024-09-14T06:16:55.424 INFO:teuthology.orchestra.run.smithi039.stdout:2690 write 0x5cb227e thru 0x5cba727 (0x84aa bytes) 2024-09-14T06:16:55.433 INFO:teuthology.orchestra.run.smithi039.stdout:2691 write 0x1a22a96 thru 0x1a23577 (0xae2 bytes) 2024-09-14T06:16:55.439 INFO:teuthology.orchestra.run.smithi039.stdout:2692 write 0xee9a38 thru 0xeeb11c (0x16e5 bytes) 2024-09-14T06:16:55.446 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.446 INFO:teuthology.orchestra.run.smithi039.stdout:2695 write 0x30a5559 thru 0x30a6a88 (0x1530 bytes) 2024-09-14T06:16:55.453 INFO:teuthology.orchestra.run.smithi039.stdout:2696 write 0x4747cf7 thru 0x47491a8 (0x14b2 bytes) 2024-09-14T06:16:55.455 INFO:teuthology.orchestra.run.smithi039.stdout:2697 read 0x126d69c thru 0x127c078 (0xe9dd bytes) 2024-09-14T06:16:55.455 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.456 INFO:teuthology.orchestra.run.smithi039.stdout:2699 writesame 0x3e21df7 thru 0x3e24a31 data_size 0xd(0x2c3b bytes) 2024-09-14T06:16:55.463 INFO:teuthology.orchestra.run.smithi039.stdout:2701 punch from 0xa09041f to 0xa098077, (0x7c58 bytes) 2024-09-14T06:16:55.464 INFO:teuthology.orchestra.run.smithi039.stdout:2703 read 0xb74b059 thru 0xb759836 (0xe7de bytes) 2024-09-14T06:16:55.464 INFO:teuthology.orchestra.run.smithi039.stdout:2704 write 0xaff239d thru 0xb001929 (0xf58d bytes) 2024-09-14T06:16:55.475 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.475 INFO:teuthology.orchestra.run.smithi039.stdout:2710 writesame 0xa4fcfb9 thru 0xa4ffc89 data_size 0xb(0x2cd1 bytes) 2024-09-14T06:16:55.482 INFO:teuthology.orchestra.run.smithi039.stdout:2711 writesame 0xa40aa5d thru 0xa40aefd data_size 0x1(0x4a1 bytes) 2024-09-14T06:16:55.486 INFO:teuthology.orchestra.run.smithi039.stdout:2713 write 0xd11d0cf thru 0xd128327 (0xb259 bytes) 2024-09-14T06:16:55.487 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:55.483+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:55.497 INFO:teuthology.orchestra.run.smithi039.stdout:2714 read 0x4dc5cde thru 0x4dcba89 (0x5dac bytes) 2024-09-14T06:16:55.499 INFO:teuthology.orchestra.run.smithi039.stdout:2715 trunc from 0xd53d855 to 0xcdb5b23 2024-09-14T06:16:55.523 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.524 INFO:teuthology.orchestra.run.smithi039.stdout:2717 read 0x8ce6188 thru 0x8cf2ed4 (0xcd4d bytes) 2024-09-14T06:16:55.524 INFO:teuthology.orchestra.run.smithi039.stdout:2718 read 0x5f026d5 thru 0x5f10328 (0xdc54 bytes) 2024-09-14T06:16:55.524 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:55.525 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:55.525 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:16:55.525 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:55.526 INFO:teuthology.orchestra.run.smithi039.stdout:2719 trunc from 0xcdb5b23 to 0xdd467a3 2024-09-14T06:16:55.538 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:16:55.540 INFO:teuthology.orchestra.run.smithi039.stdout:2720 write 0x8829dc7 thru 0x8832297 (0x84d1 bytes) 2024-09-14T06:16:55.549 INFO:teuthology.orchestra.run.smithi039.stdout:2722 read 0x9fa9b83 thru 0x9fb1531 (0x79af bytes) 2024-09-14T06:16:55.550 INFO:teuthology.orchestra.run.smithi039.stdout:2723 punch from 0x2dfbce2 to 0x2e0a1f5, (0xe513 bytes) 2024-09-14T06:16:55.550 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.550 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.550 INFO:teuthology.orchestra.run.smithi039.stdout:2726 write 0xc8b2f2b thru 0xc8b8521 (0x55f7 bytes) 2024-09-14T06:16:55.557 INFO:teuthology.orchestra.run.smithi039.stdout:2727 write 0xabb304d thru 0xabb86b6 (0x566a bytes) 2024-09-14T06:16:55.574 INFO:teuthology.orchestra.run.smithi039.stdout:2728 trunc from 0xdd467a3 to 0xa94d015 2024-09-14T06:16:55.586 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:16:55.587 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:55.586+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:55.605 INFO:teuthology.orchestra.run.smithi039.stdout:2729 write 0x2835332 thru 0x2838da3 (0x3a72 bytes) 2024-09-14T06:16:55.610 INFO:teuthology.orchestra.run.smithi039.stdout:2730 write 0x71e259f thru 0x71e7e47 (0x58a9 bytes) 2024-09-14T06:16:55.616 INFO:teuthology.orchestra.run.smithi039.stdout:2731 punch from 0xa1e6dec to 0xa1edb07, (0x6d1b bytes) 2024-09-14T06:16:55.616 INFO:teuthology.orchestra.run.smithi039.stdout:2732 writesame 0x5ed330b thru 0x5edff69 data_size 0x1(0xcc5f bytes) 2024-09-14T06:16:55.659 INFO:teuthology.orchestra.run.smithi039.stdout:2733 write 0xdcd523d thru 0xdcd933c (0x4100 bytes) 2024-09-14T06:16:55.667 INFO:teuthology.orchestra.run.smithi039.stdout:2734 write 0x34aea8f thru 0x34b8552 (0x9ac4 bytes) 2024-09-14T06:16:55.669 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:55.670 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:55.670 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:16:55.670 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:16:55.670 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:55.675 INFO:teuthology.orchestra.run.smithi039.stdout:2737 read 0xb17dd0f thru 0xb186689 (0x897b bytes) 2024-09-14T06:16:55.676 INFO:teuthology.orchestra.run.smithi039.stdout:2738 write 0xc9435be thru 0xc94caf3 (0x9536 bytes) 2024-09-14T06:16:55.686 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:16:55.686 INFO:teuthology.orchestra.run.smithi039.stdout:2740 trunc from 0xdcd933d to 0xbde7ab3 2024-09-14T06:16:55.702 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.702 INFO:teuthology.orchestra.run.smithi039.stdout:2744 write 0xf5166f thru 0xf55fff (0x4991 bytes) 2024-09-14T06:16:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:2748 write 0x70687ca thru 0x706d51d (0x4d54 bytes) 2024-09-14T06:16:55.710 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.710 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.710 INFO:teuthology.orchestra.run.smithi039.stdout:2752 read 0x18cda9a thru 0x18d19cd (0x3f34 bytes) 2024-09-14T06:16:55.711 INFO:teuthology.orchestra.run.smithi039.stdout:2753 write 0xae2d5a6 thru 0xae315e3 (0x403e bytes) 2024-09-14T06:16:55.715 INFO:teuthology.orchestra.run.smithi039.stdout:2754 read 0x5676a96 thru 0x5681925 (0xae90 bytes) 2024-09-14T06:16:55.715 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.716 INFO:teuthology.orchestra.run.smithi039.stdout:2757 trunc from 0xbde7ab3 to 0xb8b4292 2024-09-14T06:16:55.725 INFO:teuthology.orchestra.run.smithi039.stdout:2758 writesame 0x7d102 thru 0x888a7 data_size 0xb(0xb7a6 bytes) 2024-09-14T06:16:55.726 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:55.726+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:55.729 INFO:teuthology.orchestra.run.smithi039.stdout:2759 read 0x408772f thru 0x408a8e4 (0x31b6 bytes) 2024-09-14T06:16:55.730 INFO:teuthology.orchestra.run.smithi039.stdout:2760 read 0x2505a6d thru 0x2510633 (0xabc7 bytes) 2024-09-14T06:16:55.730 INFO:teuthology.orchestra.run.smithi039.stdout:2762 write 0x245daab thru 0x24655a3 (0x7af9 bytes) 2024-09-14T06:16:55.737 INFO:teuthology.orchestra.run.smithi039.stdout:2764 write 0x940b46b thru 0x9419331 (0xdec7 bytes) 2024-09-14T06:16:55.740 INFO:teuthology.orchestra.run.smithi039.stdout:2765 write 0x77063f8 thru 0x7710e48 (0xaa51 bytes) 2024-09-14T06:16:55.746 INFO:teuthology.orchestra.run.smithi039.stdout:2767 punch from 0x9b0ccad to 0x9b18878, (0xbbcb bytes) 2024-09-14T06:16:55.746 INFO:teuthology.orchestra.run.smithi039.stdout:2768 write 0x11dd182 thru 0x11eacdd (0xdb5c bytes) 2024-09-14T06:16:55.760 INFO:teuthology.orchestra.run.smithi039.stdout:2769 writesame 0xd94eca7 thru 0xd952fd8 data_size 0x6(0x4332 bytes) 2024-09-14T06:16:55.760 INFO:teuthology.orchestra.run.smithi039.stdout:2770 punch from 0x29f0182 to 0x29f8281, (0x80ff bytes) 2024-09-14T06:16:55.761 INFO:teuthology.orchestra.run.smithi039.stdout:2773 trunc from 0xd952fd9 to 0xce6b1c1 2024-09-14T06:16:55.766 INFO:teuthology.orchestra.run.smithi039.stdout:2774 writesame 0xca4fd89 thru 0xca5e47d data_size 0x37(0xe6f5 bytes) 2024-09-14T06:16:55.773 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.773 INFO:teuthology.orchestra.run.smithi039.stdout:2776 read 0x4ded374 thru 0x4df7d84 (0xaa11 bytes) 2024-09-14T06:16:55.773 INFO:teuthology.orchestra.run.smithi039.stdout:2777 write 0x76a22f6 thru 0x76a6338 (0x4043 bytes) 2024-09-14T06:16:55.774 INFO:teuthology.orchestra.run.smithi039.stdout:2779 read 0x3b9cf5b thru 0x3ba0a70 (0x3b16 bytes) 2024-09-14T06:16:55.775 INFO:teuthology.orchestra.run.smithi039.stdout:2780 write 0x374d966 thru 0x3755118 (0x77b3 bytes) 2024-09-14T06:16:55.779 INFO:teuthology.orchestra.run.smithi039.stdout:2781 read 0x3dd14d4 thru 0x3dd4419 (0x2f46 bytes) 2024-09-14T06:16:55.779 INFO:teuthology.orchestra.run.smithi039.stdout:2783 read 0x302fe7d thru 0x3031ea6 (0x202a bytes) 2024-09-14T06:16:55.779 INFO:teuthology.orchestra.run.smithi039.stdout:2784 read 0xc01a9bd thru 0xc02a5b6 (0xfbfa bytes) 2024-09-14T06:16:55.779 INFO:teuthology.orchestra.run.smithi039.stdout:2785 write 0x8e55d42 thru 0x8e63fc9 (0xe288 bytes) 2024-09-14T06:16:55.790 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:55.786+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:55.792 INFO:teuthology.orchestra.run.smithi039.stdout:2786 write 0xdd38630 thru 0xdd467ce (0xe19f bytes) 2024-09-14T06:16:55.797 INFO:teuthology.orchestra.run.smithi039.stdout:2787 write 0x185c14f thru 0x1863801 (0x76b3 bytes) 2024-09-14T06:16:55.799 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.799 INFO:teuthology.orchestra.run.smithi039.stdout:2789 write 0xa2efddc thru 0xa2ff24c (0xf471 bytes) 2024-09-14T06:16:55.803 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.807 INFO:teuthology.orchestra.run.smithi039.stdout:2794 trunc from 0xdd467cf to 0xb3bba84 2024-09-14T06:16:55.822 INFO:teuthology.orchestra.run.smithi039.stdout:2796 write 0x6db6789 thru 0x6dc28c0 (0xc138 bytes) 2024-09-14T06:16:55.829 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.829 INFO:teuthology.orchestra.run.smithi039.stdout:2798 write 0x881b8ed thru 0x88259d6 (0xa0ea bytes) 2024-09-14T06:16:55.834 INFO:teuthology.orchestra.run.smithi039.stdout:2799 punch from 0x99b9d74 to 0x99ba9a5, (0xc31 bytes) 2024-09-14T06:16:55.834 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:55.834 INFO:teuthology.orchestra.run.smithi039.stdout:2803 read 0x1191cba thru 0x119f52a (0xd871 bytes) 2024-09-14T06:16:55.834 INFO:teuthology.orchestra.run.smithi039.stdout:2804 read 0x4d0abe5 thru 0x4d0d3aa (0x27c6 bytes) 2024-09-14T06:16:55.839 INFO:teuthology.orchestra.run.smithi039.stdout:2805 write 0xb608bcb thru 0xb6138ac (0xace2 bytes) 2024-09-14T06:16:55.845 INFO:teuthology.orchestra.run.smithi039.stdout:2807 punch from 0x1bae83f to 0x1bb65d3, (0x7d94 bytes) 2024-09-14T06:16:55.852 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:55.852 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:16:55.852 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:16:55.852 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:16:55.852 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.853 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13229 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.853 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:16:55.853 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.752082140999999, 2024-09-14T06:16:55.853 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081798868999999996, 2024-09-14T06:16:55.853 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.853 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.854 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.854 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.854 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.854 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.854 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13229 2024-09-14T06:16:55.854 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.854 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.854 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.855 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.855 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:16:55.855 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.855 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.855 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.855 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.855 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:16:55.855 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.856 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.856 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.856 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098710+0000", 2024-09-14T06:16:55.856 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8199999999999999e-07 2024-09-14T06:16:55.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.856 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.856 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.857 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098714+0000", 2024-09-14T06:16:55.857 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:16:55.857 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.857 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.857 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.857 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098727+0000", 2024-09-14T06:16:55.858 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2520999999999999e-05 2024-09-14T06:16:55.858 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.858 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.858 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.858 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098749+0000", 2024-09-14T06:16:55.858 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2773999999999999e-05 2024-09-14T06:16:55.859 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.859 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.859 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.859 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160234+0000", 2024-09-14T06:16:55.859 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.061484430999999999 2024-09-14T06:16:55.859 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.859 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.859 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.859 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160281+0000", 2024-09-14T06:16:55.859 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7437e-05 2024-09-14T06:16:55.860 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.860 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.860 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.860 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160569+0000", 2024-09-14T06:16:55.860 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028788700000000002 2024-09-14T06:16:55.860 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.860 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.860 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.860 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163121+0000", 2024-09-14T06:16:55.860 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025514090000000001 2024-09-14T06:16:55.860 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.860 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.860 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.860 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180410+0000", 2024-09-14T06:16:55.861 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.017289098999999999 2024-09-14T06:16:55.861 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.861 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.861 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.861 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180472+0000", 2024-09-14T06:16:55.861 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2547999999999998e-05 2024-09-14T06:16:55.861 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.861 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.861 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.861 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180508+0000", 2024-09-14T06:16:55.861 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5534000000000002e-05 2024-09-14T06:16:55.861 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.861 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.862 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.862 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.862 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.862 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13230 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.862 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:16:55.862 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.752000938999998, 2024-09-14T06:16:55.862 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081851558000000005, 2024-09-14T06:16:55.862 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.862 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.862 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.862 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.862 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.862 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.862 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13230 2024-09-14T06:16:55.863 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.863 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.863 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.863 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.863 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:16:55.863 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.863 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.863 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.863 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.863 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:16:55.863 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.863 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.863 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.863 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.864 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098791+0000", 2024-09-14T06:16:55.864 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8800000000000001e-07 2024-09-14T06:16:55.864 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.864 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.864 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.864 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098795+0000", 2024-09-14T06:16:55.864 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.391e-06 2024-09-14T06:16:55.864 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.864 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.864 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.864 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098804+0000", 2024-09-14T06:16:55.864 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3440000000000001e-06 2024-09-14T06:16:55.864 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.864 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.865 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.865 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098810+0000", 2024-09-14T06:16:55.865 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2339999999999999e-06 2024-09-14T06:16:55.865 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.865 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.865 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.865 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160930+0000", 2024-09-14T06:16:55.865 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062120018999999999 2024-09-14T06:16:55.865 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.865 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.865 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.865 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161017+0000", 2024-09-14T06:16:55.865 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6868999999999998e-05 2024-09-14T06:16:55.865 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.866 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.866 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.866 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161300+0000", 2024-09-14T06:16:55.866 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028349499999999998 2024-09-14T06:16:55.866 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.866 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.866 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.866 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164612+0000", 2024-09-14T06:16:55.866 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033119680000000002 2024-09-14T06:16:55.866 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.866 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.866 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.866 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180559+0000", 2024-09-14T06:16:55.866 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015946612999999998 2024-09-14T06:16:55.866 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.867 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.867 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.867 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180605+0000", 2024-09-14T06:16:55.867 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5890000000000003e-05 2024-09-14T06:16:55.867 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.867 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.867 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.867 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180642+0000", 2024-09-14T06:16:55.867 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:16:55.867 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.867 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.867 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.867 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.867 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.867 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13231 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.868 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:16:55.868 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.751738525, 2024-09-14T06:16:55.868 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081750685000000003, 2024-09-14T06:16:55.868 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.868 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.868 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.868 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.868 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.868 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.868 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13231 2024-09-14T06:16:55.868 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.868 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.868 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.868 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.869 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:16:55.869 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.869 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.869 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.869 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.869 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:16:55.869 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.870 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.870 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099053+0000", 2024-09-14T06:16:55.870 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:16:55.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.871 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.871 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.871 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099059+0000", 2024-09-14T06:16:55.871 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1789999999999999e-06 2024-09-14T06:16:55.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.871 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.872 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.872 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099072+0000", 2024-09-14T06:16:55.872 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3179999999999999e-05 2024-09-14T06:16:55.872 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.872 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.872 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.873 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099080+0000", 2024-09-14T06:16:55.873 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9030000000000004e-06 2024-09-14T06:16:55.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.873 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.873 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.873 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161681+0000", 2024-09-14T06:16:55.873 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062601641999999999 2024-09-14T06:16:55.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.873 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.873 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.873 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161766+0000", 2024-09-14T06:16:55.873 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4909999999999998e-05 2024-09-14T06:16:55.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.874 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.874 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162109+0000", 2024-09-14T06:16:55.874 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00034264899999999999 2024-09-14T06:16:55.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.874 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.874 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164615+0000", 2024-09-14T06:16:55.874 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025061979999999998 2024-09-14T06:16:55.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.874 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.874 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180718+0000", 2024-09-14T06:16:55.875 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016103240000000001 2024-09-14T06:16:55.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.875 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.875 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.875 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180768+0000", 2024-09-14T06:16:55.875 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9781999999999999e-05 2024-09-14T06:16:55.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.875 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.875 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.875 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180803+0000", 2024-09-14T06:16:55.875 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5021999999999998e-05 2024-09-14T06:16:55.875 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.875 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.876 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.876 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.876 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13232 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.876 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:16:55.876 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.751696751000001, 2024-09-14T06:16:55.876 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081861981, 2024-09-14T06:16:55.876 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.876 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.876 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.876 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.876 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.877 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.877 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13232 2024-09-14T06:16:55.877 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.877 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.877 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.877 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.877 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:16:55.877 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.877 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.877 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.877 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.877 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:16:55.877 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.877 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.878 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.878 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.878 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099095+0000", 2024-09-14T06:16:55.878 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:16:55.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.878 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.878 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.878 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099098+0000", 2024-09-14T06:16:55.878 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5420000000000001e-06 2024-09-14T06:16:55.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.878 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.878 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.878 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099106+0000", 2024-09-14T06:16:55.878 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.824e-06 2024-09-14T06:16:55.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.879 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.879 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099112+0000", 2024-09-14T06:16:55.879 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9789999999999998e-06 2024-09-14T06:16:55.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.879 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.879 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162576+0000", 2024-09-14T06:16:55.879 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.063463723999999999 2024-09-14T06:16:55.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.879 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.879 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162627+0000", 2024-09-14T06:16:55.879 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1530000000000003e-05 2024-09-14T06:16:55.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.880 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162854+0000", 2024-09-14T06:16:55.880 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000226159 2024-09-14T06:16:55.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.880 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.880 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166090+0000", 2024-09-14T06:16:55.880 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032362319999999999 2024-09-14T06:16:55.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.880 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.880 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180877+0000", 2024-09-14T06:16:55.880 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014786726 2024-09-14T06:16:55.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.881 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.881 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.881 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180920+0000", 2024-09-14T06:16:55.881 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3427000000000003e-05 2024-09-14T06:16:55.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.881 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.881 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.881 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180956+0000", 2024-09-14T06:16:55.881 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6195999999999997e-05 2024-09-14T06:16:55.881 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.881 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.881 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.882 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.882 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13233 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.882 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:16:55.882 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.751671487999999, 2024-09-14T06:16:55.882 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081924860000000002, 2024-09-14T06:16:55.882 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.882 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.882 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.882 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.882 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.882 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.882 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13233 2024-09-14T06:16:55.882 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.883 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.883 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.883 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.883 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:16:55.883 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.884 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.884 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.884 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:16:55.884 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.884 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099120+0000", 2024-09-14T06:16:55.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:16:55.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.886 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.886 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099122+0000", 2024-09-14T06:16:55.886 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:16:55.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.886 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.887 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099126+0000", 2024-09-14T06:16:55.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2540000000000003e-06 2024-09-14T06:16:55.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.887 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.887 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.887 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099130+0000", 2024-09-14T06:16:55.888 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4060000000000001e-06 2024-09-14T06:16:55.888 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.888 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.888 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.888 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163169+0000", 2024-09-14T06:16:55.888 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064039415000000002 2024-09-14T06:16:55.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.889 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.889 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.889 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163260+0000", 2024-09-14T06:16:55.889 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1038000000000005e-05 2024-09-14T06:16:55.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.890 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.890 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.890 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163640+0000", 2024-09-14T06:16:55.890 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037954699999999999 2024-09-14T06:16:55.890 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.890 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.891 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.891 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166093+0000", 2024-09-14T06:16:55.891 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024533229999999999 2024-09-14T06:16:55.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.891 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.891 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.891 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180993+0000", 2024-09-14T06:16:55.892 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01489997 2024-09-14T06:16:55.892 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.892 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.892 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.892 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181020+0000", 2024-09-14T06:16:55.892 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6806999999999999e-05 2024-09-14T06:16:55.893 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.893 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.893 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.893 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181044+0000", 2024-09-14T06:16:55.893 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4405000000000001e-05 2024-09-14T06:16:55.893 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.893 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.893 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.894 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.894 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.894 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13234 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.894 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:16:55.894 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.751629283, 2024-09-14T06:16:55.894 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082015597999999995, 2024-09-14T06:16:55.895 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.895 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.895 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.895 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.895 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.895 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.895 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13234 2024-09-14T06:16:55.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.896 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.896 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.896 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.896 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:16:55.896 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.897 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.897 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.897 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:16:55.897 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.898 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.898 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.898 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:16:55.898 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0999999999999998e-07 2024-09-14T06:16:55.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.898 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.899 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.899 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099166+0000", 2024-09-14T06:16:55.899 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:16:55.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.899 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.899 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.899 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099171+0000", 2024-09-14T06:16:55.900 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6949999999999997e-06 2024-09-14T06:16:55.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.900 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.900 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.900 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099175+0000", 2024-09-14T06:16:55.900 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9029999999999997e-06 2024-09-14T06:16:55.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.901 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.901 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164063+0000", 2024-09-14T06:16:55.901 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064887608999999999 2024-09-14T06:16:55.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.901 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.902 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164111+0000", 2024-09-14T06:16:55.902 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7648999999999997e-05 2024-09-14T06:16:55.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.902 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.902 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.902 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164351+0000", 2024-09-14T06:16:55.902 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024072899999999999 2024-09-14T06:16:55.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.902 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.903 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167561+0000", 2024-09-14T06:16:55.903 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032094179999999999 2024-09-14T06:16:55.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.903 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181093+0000", 2024-09-14T06:16:55.903 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013532245 2024-09-14T06:16:55.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.904 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.904 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.904 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181139+0000", 2024-09-14T06:16:55.904 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6483999999999997e-05 2024-09-14T06:16:55.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.904 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.905 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.905 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181177+0000", 2024-09-14T06:16:55.905 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7861000000000002e-05 2024-09-14T06:16:55.905 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.905 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.905 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.906 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.906 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.906 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13235 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.906 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:16:55.907 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.751522008999999, 2024-09-14T06:16:55.907 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081988088000000001, 2024-09-14T06:16:55.907 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.907 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.907 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.907 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.908 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.908 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.908 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13235 2024-09-14T06:16:55.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.908 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.908 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.909 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:16:55.909 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.909 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:16:55.909 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.909 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099270+0000", 2024-09-14T06:16:55.909 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7400000000000003e-07 2024-09-14T06:16:55.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.910 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.910 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099276+0000", 2024-09-14T06:16:55.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:16:55.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.910 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.910 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099284+0000", 2024-09-14T06:16:55.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7979999999999995e-06 2024-09-14T06:16:55.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.910 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.910 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099289+0000", 2024-09-14T06:16:55.911 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:16:55.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.911 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.911 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.911 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164664+0000", 2024-09-14T06:16:55.911 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065374901999999999 2024-09-14T06:16:55.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.911 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.911 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.911 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164755+0000", 2024-09-14T06:16:55.911 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0790000000000003e-05 2024-09-14T06:16:55.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.911 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.911 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.912 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165143+0000", 2024-09-14T06:16:55.912 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000387904 2024-09-14T06:16:55.912 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.912 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.912 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.912 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167564+0000", 2024-09-14T06:16:55.912 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024208989999999998 2024-09-14T06:16:55.912 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.912 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.912 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.912 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181212+0000", 2024-09-14T06:16:55.912 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013648664 2024-09-14T06:16:55.912 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.913 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.913 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.913 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181237+0000", 2024-09-14T06:16:55.913 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5177999999999998e-05 2024-09-14T06:16:55.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.913 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181257+0000", 2024-09-14T06:16:55.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9660000000000002e-05 2024-09-14T06:16:55.914 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.914 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.914 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.914 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13236 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.914 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:16:55.914 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.751415189999999, 2024-09-14T06:16:55.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082010939000000005, 2024-09-14T06:16:55.914 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.914 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.915 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.915 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.915 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.915 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.915 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13236 2024-09-14T06:16:55.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.915 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.915 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.915 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:16:55.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:16:55.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.917 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.917 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.917 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:16:55.917 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.61e-07 2024-09-14T06:16:55.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.917 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.918 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.918 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099382+0000", 2024-09-14T06:16:55.918 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3589999999999999e-06 2024-09-14T06:16:55.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.918 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.918 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.919 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099388+0000", 2024-09-14T06:16:55.919 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9619999999999998e-06 2024-09-14T06:16:55.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.919 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.919 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.919 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099393+0000", 2024-09-14T06:16:55.919 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3410000000000002e-06 2024-09-14T06:16:55.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.920 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.920 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165553+0000", 2024-09-14T06:16:55.920 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066159604999999996 2024-09-14T06:16:55.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.921 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.921 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165603+0000", 2024-09-14T06:16:55.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0427999999999999e-05 2024-09-14T06:16:55.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.922 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.922 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165827+0000", 2024-09-14T06:16:55.922 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022358299999999999 2024-09-14T06:16:55.922 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.922 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.922 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.923 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177005+0000", 2024-09-14T06:16:55.923 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011178044 2024-09-14T06:16:55.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.923 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.923 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.923 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181302+0000", 2024-09-14T06:16:55.924 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042970669999999999 2024-09-14T06:16:55.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.924 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.924 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.924 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181349+0000", 2024-09-14T06:16:55.924 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7287999999999999e-05 2024-09-14T06:16:55.925 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181387+0000", 2024-09-14T06:16:55.925 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7601000000000001e-05 2024-09-14T06:16:55.926 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.926 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.926 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.926 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13237 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.926 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:16:55.927 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.751257042999999, 2024-09-14T06:16:55.927 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081927299999999995, 2024-09-14T06:16:55.927 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.927 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.927 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.928 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.928 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.928 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.928 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13237 2024-09-14T06:16:55.928 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.928 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:16:55.929 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.929 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:16:55.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.930 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099535+0000", 2024-09-14T06:16:55.931 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:16:55.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.931 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.931 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.931 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099538+0000", 2024-09-14T06:16:55.931 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:16:55.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.932 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.932 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.932 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099545+0000", 2024-09-14T06:16:55.932 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:16:55.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.932 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.933 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.933 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099551+0000", 2024-09-14T06:16:55.933 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1770000000000003e-06 2024-09-14T06:16:55.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.933 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.933 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166131+0000", 2024-09-14T06:16:55.933 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066580499000000001 2024-09-14T06:16:55.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166221+0000", 2024-09-14T06:16:55.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0000000000000006e-05 2024-09-14T06:16:55.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166622+0000", 2024-09-14T06:16:55.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040029799999999999 2024-09-14T06:16:55.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177008+0000", 2024-09-14T06:16:55.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010386772000000001 2024-09-14T06:16:55.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.935 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.935 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.935 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181420+0000", 2024-09-14T06:16:55.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044119190000000003 2024-09-14T06:16:55.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.935 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.935 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.935 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181443+0000", 2024-09-14T06:16:55.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2603999999999999e-05 2024-09-14T06:16:55.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.935 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.935 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.936 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181461+0000", 2024-09-14T06:16:55.936 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8405000000000001e-05 2024-09-14T06:16:55.936 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.936 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.936 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.936 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.936 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13238 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.936 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:16:55.936 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.751220735, 2024-09-14T06:16:55.936 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082014927000000001, 2024-09-14T06:16:55.936 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.936 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.937 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.937 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.937 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.937 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.937 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13238 2024-09-14T06:16:55.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.937 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.937 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.937 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:16:55.937 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.937 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:16:55.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099571+0000", 2024-09-14T06:16:55.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9699999999999996e-07 2024-09-14T06:16:55.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099577+0000", 2024-09-14T06:16:55.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2890000000000003e-06 2024-09-14T06:16:55.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099587+0000", 2024-09-14T06:16:55.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0002e-05 2024-09-14T06:16:55.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099593+0000", 2024-09-14T06:16:55.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4539999999999998e-06 2024-09-14T06:16:55.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167024+0000", 2024-09-14T06:16:55.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067431480000000002 2024-09-14T06:16:55.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.940 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.940 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.940 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167074+0000", 2024-09-14T06:16:55.940 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9996e-05 2024-09-14T06:16:55.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.940 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.940 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.940 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167288+0000", 2024-09-14T06:16:55.940 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021410900000000001 2024-09-14T06:16:55.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.940 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.940 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.940 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177997+0000", 2024-09-14T06:16:55.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010708756 2024-09-14T06:16:55.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181508+0000", 2024-09-14T06:16:55.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003511363 2024-09-14T06:16:55.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181552+0000", 2024-09-14T06:16:55.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3609999999999998e-05 2024-09-14T06:16:55.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181585+0000", 2024-09-14T06:16:55.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3271e-05 2024-09-14T06:16:55.942 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.942 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.942 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.942 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13239 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.942 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:16:55.942 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.751128916999999, 2024-09-14T06:16:55.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081998772999999997, 2024-09-14T06:16:55.942 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.942 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.943 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.943 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.943 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.943 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.943 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13239 2024-09-14T06:16:55.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.943 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.943 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.943 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:16:55.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.944 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.944 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:16:55.944 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.944 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.944 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099663+0000", 2024-09-14T06:16:55.944 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6100000000000001e-07 2024-09-14T06:16:55.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.944 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099666+0000", 2024-09-14T06:16:55.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.056e-06 2024-09-14T06:16:55.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099672+0000", 2024-09-14T06:16:55.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8120000000000002e-06 2024-09-14T06:16:55.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099676+0000", 2024-09-14T06:16:55.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:16:55.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167602+0000", 2024-09-14T06:16:55.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067925867000000001 2024-09-14T06:16:55.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167693+0000", 2024-09-14T06:16:55.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0736000000000007e-05 2024-09-14T06:16:55.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168065+0000", 2024-09-14T06:16:55.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037245399999999998 2024-09-14T06:16:55.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178003+0000", 2024-09-14T06:16:55.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0099372360000000003 2024-09-14T06:16:55.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181621+0000", 2024-09-14T06:16:55.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036178489999999998 2024-09-14T06:16:55.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181643+0000", 2024-09-14T06:16:55.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2155e-05 2024-09-14T06:16:55.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.948 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.948 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.948 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181661+0000", 2024-09-14T06:16:55.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8147000000000001e-05 2024-09-14T06:16:55.948 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.948 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.948 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.948 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.948 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13240 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.948 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:16:55.949 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.751009824, 2024-09-14T06:16:55.949 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081942607000000001, 2024-09-14T06:16:55.949 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.949 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.949 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.949 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.949 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.949 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.949 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13240 2024-09-14T06:16:55.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.949 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.949 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.949 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:16:55.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:16:55.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099782+0000", 2024-09-14T06:16:55.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3900000000000004e-07 2024-09-14T06:16:55.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099787+0000", 2024-09-14T06:16:55.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6189999999999999e-06 2024-09-14T06:16:55.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099793+0000", 2024-09-14T06:16:55.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.998e-06 2024-09-14T06:16:55.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099797+0000", 2024-09-14T06:16:55.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8449999999999996e-06 2024-09-14T06:16:55.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.952 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.952 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168494+0000", 2024-09-14T06:16:55.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.068696500999999993 2024-09-14T06:16:55.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.952 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.952 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168590+0000", 2024-09-14T06:16:55.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6733999999999999e-05 2024-09-14T06:16:55.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.952 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.952 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168973+0000", 2024-09-14T06:16:55.953 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038229399999999998 2024-09-14T06:16:55.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.953 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.953 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.953 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178013+0000", 2024-09-14T06:16:55.953 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0090397199999999994 2024-09-14T06:16:55.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.953 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.953 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.953 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181683+0000", 2024-09-14T06:16:55.953 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036705819999999999 2024-09-14T06:16:55.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.953 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.953 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181705+0000", 2024-09-14T06:16:55.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1551000000000001e-05 2024-09-14T06:16:55.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181724+0000", 2024-09-14T06:16:55.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9123999999999999e-05 2024-09-14T06:16:55.954 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.954 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.954 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.954 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13243 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.954 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:16:55.955 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.750476094, 2024-09-14T06:16:55.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081767815999999993, 2024-09-14T06:16:55.955 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.955 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.955 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.955 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.955 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.955 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.955 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13243 2024-09-14T06:16:55.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.955 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:16:55.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:16:55.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:16:55.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1e-07 2024-09-14T06:16:55.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.957 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.957 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100317+0000", 2024-09-14T06:16:55.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.092e-06 2024-09-14T06:16:55.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.957 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.957 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100321+0000", 2024-09-14T06:16:55.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7809999999999999e-06 2024-09-14T06:16:55.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.958 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100324+0000", 2024-09-14T06:16:55.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.117e-06 2024-09-14T06:16:55.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.958 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171113+0000", 2024-09-14T06:16:55.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.070788816000000004 2024-09-14T06:16:55.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171202+0000", 2024-09-14T06:16:55.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8843000000000003e-05 2024-09-14T06:16:55.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171584+0000", 2024-09-14T06:16:55.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038231099999999998 2024-09-14T06:16:55.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178101+0000", 2024-09-14T06:16:55.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0065168329999999997 2024-09-14T06:16:55.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.960 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.960 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182041+0000", 2024-09-14T06:16:55.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039401810000000001 2024-09-14T06:16:55.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.960 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.960 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182065+0000", 2024-09-14T06:16:55.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3623000000000001e-05 2024-09-14T06:16:55.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.960 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.961 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182083+0000", 2024-09-14T06:16:55.961 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7909000000000001e-05 2024-09-14T06:16:55.961 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.961 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.961 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.961 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.961 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.961 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13244 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.961 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:16:55.961 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.750461273999999, 2024-09-14T06:16:55.961 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081837800000000002, 2024-09-14T06:16:55.961 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.961 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.961 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.962 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.962 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.962 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.962 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13244 2024-09-14T06:16:55.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.962 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.962 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.962 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.962 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:16:55.962 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.962 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.962 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.962 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:16:55.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:16:55.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:16:55.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100337+0000", 2024-09-14T06:16:55.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.939e-06 2024-09-14T06:16:55.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100341+0000", 2024-09-14T06:16:55.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2320000000000003e-06 2024-09-14T06:16:55.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100344+0000", 2024-09-14T06:16:55.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0759999999999999e-06 2024-09-14T06:16:55.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171996+0000", 2024-09-14T06:16:55.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.071651919999999994 2024-09-14T06:16:55.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172079+0000", 2024-09-14T06:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2579999999999999e-05 2024-09-14T06:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172457+0000", 2024-09-14T06:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037785100000000001 2024-09-14T06:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178104+0000", 2024-09-14T06:16:55.965 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056471730000000001 2024-09-14T06:16:55.966 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.966 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.966 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.966 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182123+0000", 2024-09-14T06:16:55.966 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040190019999999998 2024-09-14T06:16:55.966 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.966 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.966 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.966 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182147+0000", 2024-09-14T06:16:55.966 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4241000000000001e-05 2024-09-14T06:16:55.966 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.966 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.966 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.967 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182168+0000", 2024-09-14T06:16:55.967 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0492999999999998e-05 2024-09-14T06:16:55.967 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.967 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.967 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.967 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.967 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13245 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.967 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:16:55.967 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.750441767000002, 2024-09-14T06:16:55.967 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081893899000000006, 2024-09-14T06:16:55.968 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.968 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.968 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.968 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.968 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.968 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.968 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13245 2024-09-14T06:16:55.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.968 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.968 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.968 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.968 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:16:55.968 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.969 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.969 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:16:55.969 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.969 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.969 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100350+0000", 2024-09-14T06:16:55.969 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5499999999999999e-07 2024-09-14T06:16:55.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.969 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.970 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100353+0000", 2024-09-14T06:16:55.970 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:16:55.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.970 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.970 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.970 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100357+0000", 2024-09-14T06:16:55.970 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8500000000000004e-06 2024-09-14T06:16:55.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.970 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.971 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.971 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100360+0000", 2024-09-14T06:16:55.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8700000000000001e-06 2024-09-14T06:16:55.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.971 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.971 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.971 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172862+0000", 2024-09-14T06:16:55.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.072501992000000001 2024-09-14T06:16:55.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.971 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.972 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.972 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172950+0000", 2024-09-14T06:16:55.972 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8988999999999998e-05 2024-09-14T06:16:55.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.972 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.972 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.972 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173324+0000", 2024-09-14T06:16:55.972 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037366199999999997 2024-09-14T06:16:55.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.972 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.972 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178107+0000", 2024-09-14T06:16:55.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047830520000000003 2024-09-14T06:16:55.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.973 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182202+0000", 2024-09-14T06:16:55.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040949280000000003 2024-09-14T06:16:55.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.973 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182226+0000", 2024-09-14T06:16:55.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3535000000000001e-05 2024-09-14T06:16:55.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.973 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.974 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.974 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182243+0000", 2024-09-14T06:16:55.974 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7445e-05 2024-09-14T06:16:55.974 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.974 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.974 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.974 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.974 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13246 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.974 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:16:55.974 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.750427445, 2024-09-14T06:16:55.974 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081962276000000001, 2024-09-14T06:16:55.974 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.974 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.975 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.975 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.975 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.975 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.975 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13246 2024-09-14T06:16:55.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.975 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.975 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.975 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.975 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:16:55.975 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.975 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:16:55.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.976 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:16:55.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0800000000000001e-07 2024-09-14T06:16:55.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.976 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100367+0000", 2024-09-14T06:16:55.977 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8439999999999998e-06 2024-09-14T06:16:55.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.977 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.977 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100372+0000", 2024-09-14T06:16:55.977 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4009999999999996e-06 2024-09-14T06:16:55.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.977 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.977 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100378+0000", 2024-09-14T06:16:55.977 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7180000000000001e-06 2024-09-14T06:16:55.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.978 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.978 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173780+0000", 2024-09-14T06:16:55.978 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.073401991 2024-09-14T06:16:55.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.978 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.978 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.978 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173879+0000", 2024-09-14T06:16:55.978 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9449000000000003e-05 2024-09-14T06:16:55.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.978 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.978 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.978 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174240+0000", 2024-09-14T06:16:55.978 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036118100000000002 2024-09-14T06:16:55.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.978 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.978 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178109+0000", 2024-09-14T06:16:55.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038690619999999999 2024-09-14T06:16:55.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182281+0000", 2024-09-14T06:16:55.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041715900000000002 2024-09-14T06:16:55.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182306+0000", 2024-09-14T06:16:55.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4477e-05 2024-09-14T06:16:55.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182326+0000", 2024-09-14T06:16:55.980 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0254999999999998e-05 2024-09-14T06:16:55.980 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.980 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.980 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.980 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.980 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13247 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.980 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:16:55.980 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.750330267999999, 2024-09-14T06:16:55.980 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966593000000004, 2024-09-14T06:16:55.980 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.980 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.980 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.981 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.981 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.981 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.981 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13247 2024-09-14T06:16:55.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.981 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.981 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.981 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.981 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:16:55.981 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.981 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:16:55.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.982 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100462+0000", 2024-09-14T06:16:55.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9800000000000002e-07 2024-09-14T06:16:55.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.982 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100466+0000", 2024-09-14T06:16:55.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4039999999999997e-06 2024-09-14T06:16:55.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.983 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100475+0000", 2024-09-14T06:16:55.983 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3680000000000008e-06 2024-09-14T06:16:55.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.983 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100482+0000", 2024-09-14T06:16:55.983 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8630000000000002e-06 2024-09-14T06:16:55.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.983 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174669+0000", 2024-09-14T06:16:55.983 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074186849999999999 2024-09-14T06:16:55.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.984 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.984 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174758+0000", 2024-09-14T06:16:55.984 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9021e-05 2024-09-14T06:16:55.984 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.984 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.984 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.984 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175137+0000", 2024-09-14T06:16:55.984 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378511 2024-09-14T06:16:55.984 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.984 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.984 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.984 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178158+0000", 2024-09-14T06:16:55.984 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003020956 2024-09-14T06:16:55.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.985 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.985 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.985 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182386+0000", 2024-09-14T06:16:55.985 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042278840000000003 2024-09-14T06:16:55.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.985 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.985 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.985 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182410+0000", 2024-09-14T06:16:55.985 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4026e-05 2024-09-14T06:16:55.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.985 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.985 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.985 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182427+0000", 2024-09-14T06:16:55.986 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7711999999999999e-05 2024-09-14T06:16:55.986 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.986 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.986 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.986 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.986 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.986 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13248 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.986 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:16:55.986 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.750248187, 2024-09-14T06:16:55.986 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966527999999997, 2024-09-14T06:16:55.986 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.986 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.986 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.986 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.987 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.987 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.987 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13248 2024-09-14T06:16:55.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.987 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.987 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.987 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.987 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:16:55.987 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.987 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.987 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.987 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:16:55.987 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.988 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.988 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.988 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:16:55.988 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:16:55.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.988 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.988 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.988 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100548+0000", 2024-09-14T06:16:55.988 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:16:55.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.988 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.988 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.989 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100557+0000", 2024-09-14T06:16:55.989 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:16:55.989 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.989 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.989 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.989 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100563+0000", 2024-09-14T06:16:55.989 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2519999999999996e-06 2024-09-14T06:16:55.989 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.989 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.989 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.989 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175572+0000", 2024-09-14T06:16:55.990 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075008459 2024-09-14T06:16:55.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.990 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.990 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.990 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175656+0000", 2024-09-14T06:16:55.990 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3906999999999995e-05 2024-09-14T06:16:55.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.990 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.990 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.990 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176010+0000", 2024-09-14T06:16:55.990 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00035447499999999998 2024-09-14T06:16:55.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.990 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.990 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.990 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178304+0000", 2024-09-14T06:16:55.991 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0022941379999999998 2024-09-14T06:16:55.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.991 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.991 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.991 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182467+0000", 2024-09-14T06:16:55.991 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041632020000000004 2024-09-14T06:16:55.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.991 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.991 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.992 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182491+0000", 2024-09-14T06:16:55.992 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3733000000000001e-05 2024-09-14T06:16:55.992 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.992 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.992 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.992 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182509+0000", 2024-09-14T06:16:55.992 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8113000000000001e-05 2024-09-14T06:16:55.992 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.992 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.992 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.992 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.992 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.993 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13249 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.993 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:16:55.993 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.750213252999998, 2024-09-14T06:16:55.993 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082037225000000005, 2024-09-14T06:16:55.993 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.993 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.993 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.993 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.993 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.993 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.993 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13249 2024-09-14T06:16:55.993 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.993 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.993 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.994 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.994 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:16:55.994 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.994 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.994 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:16:55.994 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.994 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:55.994 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:16:55.994 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4600000000000005e-07 2024-09-14T06:16:55.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:55.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100584+0000", 2024-09-14T06:16:55.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.947e-06 2024-09-14T06:16:55.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:55.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100593+0000", 2024-09-14T06:16:55.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9760000000000001e-06 2024-09-14T06:16:55.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:55.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100597+0000", 2024-09-14T06:16:55.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9689999999999996e-06 2024-09-14T06:16:55.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.996 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:55.996 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176461+0000", 2024-09-14T06:16:55.996 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075864132000000001 2024-09-14T06:16:55.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.996 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.996 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:55.996 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176516+0000", 2024-09-14T06:16:55.996 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4416999999999998e-05 2024-09-14T06:16:55.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.996 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.996 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:55.996 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176734+0000", 2024-09-14T06:16:55.996 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021859399999999999 2024-09-14T06:16:55.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.997 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.997 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:55.997 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178308+0000", 2024-09-14T06:16:55.997 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015733660000000001 2024-09-14T06:16:55.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.997 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.997 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:55.997 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182573+0000", 2024-09-14T06:16:55.997 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042649200000000002 2024-09-14T06:16:55.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.997 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.997 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:55.997 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182597+0000", 2024-09-14T06:16:55.997 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4355e-05 2024-09-14T06:16:55.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.998 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.998 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:55.998 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182615+0000", 2024-09-14T06:16:55.998 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8003e-05 2024-09-14T06:16:55.998 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.998 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:55.998 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:55.998 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.998 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.998 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13250 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:55.998 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:16:55.998 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.750048648, 2024-09-14T06:16:55.998 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081983890000000004, 2024-09-14T06:16:55.998 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:55.998 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:55.999 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:55.999 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:55.999 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:55.999 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:55.999 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13250 2024-09-14T06:16:55.999 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.999 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:55.999 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.999 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:55.999 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:16:55.999 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:55.999 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:55.999 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:55.999 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:55.999 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:16:55.999 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.000 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.000 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100743+0000", 2024-09-14T06:16:56.000 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:16:56.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.000 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.000 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100753+0000", 2024-09-14T06:16:56.000 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9979999999999998e-06 2024-09-14T06:16:56.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100772+0000", 2024-09-14T06:16:56.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8664e-05 2024-09-14T06:16:56.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100779+0000", 2024-09-14T06:16:56.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.046e-06 2024-09-14T06:16:56.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.002 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177049+0000", 2024-09-14T06:16:56.002 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076270819000000004 2024-09-14T06:16:56.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.002 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.002 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177140+0000", 2024-09-14T06:16:56.002 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0746000000000002e-05 2024-09-14T06:16:56.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.002 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:16:56.002 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177519+0000", 2024-09-14T06:16:56.002 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378656 2024-09-14T06:16:56.003 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.003 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.003 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:56.003 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178349+0000", 2024-09-14T06:16:56.003 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00083061599999999995 2024-09-14T06:16:56.003 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.003 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.003 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:56.003 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182679+0000", 2024-09-14T06:16:56.003 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043298379999999999 2024-09-14T06:16:56.003 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.003 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.003 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:56.003 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182705+0000", 2024-09-14T06:16:56.004 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5602999999999999e-05 2024-09-14T06:16:56.004 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.004 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.004 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.004 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182726+0000", 2024-09-14T06:16:56.004 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1297e-05 2024-09-14T06:16:56.004 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.004 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.004 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.004 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.004 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.004 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:56.006 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:16:56.007 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:56.002+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:56.009 INFO:teuthology.orchestra.run.smithi039.stdout:2808 write 0xe2ac06a thru 0xe2afbec (0x3b83 bytes) 2024-09-14T06:16:56.010 INFO:teuthology.orchestra.run.smithi039.stdout:2809 read 0x59d3a3c thru 0x59d9623 (0x5be8 bytes) 2024-09-14T06:16:56.010 INFO:teuthology.orchestra.run.smithi039.stdout:2810 read 0x76e4324 thru 0x76eeb8e (0xa86b bytes) 2024-09-14T06:16:56.010 INFO:teuthology.orchestra.run.smithi039.stdout:2811 trunc from 0xe2afbed to 0x7c43d21 2024-09-14T06:16:56.010 INFO:teuthology.orchestra.run.smithi039.stdout:2812 read 0x391671f thru 0x391f5fe (0x8ee0 bytes) 2024-09-14T06:16:56.010 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:56.010 INFO:teuthology.orchestra.run.smithi039.stdout:2814 trunc from 0x7c43d21 to 0x8ad9bd7 2024-09-14T06:16:56.010 INFO:teuthology.orchestra.run.smithi039.stdout:2815 write 0x3120d8c thru 0x3124dcc (0x4041 bytes) 2024-09-14T06:16:56.010 INFO:teuthology.orchestra.run.smithi039.stdout:2817 write 0xb9999d7 thru 0xb99afe6 (0x1610 bytes) 2024-09-14T06:16:56.010 INFO:teuthology.orchestra.run.smithi039.stdout:2818 writesame 0x9a413b1 thru 0x9a4319f data_size 0x1(0x1def bytes) 2024-09-14T06:16:56.010 INFO:teuthology.orchestra.run.smithi039.stdout:2819 write 0x3538f98 thru 0x353e120 (0x5189 bytes) 2024-09-14T06:16:56.010 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:56.010 INFO:teuthology.orchestra.run.smithi039.stdout:2821 write 0x51dc786 thru 0x51e8721 (0xbf9c bytes) 2024-09-14T06:16:56.010 INFO:teuthology.orchestra.run.smithi039.stdout:2823 punch from 0x91dfa19 to 0x91e6a86, (0x706d bytes) 2024-09-14T06:16:56.010 INFO:teuthology.orchestra.run.smithi039.stdout:2824 write 0x2712eff thru 0x2715a44 (0x2b46 bytes) 2024-09-14T06:16:56.011 INFO:teuthology.orchestra.run.smithi039.stdout:2825 read 0x9616681 thru 0x9620617 (0x9f97 bytes) 2024-09-14T06:16:56.011 INFO:teuthology.orchestra.run.smithi039.stdout:2826 punch from 0x38483bd to 0x3850bed, (0x8830 bytes) 2024-09-14T06:16:56.011 INFO:teuthology.orchestra.run.smithi039.stdout:2827 writesame 0x199c4a9 thru 0x199f836 data_size 0x1(0x338e bytes) 2024-09-14T06:16:56.011 INFO:teuthology.orchestra.run.smithi039.stdout:2828 writesame 0x817cc8e thru 0x817db2b data_size 0x1(0xe9e bytes) 2024-09-14T06:16:56.011 INFO:teuthology.orchestra.run.smithi039.stdout:2829 trunc from 0xb99afe7 to 0x8b33751 2024-09-14T06:16:56.011 INFO:teuthology.orchestra.run.smithi039.stdout:2830 read 0x67f5ab7 thru 0x67f68e3 (0xe2d bytes) 2024-09-14T06:16:56.011 INFO:teuthology.orchestra.run.smithi039.stdout:2831 punch from 0x169b1ff to 0x16a2722, (0x7523 bytes) 2024-09-14T06:16:56.011 INFO:teuthology.orchestra.run.smithi039.stdout:2833 write 0xbe276a9 thru 0xbe375ea (0xff42 bytes) 2024-09-14T06:16:56.011 INFO:teuthology.orchestra.run.smithi039.stdout:2834 punch from 0x3807794 to 0x380d4e0, (0x5d4c bytes) 2024-09-14T06:16:56.011 INFO:teuthology.orchestra.run.smithi039.stdout:2835 write 0x415b594 thru 0x415d202 (0x1c6f bytes) 2024-09-14T06:16:56.011 INFO:teuthology.orchestra.run.smithi039.stdout:2836 punch from 0x60a9f1a to 0x60b64d6, (0xc5bc bytes) 2024-09-14T06:16:56.011 INFO:teuthology.orchestra.run.smithi039.stdout:2838 write 0x5505965 thru 0x5510c37 (0xb2d3 bytes) 2024-09-14T06:16:56.011 INFO:teuthology.orchestra.run.smithi039.stdout:2839 read 0x343f848 thru 0x3441c1d (0x23d6 bytes) 2024-09-14T06:16:56.012 INFO:teuthology.orchestra.run.smithi039.stdout:2841 read 0x9a62deb thru 0x9a67fe6 (0x51fc bytes) 2024-09-14T06:16:56.012 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:56.012 INFO:teuthology.orchestra.run.smithi039.stdout:2843 write 0xd0f1b29 thru 0xd0fdc61 (0xc139 bytes) 2024-09-14T06:16:56.012 INFO:teuthology.orchestra.run.smithi039.stdout:2844 read 0x2b12c20 thru 0x2b1f796 (0xcb77 bytes) 2024-09-14T06:16:56.012 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:56.013 INFO:teuthology.orchestra.run.smithi039.stdout:2847 read 0x51ec1fc thru 0x51faf5e (0xed63 bytes) 2024-09-14T06:16:56.013 INFO:teuthology.orchestra.run.smithi039.stdout:2850 write 0xc5843bc thru 0xc588053 (0x3c98 bytes) 2024-09-14T06:16:56.026 INFO:teuthology.orchestra.run.smithi039.stdout:2852 trunc from 0xd0fdc62 to 0xa0de04c 2024-09-14T06:16:56.028 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-09-14T06:16:56.060 INFO:teuthology.orchestra.run.smithi039.stdout:2854 trunc from 0xa0de04c to 0x801436a 2024-09-14T06:16:56.085 INFO:teuthology.orchestra.run.smithi039.stdout:2855 write 0xd4d8cef thru 0xd4e0e06 (0x8118 bytes) 2024-09-14T06:16:56.096 INFO:teuthology.orchestra.run.smithi039.stdout:2856 trunc from 0xd4e0e07 to 0xa1aba17 2024-09-14T06:16:56.106 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:56.103+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:56.122 INFO:teuthology.orchestra.run.smithi039.stdout:2857 write 0xdab468e thru 0xdab7402 (0x2d75 bytes) 2024-09-14T06:16:56.126 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:56.127 INFO:teuthology.orchestra.run.smithi039.stdout:2859 write 0x4e2acc8 thru 0x4e33d55 (0x908e bytes) 2024-09-14T06:16:56.129 INFO:teuthology.orchestra.run.smithi039.stdout:2860 read 0xba76b31 thru 0xba7731a (0x7ea bytes) 2024-09-14T06:16:56.129 INFO:teuthology.orchestra.run.smithi039.stdout:2861 write 0x581a657 thru 0x5821c63 (0x760d bytes) 2024-09-14T06:16:56.134 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:56.134 INFO:teuthology.orchestra.run.smithi039.stdout:2863 write 0x7da5e85 thru 0x7db00da (0xa256 bytes) 2024-09-14T06:16:56.140 INFO:teuthology.orchestra.run.smithi039.stdout:2865 writesame 0x16569fe thru 0x1663d7b data_size 0x6b(0xd37e bytes) 2024-09-14T06:16:56.143 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:56.143 INFO:teuthology.orchestra.run.smithi039.stdout:2868 read 0x87dabb7 thru 0x87e4236 (0x9680 bytes) 2024-09-14T06:16:56.143 INFO:teuthology.orchestra.run.smithi039.stdout:2869 write 0x3a7970b thru 0x3a802c0 (0x6bb6 bytes) 2024-09-14T06:16:56.146 INFO:teuthology.orchestra.run.smithi039.stdout:2870 read 0x39e1b0c thru 0x39e81d3 (0x66c8 bytes) 2024-09-14T06:16:56.146 INFO:teuthology.orchestra.run.smithi039.stdout:2872 punch from 0xa4b19df to 0xa4baedf, (0x9500 bytes) 2024-09-14T06:16:56.146 INFO:teuthology.orchestra.run.smithi039.stdout:2873 write 0xc5ff026 thru 0xc60bba5 (0xcb80 bytes) 2024-09-14T06:16:56.172 INFO:teuthology.orchestra.run.smithi039.stdout:2876 trunc from 0xdab7403 to 0x3a74c78 2024-09-14T06:16:56.189 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:56.189 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:16:56.189 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.189 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:28223 3.1e 3:7a61f838:::rbd_data.1152ff705991.00000000000000c8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-14T06:16:56.189 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:56.187453+0000", 2024-09-14T06:16:56.189 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.002798287, 2024-09-14T06:16:56.189 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0028300450000000002, 2024-09-14T06:16:56.189 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.189 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.189 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:56.189 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.190 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.190 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.190 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 28223 2024-09-14T06:16:56.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.190 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.190 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.190 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.187453+0000", 2024-09-14T06:16:56.190 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.190 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.190 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.187453+0000", 2024-09-14T06:16:56.190 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.191 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.187455+0000", 2024-09-14T06:16:56.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.973e-06 2024-09-14T06:16:56.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.191 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.187465+0000", 2024-09-14T06:16:56.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5470000000000002e-06 2024-09-14T06:16:56.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.191 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.187493+0000", 2024-09-14T06:16:56.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7963999999999999e-05 2024-09-14T06:16:56.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.187512+0000", 2024-09-14T06:16:56.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8813000000000001e-05 2024-09-14T06:16:56.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.187660+0000", 2024-09-14T06:16:56.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014876200000000001 2024-09-14T06:16:56.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.187945+0000", 2024-09-14T06:16:56.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028470299999999997 2024-09-14T06:16:56.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.193 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:16:56.193 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.188311+0000", 2024-09-14T06:16:56.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036649000000000002 2024-09-14T06:16:56.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.193 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:56.193 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.189290+0000", 2024-09-14T06:16:56.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0009781569999999999 2024-09-14T06:16:56.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.193 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:56.194 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.190201+0000", 2024-09-14T06:16:56.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00091096299999999999 2024-09-14T06:16:56.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.194 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:56.194 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.190247+0000", 2024-09-14T06:16:56.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6885999999999998e-05 2024-09-14T06:16:56.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.194 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.194 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.190283+0000", 2024-09-14T06:16:56.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5787000000000001e-05 2024-09-14T06:16:56.194 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.194 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.194 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop_reply(client.4434.0:28223 3.1e e50/29 ondisk, result = 0)", 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:56.190028+0000", 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00022367900000000001, 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00035038199999999999, 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.190028+0000", 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.190028+0000", 2024-09-14T06:16:56.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.196 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.190030+0000", 2024-09-14T06:16:56.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8500000000000001e-06 2024-09-14T06:16:56.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.196 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.190042+0000", 2024-09-14T06:16:56.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2075999999999999e-05 2024-09-14T06:16:56.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.196 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.190079+0000", 2024-09-14T06:16:56.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7450999999999998e-05 2024-09-14T06:16:56.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.190106+0000", 2024-09-14T06:16:56.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6480999999999999e-05 2024-09-14T06:16:56.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.190184+0000", 2024-09-14T06:16:56.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8483000000000001e-05 2024-09-14T06:16:56.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.198 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:56.190194+0000", 2024-09-14T06:16:56.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5990000000000008e-06 2024-09-14T06:16:56.198 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.198 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.198 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.198 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.198 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-09-14T06:16:56.198 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 2 2024-09-14T06:16:56.198 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:56.203 INFO:teuthology.orchestra.run.smithi039.stdout:2877 read 0x16b01df thru 0x16bfdb9 (0xfbdb bytes) 2024-09-14T06:16:56.204 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:56.204 INFO:teuthology.orchestra.run.smithi039.stdout:2879 punch from 0x18d41c9 to 0x18d83d0, (0x4207 bytes) 2024-09-14T06:16:56.204 INFO:teuthology.orchestra.run.smithi039.stdout:2880 read 0xd71a13 thru 0xd79861 (0x7e4f bytes) 2024-09-14T06:16:56.205 INFO:teuthology.orchestra.run.smithi039.stdout:2881 read 0x10cd167 thru 0x10d415b (0x6ff5 bytes) 2024-09-14T06:16:56.206 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:16:56.207 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:56.206+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:56.232 INFO:teuthology.orchestra.run.smithi039.stdout:2882 write 0xecff986 thru 0xed08afe (0x9179 bytes) 2024-09-14T06:16:56.240 INFO:teuthology.orchestra.run.smithi039.stdout:2884 read 0x3e42785 thru 0x3e4e85e (0xc0da bytes) 2024-09-14T06:16:56.240 INFO:teuthology.orchestra.run.smithi039.stdout:2885 read 0x6041181 thru 0x604a57e (0x93fe bytes) 2024-09-14T06:16:56.240 INFO:teuthology.orchestra.run.smithi039.stdout:2886 write 0x783e639 thru 0x78469fe (0x83c6 bytes) 2024-09-14T06:16:56.246 INFO:teuthology.orchestra.run.smithi039.stdout:2887 write 0xc53ff thru 0xc560e (0x210 bytes) 2024-09-14T06:16:56.248 INFO:teuthology.orchestra.run.smithi039.stdout:2888 punch from 0x423ca93 to 0x4240deb, (0x4358 bytes) 2024-09-14T06:16:56.248 INFO:teuthology.orchestra.run.smithi039.stdout:2889 write 0xc80e99 thru 0xc8e5e8 (0xd750 bytes) 2024-09-14T06:16:56.250 INFO:teuthology.orchestra.run.smithi039.stdout:2890 clone 15 order 25 su 262144 sc 12 2024-09-14T06:16:56.308 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:56.306+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:56.326 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:56.326 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:56.326 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:16:56.326 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:16:56.326 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:56.340 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:16:56.427 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:56.426+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:56.508 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:56.508 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:16:56.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:16:56.508 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:16:56.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.508 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16116 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 851968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.508 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654230+0000", 2024-09-14T06:16:56.508 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.852330211000002, 2024-09-14T06:16:56.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.033897548, 2024-09-14T06:16:56.509 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.509 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.509 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.509 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.509 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.509 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.509 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16116 2024-09-14T06:16:56.509 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.509 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.509 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.509 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654230+0000", 2024-09-14T06:16:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654230+0000", 2024-09-14T06:16:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654231+0000", 2024-09-14T06:16:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2699999999999999e-07 2024-09-14T06:16:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.510 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654233+0000", 2024-09-14T06:16:56.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6259999999999999e-06 2024-09-14T06:16:56.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.511 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.511 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654243+0000", 2024-09-14T06:16:56.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5969999999999998e-06 2024-09-14T06:16:56.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.511 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.512 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654250+0000", 2024-09-14T06:16:56.512 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.007e-06 2024-09-14T06:16:56.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.512 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.512 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.687631+0000", 2024-09-14T06:16:56.512 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.033381704999999998 2024-09-14T06:16:56.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.512 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.512 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.687852+0000", 2024-09-14T06:16:56.512 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022096099999999999 2024-09-14T06:16:56.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.513 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.513 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688127+0000", 2024-09-14T06:16:56.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00027492499999999997 2024-09-14T06:16:56.513 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.513 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.513 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.513 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16117 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.513 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654259+0000", 2024-09-14T06:16:56.513 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.852300767999999, 2024-09-14T06:16:56.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.034316067999999998, 2024-09-14T06:16:56.513 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.514 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.514 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.514 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.514 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.514 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.514 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16117 2024-09-14T06:16:56.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.514 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.514 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.514 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654259+0000", 2024-09-14T06:16:56.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654259+0000", 2024-09-14T06:16:56.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654260+0000", 2024-09-14T06:16:56.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1699999999999997e-07 2024-09-14T06:16:56.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.516 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.516 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654263+0000", 2024-09-14T06:16:56.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6029999999999999e-06 2024-09-14T06:16:56.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.516 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.516 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654272+0000", 2024-09-14T06:16:56.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1600000000000004e-06 2024-09-14T06:16:56.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.517 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.517 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654279+0000", 2024-09-14T06:16:56.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9469999999999997e-06 2024-09-14T06:16:56.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.517 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.517 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.517 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688248+0000", 2024-09-14T06:16:56.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.033969223999999999 2024-09-14T06:16:56.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.517 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.517 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.517 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688387+0000", 2024-09-14T06:16:56.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013955200000000001 2024-09-14T06:16:56.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.518 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.518 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688575+0000", 2024-09-14T06:16:56.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000187865 2024-09-14T06:16:56.518 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.518 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.518 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.518 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16118 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.518 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654288+0000", 2024-09-14T06:16:56.518 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.852272381999999, 2024-09-14T06:16:56.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.034884971000000001, 2024-09-14T06:16:56.518 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.518 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.518 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.519 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.519 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.519 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.519 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16118 2024-09-14T06:16:56.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.519 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654288+0000", 2024-09-14T06:16:56.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654288+0000", 2024-09-14T06:16:56.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.520 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.520 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.520 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654288+0000", 2024-09-14T06:16:56.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1399999999999996e-07 2024-09-14T06:16:56.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.520 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.520 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.520 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654291+0000", 2024-09-14T06:16:56.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4700000000000001e-06 2024-09-14T06:16:56.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.520 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.520 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.520 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654300+0000", 2024-09-14T06:16:56.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:16:56.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654307+0000", 2024-09-14T06:16:56.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1430000000000001e-06 2024-09-14T06:16:56.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688682+0000", 2024-09-14T06:16:56.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.034375304000000002 2024-09-14T06:16:56.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688897+0000", 2024-09-14T06:16:56.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021480499999999999 2024-09-14T06:16:56.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689173+0000", 2024-09-14T06:16:56.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00027551499999999999 2024-09-14T06:16:56.522 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.522 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.522 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.522 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16119 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.522 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654316+0000", 2024-09-14T06:16:56.522 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.852243918999999, 2024-09-14T06:16:56.523 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035292513999999997, 2024-09-14T06:16:56.523 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.523 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.523 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.523 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.523 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.523 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.523 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16119 2024-09-14T06:16:56.523 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.523 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.523 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.523 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654316+0000", 2024-09-14T06:16:56.524 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.524 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.524 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654316+0000", 2024-09-14T06:16:56.524 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654317+0000", 2024-09-14T06:16:56.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.37e-07 2024-09-14T06:16:56.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654319+0000", 2024-09-14T06:16:56.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6469999999999998e-06 2024-09-14T06:16:56.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654332+0000", 2024-09-14T06:16:56.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2330000000000001e-05 2024-09-14T06:16:56.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654339+0000", 2024-09-14T06:16:56.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2130000000000002e-06 2024-09-14T06:16:56.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689282+0000", 2024-09-14T06:16:56.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.034942742999999998 2024-09-14T06:16:56.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689424+0000", 2024-09-14T06:16:56.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014175900000000001 2024-09-14T06:16:56.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.527 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689609+0000", 2024-09-14T06:16:56.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00018508500000000001 2024-09-14T06:16:56.527 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.527 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.527 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.527 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16120 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.527 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654348+0000", 2024-09-14T06:16:56.527 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.852211671999999, 2024-09-14T06:16:56.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035856590000000001, 2024-09-14T06:16:56.528 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.528 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.528 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.528 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.528 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.528 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.528 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16120 2024-09-14T06:16:56.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.528 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654348+0000", 2024-09-14T06:16:56.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.529 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.529 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654348+0000", 2024-09-14T06:16:56.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.529 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.529 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654349+0000", 2024-09-14T06:16:56.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2699999999999999e-07 2024-09-14T06:16:56.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.529 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.530 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654352+0000", 2024-09-14T06:16:56.530 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6400000000000001e-06 2024-09-14T06:16:56.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.530 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.530 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.530 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654363+0000", 2024-09-14T06:16:56.530 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1715999999999999e-05 2024-09-14T06:16:56.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.530 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.530 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.530 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654371+0000", 2024-09-14T06:16:56.530 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5769999999999996e-06 2024-09-14T06:16:56.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689718+0000", 2024-09-14T06:16:56.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035347376999999999 2024-09-14T06:16:56.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689941+0000", 2024-09-14T06:16:56.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022233499999999999 2024-09-14T06:16:56.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.690205+0000", 2024-09-14T06:16:56.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00026421799999999998 2024-09-14T06:16:56.532 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.532 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.532 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.532 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16121 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 3145728~7288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.532 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654381+0000", 2024-09-14T06:16:56.532 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.852179355000001, 2024-09-14T06:16:56.533 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.036696549000000002, 2024-09-14T06:16:56.533 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.533 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.533 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.533 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.533 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.533 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.533 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16121 2024-09-14T06:16:56.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.534 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.534 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.534 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654381+0000", 2024-09-14T06:16:56.534 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.534 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.534 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654381+0000", 2024-09-14T06:16:56.534 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.534 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.535 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654381+0000", 2024-09-14T06:16:56.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9999999999999997e-07 2024-09-14T06:16:56.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.535 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654384+0000", 2024-09-14T06:16:56.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8499999999999998e-06 2024-09-14T06:16:56.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.535 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654404+0000", 2024-09-14T06:16:56.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9355999999999999e-05 2024-09-14T06:16:56.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654417+0000", 2024-09-14T06:16:56.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.295e-05 2024-09-14T06:16:56.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.690327+0000", 2024-09-14T06:16:56.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035910899000000003 2024-09-14T06:16:56.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.690557+0000", 2024-09-14T06:16:56.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022935499999999999 2024-09-14T06:16:56.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.691077+0000", 2024-09-14T06:16:56.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00052043900000000004 2024-09-14T06:16:56.537 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.537 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.537 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.537 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16122 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.537 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654428+0000", 2024-09-14T06:16:56.538 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.852132009000002, 2024-09-14T06:16:56.538 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.037792064, 2024-09-14T06:16:56.538 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.538 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.538 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.538 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.538 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.538 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.538 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16122 2024-09-14T06:16:56.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.538 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.538 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.539 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654428+0000", 2024-09-14T06:16:56.539 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.539 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.539 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.539 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654428+0000", 2024-09-14T06:16:56.539 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.539 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.539 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.539 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654429+0000", 2024-09-14T06:16:56.539 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8999999999999995e-07 2024-09-14T06:16:56.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.539 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.539 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654432+0000", 2024-09-14T06:16:56.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0130000000000002e-06 2024-09-14T06:16:56.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654442+0000", 2024-09-14T06:16:56.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9199999999999999e-06 2024-09-14T06:16:56.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654449+0000", 2024-09-14T06:16:56.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:16:56.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.541 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.541 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.691192+0000", 2024-09-14T06:16:56.541 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.036742958999999999 2024-09-14T06:16:56.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.541 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.541 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.541 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.691445+0000", 2024-09-14T06:16:56.541 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000253608 2024-09-14T06:16:56.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.541 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.541 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.541 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692220+0000", 2024-09-14T06:16:56.541 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00077489400000000004 2024-09-14T06:16:56.541 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.542 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.542 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.542 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16123 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.542 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654458+0000", 2024-09-14T06:16:56.542 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.85210202, 2024-09-14T06:16:56.542 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038257820999999997, 2024-09-14T06:16:56.542 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.542 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.542 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.542 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.542 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.542 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.542 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16123 2024-09-14T06:16:56.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.543 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654458+0000", 2024-09-14T06:16:56.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654458+0000", 2024-09-14T06:16:56.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654459+0000", 2024-09-14T06:16:56.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0699999999999996e-07 2024-09-14T06:16:56.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.544 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.544 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654461+0000", 2024-09-14T06:16:56.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.633e-06 2024-09-14T06:16:56.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.544 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.544 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654470+0000", 2024-09-14T06:16:56.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0440000000000001e-06 2024-09-14T06:16:56.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.545 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.545 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654496+0000", 2024-09-14T06:16:56.545 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5279999999999999e-05 2024-09-14T06:16:56.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.545 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.545 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692393+0000", 2024-09-14T06:16:56.545 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.037897532999999997 2024-09-14T06:16:56.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.545 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.546 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692526+0000", 2024-09-14T06:16:56.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013326899999999999 2024-09-14T06:16:56.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.546 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.546 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692716+0000", 2024-09-14T06:16:56.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00018935499999999999 2024-09-14T06:16:56.546 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.546 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.546 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.546 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16124 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.546 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654506+0000", 2024-09-14T06:16:56.547 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.852053683000001, 2024-09-14T06:16:56.547 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038503725000000003, 2024-09-14T06:16:56.547 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.547 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.547 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.547 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.547 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.547 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.547 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16124 2024-09-14T06:16:56.547 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.547 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.547 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.547 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.548 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654506+0000", 2024-09-14T06:16:56.548 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.548 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.548 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.548 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654506+0000", 2024-09-14T06:16:56.548 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.548 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.548 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.548 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654507+0000", 2024-09-14T06:16:56.548 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1299999999999999e-07 2024-09-14T06:16:56.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654518+0000", 2024-09-14T06:16:56.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0696999999999999e-05 2024-09-14T06:16:56.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654531+0000", 2024-09-14T06:16:56.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3596e-05 2024-09-14T06:16:56.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654539+0000", 2024-09-14T06:16:56.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6170000000000001e-06 2024-09-14T06:16:56.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692766+0000", 2024-09-14T06:16:56.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038227401000000001 2024-09-14T06:16:56.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692869+0000", 2024-09-14T06:16:56.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000102216 2024-09-14T06:16:56.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693010+0000", 2024-09-14T06:16:56.551 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000141385 2024-09-14T06:16:56.551 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.551 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.551 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.551 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.551 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.551 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16125 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.551 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654549+0000", 2024-09-14T06:16:56.551 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.852010527000001, 2024-09-14T06:16:56.551 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038734667, 2024-09-14T06:16:56.551 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.551 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.551 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.551 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.552 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.552 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.552 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16125 2024-09-14T06:16:56.552 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.552 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.552 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.552 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.552 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654549+0000", 2024-09-14T06:16:56.552 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.552 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.552 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.552 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.552 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654549+0000", 2024-09-14T06:16:56.553 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.553 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.553 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.553 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.553 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654550+0000", 2024-09-14T06:16:56.553 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6700000000000003e-07 2024-09-14T06:16:56.553 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.553 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.553 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.553 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654562+0000", 2024-09-14T06:16:56.553 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1926000000000001e-05 2024-09-14T06:16:56.553 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.553 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.553 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654575+0000", 2024-09-14T06:16:56.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3217000000000001e-05 2024-09-14T06:16:56.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654583+0000", 2024-09-14T06:16:56.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3660000000000003e-06 2024-09-14T06:16:56.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693055+0000", 2024-09-14T06:16:56.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038471752999999997 2024-09-14T06:16:56.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.555 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.555 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693152+0000", 2024-09-14T06:16:56.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7548999999999997e-05 2024-09-14T06:16:56.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.555 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.555 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.555 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693284+0000", 2024-09-14T06:16:56.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000132089 2024-09-14T06:16:56.555 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.555 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.555 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.555 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.555 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16126 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 5439488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.556 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654595+0000", 2024-09-14T06:16:56.556 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.851964824, 2024-09-14T06:16:56.556 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038961922000000003, 2024-09-14T06:16:56.556 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.556 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.556 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.556 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.556 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.556 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.556 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16126 2024-09-14T06:16:56.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.556 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.557 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.557 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654595+0000", 2024-09-14T06:16:56.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.557 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.557 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.557 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654595+0000", 2024-09-14T06:16:56.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.557 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.557 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.557 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654596+0000", 2024-09-14T06:16:56.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7700000000000004e-07 2024-09-14T06:16:56.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.558 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.558 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654600+0000", 2024-09-14T06:16:56.558 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.216e-06 2024-09-14T06:16:56.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.558 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.558 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654610+0000", 2024-09-14T06:16:56.558 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7070000000000004e-06 2024-09-14T06:16:56.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.558 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.558 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654617+0000", 2024-09-14T06:16:56.559 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2200000000000003e-06 2024-09-14T06:16:56.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.559 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.559 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.559 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693327+0000", 2024-09-14T06:16:56.559 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038709427999999997 2024-09-14T06:16:56.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.559 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.559 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.559 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693426+0000", 2024-09-14T06:16:56.559 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9401999999999995e-05 2024-09-14T06:16:56.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.559 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.559 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693557+0000", 2024-09-14T06:16:56.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013117200000000001 2024-09-14T06:16:56.560 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.560 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.560 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.560 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.560 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.560 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16127 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 5898240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.560 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654627+0000", 2024-09-14T06:16:56.560 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.851933390999999, 2024-09-14T06:16:56.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.040234709, 2024-09-14T06:16:56.560 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.560 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.561 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.561 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.561 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.561 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.561 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16127 2024-09-14T06:16:56.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.561 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.561 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.561 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654627+0000", 2024-09-14T06:16:56.561 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.562 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.562 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654627+0000", 2024-09-14T06:16:56.562 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.562 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.562 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654627+0000", 2024-09-14T06:16:56.562 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4700000000000001e-07 2024-09-14T06:16:56.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.563 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654630+0000", 2024-09-14T06:16:56.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8600000000000001e-06 2024-09-14T06:16:56.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.563 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.563 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654639+0000", 2024-09-14T06:16:56.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2599999999999994e-06 2024-09-14T06:16:56.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.563 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.563 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654647+0000", 2024-09-14T06:16:56.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2269999999999996e-06 2024-09-14T06:16:56.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.564 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.564 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693851+0000", 2024-09-14T06:16:56.564 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.039203937000000001 2024-09-14T06:16:56.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.564 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.564 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.694465+0000", 2024-09-14T06:16:56.564 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00061404799999999996 2024-09-14T06:16:56.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.564 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.565 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.694861+0000", 2024-09-14T06:16:56.565 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00039662999999999998 2024-09-14T06:16:56.565 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.565 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.565 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.565 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.565 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.565 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16128 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.565 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654656+0000", 2024-09-14T06:16:56.565 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.851903927999999, 2024-09-14T06:16:56.565 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.040797799000000003, 2024-09-14T06:16:56.565 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.565 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.566 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.566 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.566 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.566 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.566 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16128 2024-09-14T06:16:56.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.566 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.566 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.566 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654656+0000", 2024-09-14T06:16:56.566 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654656+0000", 2024-09-14T06:16:56.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654657+0000", 2024-09-14T06:16:56.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2699999999999999e-07 2024-09-14T06:16:56.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654659+0000", 2024-09-14T06:16:56.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6400000000000001e-06 2024-09-14T06:16:56.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.568 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.568 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.568 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654669+0000", 2024-09-14T06:16:56.568 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1730000000000001e-06 2024-09-14T06:16:56.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.568 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.568 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.568 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654685+0000", 2024-09-14T06:16:56.568 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6237000000000001e-05 2024-09-14T06:16:56.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.569 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.569 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.694952+0000", 2024-09-14T06:16:56.569 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.040267078999999997 2024-09-14T06:16:56.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.569 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.569 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695174+0000", 2024-09-14T06:16:56.569 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000221745 2024-09-14T06:16:56.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.569 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.569 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695454+0000", 2024-09-14T06:16:56.570 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000280198 2024-09-14T06:16:56.570 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.570 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.570 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.570 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16129 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.570 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654697+0000", 2024-09-14T06:16:56.570 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.851862831999998, 2024-09-14T06:16:56.570 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.041164486, 2024-09-14T06:16:56.570 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.570 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.570 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.570 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.571 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.571 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.571 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16129 2024-09-14T06:16:56.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.571 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.571 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.571 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.571 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654697+0000", 2024-09-14T06:16:56.571 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.571 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.571 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.571 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654697+0000", 2024-09-14T06:16:56.572 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.572 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.572 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.572 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654698+0000", 2024-09-14T06:16:56.573 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:16:56.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.573 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.573 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.573 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654701+0000", 2024-09-14T06:16:56.573 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.756e-06 2024-09-14T06:16:56.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.574 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654710+0000", 2024-09-14T06:16:56.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7669999999999998e-06 2024-09-14T06:16:56.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.574 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654718+0000", 2024-09-14T06:16:56.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.097e-06 2024-09-14T06:16:56.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.574 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695552+0000", 2024-09-14T06:16:56.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.040834148000000001 2024-09-14T06:16:56.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695690+0000", 2024-09-14T06:16:56.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000138479 2024-09-14T06:16:56.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695862+0000", 2024-09-14T06:16:56.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017150499999999999 2024-09-14T06:16:56.575 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.575 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.576 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.576 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.576 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.576 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16130 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.576 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654727+0000", 2024-09-14T06:16:56.576 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.851833025000001, 2024-09-14T06:16:56.576 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.041755107999999999, 2024-09-14T06:16:56.576 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.576 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.576 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.576 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.576 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.576 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.577 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16130 2024-09-14T06:16:56.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.577 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.577 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.577 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.577 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654727+0000", 2024-09-14T06:16:56.577 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.577 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.577 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.577 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654727+0000", 2024-09-14T06:16:56.577 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.578 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.578 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.578 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654728+0000", 2024-09-14T06:16:56.578 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6300000000000004e-07 2024-09-14T06:16:56.580 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.581 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.581 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654730+0000", 2024-09-14T06:16:56.581 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7439999999999999e-06 2024-09-14T06:16:56.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654742+0000", 2024-09-14T06:16:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1036000000000001e-05 2024-09-14T06:16:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654748+0000", 2024-09-14T06:16:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9199999999999998e-06 2024-09-14T06:16:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695969+0000", 2024-09-14T06:16:56.582 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.041220672 2024-09-14T06:16:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696192+0000", 2024-09-14T06:16:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022279799999999999 2024-09-14T06:16:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696482+0000", 2024-09-14T06:16:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00029017499999999999 2024-09-14T06:16:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.583 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16131 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.584 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654758+0000", 2024-09-14T06:16:56.584 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.851802185, 2024-09-14T06:16:56.584 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.042069361999999999, 2024-09-14T06:16:56.584 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.584 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.584 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.584 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.584 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.584 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.584 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16131 2024-09-14T06:16:56.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.584 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.585 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654758+0000", 2024-09-14T06:16:56.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.585 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654758+0000", 2024-09-14T06:16:56.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.585 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654759+0000", 2024-09-14T06:16:56.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3e-07 2024-09-14T06:16:56.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.586 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.586 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.586 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654761+0000", 2024-09-14T06:16:56.586 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7599999999999998e-06 2024-09-14T06:16:56.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.586 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.586 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.586 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654773+0000", 2024-09-14T06:16:56.586 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1216999999999999e-05 2024-09-14T06:16:56.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.586 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.586 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.586 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654780+0000", 2024-09-14T06:16:56.587 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3529999999999997e-06 2024-09-14T06:16:56.587 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.587 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.587 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.587 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696524+0000", 2024-09-14T06:16:56.587 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.041744594000000003 2024-09-14T06:16:56.587 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.587 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.587 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.587 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696652+0000", 2024-09-14T06:16:56.587 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000127866 2024-09-14T06:16:56.587 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.588 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.588 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.588 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696827+0000", 2024-09-14T06:16:56.588 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000174842 2024-09-14T06:16:56.588 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.588 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.588 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.588 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.588 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.588 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16132 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.588 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654840+0000", 2024-09-14T06:16:56.588 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.851719656, 2024-09-14T06:16:56.588 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.042362179999999999, 2024-09-14T06:16:56.589 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.589 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.589 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.589 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.589 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.589 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.589 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16132 2024-09-14T06:16:56.589 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.589 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.589 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.589 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.589 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654840+0000", 2024-09-14T06:16:56.589 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.589 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.590 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.590 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.590 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654840+0000", 2024-09-14T06:16:56.590 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.590 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.590 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.590 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654841+0000", 2024-09-14T06:16:56.590 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9300000000000006e-07 2024-09-14T06:16:56.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.590 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.590 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.590 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654845+0000", 2024-09-14T06:16:56.591 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.18e-06 2024-09-14T06:16:56.591 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.591 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.591 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.591 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654867+0000", 2024-09-14T06:16:56.591 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.27e-05 2024-09-14T06:16:56.591 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.591 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.591 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.591 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654892+0000", 2024-09-14T06:16:56.591 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4284e-05 2024-09-14T06:16:56.591 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.591 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.591 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.592 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696907+0000", 2024-09-14T06:16:56.592 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.042015957999999999 2024-09-14T06:16:56.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.592 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.592 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.592 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.697032+0000", 2024-09-14T06:16:56.592 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012415899999999999 2024-09-14T06:16:56.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.593 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.593 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.593 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.697203+0000", 2024-09-14T06:16:56.593 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017090599999999999 2024-09-14T06:16:56.593 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.593 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.593 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.593 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.593 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.593 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16181 3.0 3:067f695c:::rbd_data.1152fc09e7e6.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:56.593 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.757965+0000", 2024-09-14T06:16:56.594 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.748595332000001, 2024-09-14T06:16:56.594 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065959498000000005, 2024-09-14T06:16:56.594 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.594 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.594 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:56.594 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.594 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.594 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.594 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16181 2024-09-14T06:16:56.594 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.594 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.594 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.594 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.595 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.757965+0000", 2024-09-14T06:16:56.595 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.595 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.595 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.595 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.757965+0000", 2024-09-14T06:16:56.595 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.595 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.595 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.595 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.757967+0000", 2024-09-14T06:16:56.595 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6869999999999999e-06 2024-09-14T06:16:56.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.595 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.596 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.596 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.757984+0000", 2024-09-14T06:16:56.596 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.658e-05 2024-09-14T06:16:56.596 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.596 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.596 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.596 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.758025+0000", 2024-09-14T06:16:56.596 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1158999999999997e-05 2024-09-14T06:16:56.596 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.596 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.596 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.596 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.758049+0000", 2024-09-14T06:16:56.596 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3442999999999999e-05 2024-09-14T06:16:56.596 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.597 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.597 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.597 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.758200+0000", 2024-09-14T06:16:56.597 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015118900000000001 2024-09-14T06:16:56.597 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.597 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.597 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.597 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.758340+0000", 2024-09-14T06:16:56.597 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000140722 2024-09-14T06:16:56.597 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.597 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.597 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 7", 2024-09-14T06:16:56.597 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.769990+0000", 2024-09-14T06:16:56.598 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011649286999999999 2024-09-14T06:16:56.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.598 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.598 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:16:56.598 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.823608+0000", 2024-09-14T06:16:56.598 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.053617797000000002 2024-09-14T06:16:56.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.598 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.598 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:56.598 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.823740+0000", 2024-09-14T06:16:56.598 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000132132 2024-09-14T06:16:56.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.598 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.598 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:16:56.598 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.823843+0000", 2024-09-14T06:16:56.598 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000103712 2024-09-14T06:16:56.599 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.599 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.599 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.599 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.823924+0000", 2024-09-14T06:16:56.599 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0790000000000004e-05 2024-09-14T06:16:56.599 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.599 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.599 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.599 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.599 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.599 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:18155 3.17 3:e91637d8:::rbd_data.115232f4ac2b.0000000000000000:a [sparse-read 5767168~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T06:16:56.599 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:41.255599+0000", 2024-09-14T06:16:56.599 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.25096126, 2024-09-14T06:16:56.599 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035557431, 2024-09-14T06:16:56.600 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.600 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.600 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.600 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.600 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.600 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.600 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 18155 2024-09-14T06:16:56.600 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.600 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.600 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.600 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.600 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255599+0000", 2024-09-14T06:16:56.600 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.600 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.601 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.601 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255599+0000", 2024-09-14T06:16:56.601 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.601 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.601 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255600+0000", 2024-09-14T06:16:56.601 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.37e-06 2024-09-14T06:16:56.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.601 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.602 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255624+0000", 2024-09-14T06:16:56.602 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3938e-05 2024-09-14T06:16:56.602 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.602 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.602 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.602 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255679+0000", 2024-09-14T06:16:56.602 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4768000000000001e-05 2024-09-14T06:16:56.602 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.602 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.602 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.602 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255699+0000", 2024-09-14T06:16:56.603 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0058e-05 2024-09-14T06:16:56.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.603 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.603 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.289431+0000", 2024-09-14T06:16:56.603 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.03373259 2024-09-14T06:16:56.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.603 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.603 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.289640+0000", 2024-09-14T06:16:56.603 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000208464 2024-09-14T06:16:56.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.604 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.604 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.291156+0000", 2024-09-14T06:16:56.604 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001516243 2024-09-14T06:16:56.604 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.604 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.604 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.604 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.604 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:18156 3.17 3:e91637d8:::rbd_data.115232f4ac2b.0000000000000000:a [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T06:16:56.604 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:41.255741+0000", 2024-09-14T06:16:56.604 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.250819485999999, 2024-09-14T06:16:56.604 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035889028000000003, 2024-09-14T06:16:56.604 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:16:56.604 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:16:56.605 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:16:56.605 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:16:56.605 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:16:56.605 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:56.605 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 18156 2024-09-14T06:16:56.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.605 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:16:56.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.605 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:16:56.605 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255741+0000", 2024-09-14T06:16:56.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.605 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:16:56.606 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255741+0000", 2024-09-14T06:16:56.606 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:16:56.606 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.606 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.606 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:16:56.606 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255742+0000", 2024-09-14T06:16:56.606 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3659999999999999e-06 2024-09-14T06:16:56.606 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.606 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.606 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:16:56.606 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255760+0000", 2024-09-14T06:16:56.606 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8031999999999999e-05 2024-09-14T06:16:56.606 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.606 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.607 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:16:56.607 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255784+0000", 2024-09-14T06:16:56.607 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4454999999999999e-05 2024-09-14T06:16:56.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.607 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.607 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:16:56.607 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255803+0000", 2024-09-14T06:16:56.607 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8541000000000001e-05 2024-09-14T06:16:56.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.607 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.607 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:16:56.607 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.291231+0000", 2024-09-14T06:16:56.607 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035428098999999998 2024-09-14T06:16:56.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.607 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:16:56.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.291420+0000", 2024-09-14T06:16:56.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000188786 2024-09-14T06:16:56.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:16:56.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:16:56.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:16:56.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.291630+0000", 2024-09-14T06:16:56.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00020974899999999999 2024-09-14T06:16:56.608 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.608 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.608 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.608 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:16:56.608 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:16:56.608 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:56.609 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:16:56.611 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:16:56.611 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:56.606+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:56.711 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:56.710+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:56.771 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:56.772 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:56.781 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:16:56.781 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:16:56.781 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:16:56.781 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:16:56.788 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:56.788 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:56.795 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:16:56.804 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:56.805 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:56.812 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:56.811+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:56.821 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:56.821 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:56.841 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:56.841 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:56.859 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:56.859 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:56.879 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:56.880 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:56.899 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:56.899 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:56.912 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:16:56.911+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:16:56.963 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:56.963 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:56.977 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:16:57.013 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:57.011+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:57.101 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:57.101 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:57.114 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:57.110+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:57.115 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:16:57.215 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:57.214+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:57.278 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:57.278 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:57.292 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:16:57.316 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:57.315+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:57.417 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:57.414+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:57.418 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:16:57.418 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:16:57.418 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T06:16:57.418 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T06:16:57.434 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-14T06:16:57.460 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:57.460 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:57.476 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:16:57.485 INFO:teuthology.orchestra.run.smithi039.stdout:2891 write 0xb358fc4 thru 0xb367a6e (0xeaab bytes) 2024-09-14T06:16:57.516 INFO:teuthology.orchestra.run.smithi039.stdout:2892 punch from 0x43b1186 to 0x43b52db, (0x4155 bytes) 2024-09-14T06:16:57.519 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:57.515+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:57.537 INFO:teuthology.orchestra.run.smithi039.stdout:2893 trunc from 0xed08aff to 0x193b820 2024-09-14T06:16:57.620 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:57.619+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:57.637 INFO:teuthology.orchestra.run.smithi039.stdout:2894 write 0x29a7c7b thru 0x29a9d47 (0x20cd bytes) 2024-09-14T06:16:57.640 INFO:teuthology.orchestra.run.smithi039.stdout:2895 trunc from 0x29a9d48 to 0x1ffc815 2024-09-14T06:16:57.669 INFO:teuthology.orchestra.run.smithi039.stdout:2896 writesame 0xb7d388a thru 0xb7d63fd data_size 0x36(0x2b74 bytes) 2024-09-14T06:16:57.670 INFO:teuthology.orchestra.run.smithi039.stdout:2897 read 0x9e02a88 thru 0x9e0572e (0x2ca7 bytes) 2024-09-14T06:16:57.671 INFO:teuthology.orchestra.run.smithi039.stdout:2898 write 0x16e312d thru 0x16e7646 (0x451a bytes) 2024-09-14T06:16:57.673 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:57.673 INFO:teuthology.orchestra.run.smithi039.stdout:2900 write 0xa1c36cd thru 0xa1ccb1a (0x944e bytes) 2024-09-14T06:16:57.675 INFO:teuthology.orchestra.run.smithi039.stdout:2902 read 0x110ef72 thru 0x111c832 (0xd8c1 bytes) 2024-09-14T06:16:57.676 INFO:teuthology.orchestra.run.smithi039.stdout:2903 read 0x138492b thru 0x1386529 (0x1bff bytes) 2024-09-14T06:16:57.676 INFO:teuthology.orchestra.run.smithi039.stdout:2904 punch from 0x7746d23 to 0x774fc15, (0x8ef2 bytes) 2024-09-14T06:16:57.676 INFO:teuthology.orchestra.run.smithi039.stdout:2905 read 0x578f856 thru 0x579d3c1 (0xdb6c bytes) 2024-09-14T06:16:57.677 INFO:teuthology.orchestra.run.smithi039.stdout:2906 read 0x86abb52 thru 0x86ad866 (0x1d15 bytes) 2024-09-14T06:16:57.677 INFO:teuthology.orchestra.run.smithi039.stdout:2907 write 0x673e83e thru 0x674a017 (0xb7da bytes) 2024-09-14T06:16:57.677 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:57.677 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:57.679 INFO:teuthology.orchestra.run.smithi039.stdout:2908 write 0x858e8f2 thru 0x8590068 (0x1777 bytes) 2024-09-14T06:16:57.681 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:57.681 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:57.681 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:57.681 INFO:teuthology.orchestra.run.smithi039.stdout:2912 read 0xb504be1 thru 0xb507606 (0x2a26 bytes) 2024-09-14T06:16:57.692 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:16:57.693 INFO:teuthology.orchestra.run.smithi039.stdout:2913 trunc from 0xb7d63fe to 0x394ba16 2024-09-14T06:16:57.722 INFO:teuthology.orchestra.run.smithi039.stdout:2915 writesame 0xa70265d thru 0xa70620e data_size 0x1b(0x3bb2 bytes) 2024-09-14T06:16:57.723 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:57.719+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:57.735 INFO:teuthology.orchestra.run.smithi039.stdout:2916 write 0xcb96ad3 thru 0xcb9a851 (0x3d7f bytes) 2024-09-14T06:16:57.736 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:57.743 INFO:teuthology.orchestra.run.smithi039.stdout:2919 trunc from 0xcb9a852 to 0x7c197d2 2024-09-14T06:16:57.756 INFO:teuthology.orchestra.run.smithi039.stdout:2922 trunc from 0x7c197d2 to 0x8a0b1ec 2024-09-14T06:16:57.774 INFO:teuthology.orchestra.run.smithi039.stdout:2923 write 0x95084a9 thru 0x9515775 (0xd2cd bytes) 2024-09-14T06:16:57.787 INFO:teuthology.orchestra.run.smithi039.stdout:2924 writesame 0xab18f83 thru 0xab20aa6 data_size 0x6(0x7b24 bytes) 2024-09-14T06:16:57.788 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:57.789 INFO:teuthology.orchestra.run.smithi039.stdout:2927 punch from 0x6e5df29 to 0x6e5e58a, (0x661 bytes) 2024-09-14T06:16:57.789 INFO:teuthology.orchestra.run.smithi039.stdout:2929 clone 16 order 25 su 8388608 sc 10 2024-09-14T06:16:57.805 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:16:57.805 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:57.805 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 51, 2024-09-14T06:16:57.805 INFO:teuthology.orchestra.run.smithi039.stdout: "fsid": "038563b1-268f-461f-b6a6-4b4b41ad0236", 2024-09-14T06:16:57.805 INFO:teuthology.orchestra.run.smithi039.stdout: "created": "2024-09-14T06:15:49.493035+0000", 2024-09-14T06:16:57.805 INFO:teuthology.orchestra.run.smithi039.stdout: "modified": "2024-09-14T06:16:57.037907+0000", 2024-09-14T06:16:57.805 INFO:teuthology.orchestra.run.smithi039.stdout: "last_up_change": "2024-09-14T06:16:02.014989+0000", 2024-09-14T06:16:57.805 INFO:teuthology.orchestra.run.smithi039.stdout: "last_in_change": "2024-09-14T06:16:26.221200+0000", 2024-09-14T06:16:57.805 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T06:16:57.805 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_num": 5801984, 2024-09-14T06:16:57.806 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_set": [ 2024-09-14T06:16:57.806 INFO:teuthology.orchestra.run.smithi039.stdout: "noscrub", 2024-09-14T06:16:57.806 INFO:teuthology.orchestra.run.smithi039.stdout: "pglog_hardlimit", 2024-09-14T06:16:57.806 INFO:teuthology.orchestra.run.smithi039.stdout: "purged_snapdirs", 2024-09-14T06:16:57.806 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_deletes", 2024-09-14T06:16:57.806 INFO:teuthology.orchestra.run.smithi039.stdout: "sortbitwise" 2024-09-14T06:16:57.806 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:16:57.806 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_version": 5, 2024-09-14T06:16:57.806 INFO:teuthology.orchestra.run.smithi039.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T06:16:57.806 INFO:teuthology.orchestra.run.smithi039.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T06:16:57.806 INFO:teuthology.orchestra.run.smithi039.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T06:16:57.806 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_snapshot": "", 2024-09-14T06:16:57.806 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_max": 3, 2024-09-14T06:16:57.806 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd": 8, 2024-09-14T06:16:57.807 INFO:teuthology.orchestra.run.smithi039.stdout: "require_min_compat_client": "luminous", 2024-09-14T06:16:57.807 INFO:teuthology.orchestra.run.smithi039.stdout: "min_compat_client": "luminous", 2024-09-14T06:16:57.807 INFO:teuthology.orchestra.run.smithi039.stdout: "require_osd_release": "squid", 2024-09-14T06:16:57.807 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_crimson": false, 2024-09-14T06:16:57.807 INFO:teuthology.orchestra.run.smithi039.stdout: "pools": [ 2024-09-14T06:16:57.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.807 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 1, 2024-09-14T06:16:57.807 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": ".mgr", 2024-09-14T06:16:57.807 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-09-14T06:16:02.429225+0000", 2024-09-14T06:16:57.807 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-14T06:16:57.807 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool", 2024-09-14T06:16:57.807 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-09-14T06:16:57.807 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-09-14T06:16:57.807 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-09-14T06:16:57.808 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-09-14T06:16:57.808 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:16:57.808 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:16:57.808 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:16:57.808 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:16:57.808 INFO:teuthology.orchestra.run.smithi039.stdout: "is_stretch_pool": false, 2024-09-14T06:16:57.808 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-09-14T06:16:57.808 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:16:57.808 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 1, 2024-09-14T06:16:57.808 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 1, 2024-09-14T06:16:57.808 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 1, 2024-09-14T06:16:57.808 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 1, 2024-09-14T06:16:57.808 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 1, 2024-09-14T06:16:57.808 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-09-14T06:16:57.808 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-09-14T06:16:57.809 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-09-14T06:16:57.809 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-09-14T06:16:57.809 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-09-14T06:16:57.809 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-09-14T06:16:57.809 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-09-14T06:16:57.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.809 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "18", 2024-09-14T06:16:57.809 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-09-14T06:16:57.809 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:16:57.809 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:16:57.809 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-09-14T06:16:57.809 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:16:57.809 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 0, 2024-09-14T06:16:57.810 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 0, 2024-09-14T06:16:57.810 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-09-14T06:16:57.810 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-09-14T06:16:57.810 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-09-14T06:16:57.810 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-09-14T06:16:57.810 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-09-14T06:16:57.810 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-09-14T06:16:57.810 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-09-14T06:16:57.810 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-09-14T06:16:57.810 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-09-14T06:16:57.811 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-09-14T06:16:57.811 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-09-14T06:16:57.811 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:16:57.811 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:16:57.811 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:16:57.811 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-09-14T06:16:57.811 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-09-14T06:16:57.811 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-09-14T06:16:57.811 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-09-14T06:16:57.811 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-09-14T06:16:57.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.811 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-09-14T06:16:57.811 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-09-14T06:16:57.811 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-09-14T06:16:57.811 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:16:57.812 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:16:57.812 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:16:57.812 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:16:57.812 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-09-14T06:16:57.812 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-09-14T06:16:57.812 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-09-14T06:16:57.812 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-09-14T06:16:57.812 INFO:teuthology.orchestra.run.smithi039.stdout: "options": { 2024-09-14T06:16:57.812 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_max": 32, 2024-09-14T06:16:57.812 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_min": 1 2024-09-14T06:16:57.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.812 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-09-14T06:16:57.812 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": {} 2024-09-14T06:16:57.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.813 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-09-14T06:16:57.813 INFO:teuthology.orchestra.run.smithi039.stdout: "score_type": "Fair distribution", 2024-09-14T06:16:57.813 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 6.9000000953674316, 2024-09-14T06:16:57.813 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 6.9000000953674316, 2024-09-14T06:16:57.813 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T06:16:57.813 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 2, 2024-09-14T06:16:57.813 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 2, 2024-09-14T06:16:57.814 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:16:57.814 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 1, 2024-09-14T06:16:57.814 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:16:57.814 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.814 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-14T06:16:57.814 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": "rbd", 2024-09-14T06:16:57.814 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-09-14T06:16:08.129454+0000", 2024-09-14T06:16:57.814 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 8193, 2024-09-14T06:16:57.815 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:16:57.815 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-09-14T06:16:57.815 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-09-14T06:16:57.815 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-09-14T06:16:57.815 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-09-14T06:16:57.815 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:16:57.815 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:16:57.815 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:16:57.815 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:16:57.815 INFO:teuthology.orchestra.run.smithi039.stdout: "is_stretch_pool": false, 2024-09-14T06:16:57.815 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-09-14T06:16:57.815 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:16:57.815 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 8, 2024-09-14T06:16:57.815 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 8, 2024-09-14T06:16:57.816 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 8, 2024-09-14T06:16:57.816 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 8, 2024-09-14T06:16:57.816 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 8, 2024-09-14T06:16:57.816 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-09-14T06:16:57.816 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-09-14T06:16:57.816 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-09-14T06:16:57.816 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-09-14T06:16:57.816 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-09-14T06:16:57.816 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-09-14T06:16:57.816 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-09-14T06:16:57.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.816 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "22", 2024-09-14T06:16:57.816 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-09-14T06:16:57.816 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:16:57.817 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:16:57.817 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-09-14T06:16:57.817 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:16:57.817 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 2, 2024-09-14T06:16:57.817 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 22, 2024-09-14T06:16:57.817 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-09-14T06:16:57.817 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-09-14T06:16:57.817 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-09-14T06:16:57.817 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-09-14T06:16:57.817 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-09-14T06:16:57.817 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-09-14T06:16:57.817 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-09-14T06:16:57.817 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-09-14T06:16:57.817 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-09-14T06:16:57.818 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-09-14T06:16:57.818 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-09-14T06:16:57.818 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:16:57.818 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:16:57.818 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:16:57.818 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-09-14T06:16:57.818 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-09-14T06:16:57.818 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-09-14T06:16:57.818 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-09-14T06:16:57.818 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-09-14T06:16:57.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.818 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-09-14T06:16:57.818 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-09-14T06:16:57.818 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-09-14T06:16:57.819 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:16:57.819 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:16:57.819 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:16:57.819 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:16:57.819 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-09-14T06:16:57.819 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-09-14T06:16:57.819 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-09-14T06:16:57.819 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-09-14T06:16:57.819 INFO:teuthology.orchestra.run.smithi039.stdout: "options": {}, 2024-09-14T06:16:57.819 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-09-14T06:16:57.819 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd": {} 2024-09-14T06:16:57.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.819 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-09-14T06:16:57.819 INFO:teuthology.orchestra.run.smithi039.stdout: "score_type": "Fair distribution", 2024-09-14T06:16:57.820 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 4.380000114440918, 2024-09-14T06:16:57.820 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 4.380000114440918, 2024-09-14T06:16:57.820 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 1, 2024-09-14T06:16:57.820 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 4.380000114440918, 2024-09-14T06:16:57.820 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 4.380000114440918, 2024-09-14T06:16:57.820 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:16:57.820 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 1, 2024-09-14T06:16:57.820 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:16:57.820 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.821 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.821 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 3, 2024-09-14T06:16:57.821 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": "pool_client.0", 2024-09-14T06:16:57.821 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-09-14T06:16:18.928556+0000", 2024-09-14T06:16:57.821 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 8193, 2024-09-14T06:16:57.821 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:16:57.821 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-09-14T06:16:57.821 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-09-14T06:16:57.821 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-09-14T06:16:57.821 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-09-14T06:16:57.821 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:16:57.821 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:16:57.821 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:16:57.822 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:16:57.822 INFO:teuthology.orchestra.run.smithi039.stdout: "is_stretch_pool": false, 2024-09-14T06:16:57.822 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-09-14T06:16:57.822 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:16:57.822 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 32, 2024-09-14T06:16:57.822 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 32, 2024-09-14T06:16:57.822 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 32, 2024-09-14T06:16:57.822 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 32, 2024-09-14T06:16:57.822 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 32, 2024-09-14T06:16:57.822 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-09-14T06:16:57.822 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-09-14T06:16:57.822 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-09-14T06:16:57.822 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-09-14T06:16:57.822 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-09-14T06:16:57.823 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-09-14T06:16:57.823 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-09-14T06:16:57.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.823 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "51", 2024-09-14T06:16:57.823 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-09-14T06:16:57.823 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:16:57.823 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:16:57.823 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-09-14T06:16:57.823 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:16:57.823 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 17, 2024-09-14T06:16:57.823 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 51, 2024-09-14T06:16:57.823 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-09-14T06:16:57.823 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-09-14T06:16:57.824 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-09-14T06:16:57.824 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-09-14T06:16:57.824 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-09-14T06:16:57.824 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-09-14T06:16:57.824 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-09-14T06:16:57.824 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-09-14T06:16:57.824 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-09-14T06:16:57.824 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-09-14T06:16:57.824 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-09-14T06:16:57.824 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:16:57.824 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:16:57.824 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:16:57.824 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-09-14T06:16:57.824 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-09-14T06:16:57.825 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-09-14T06:16:57.825 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-09-14T06:16:57.825 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-09-14T06:16:57.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.825 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-09-14T06:16:57.825 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-09-14T06:16:57.825 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-09-14T06:16:57.825 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:16:57.825 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:16:57.825 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:16:57.826 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:16:57.826 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-09-14T06:16:57.826 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-09-14T06:16:57.826 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-09-14T06:16:57.826 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-09-14T06:16:57.826 INFO:teuthology.orchestra.run.smithi039.stdout: "options": {}, 2024-09-14T06:16:57.826 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-09-14T06:16:57.826 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd": {} 2024-09-14T06:16:57.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.826 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-09-14T06:16:57.826 INFO:teuthology.orchestra.run.smithi039.stdout: "score_type": "Fair distribution", 2024-09-14T06:16:57.827 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 1.5399999618530273, 2024-09-14T06:16:57.827 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 1.5399999618530273, 2024-09-14T06:16:57.827 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 1.1399999856948853, 2024-09-14T06:16:57.827 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 1.75, 2024-09-14T06:16:57.827 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 1.75, 2024-09-14T06:16:57.827 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:16:57.827 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 1, 2024-09-14T06:16:57.827 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:16:57.827 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.827 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.827 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:16:57.827 INFO:teuthology.orchestra.run.smithi039.stdout: "osds": [ 2024-09-14T06:16:57.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.828 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 0, 2024-09-14T06:16:57.828 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "c669ea76-ed19-4c12-9e56-d79d62846f95", 2024-09-14T06:16:57.828 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:16:57.828 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:16:57.828 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:16:57.828 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:16:57.828 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:16:57.828 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:16:57.828 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:16:57.828 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 34, 2024-09-14T06:16:57.828 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:16:57.828 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:16:57.828 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:16:57.828 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.829 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.829 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6808", 2024-09-14T06:16:57.829 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:16:57.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.829 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.829 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6809", 2024-09-14T06:16:57.829 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:16:57.829 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.829 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.829 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:16:57.829 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.830 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.830 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6810", 2024-09-14T06:16:57.830 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:16:57.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.830 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.830 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6811", 2024-09-14T06:16:57.830 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:16:57.830 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.830 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.830 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:16:57.831 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.831 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.831 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.831 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6814", 2024-09-14T06:16:57.831 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:16:57.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.831 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.831 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.831 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6815", 2024-09-14T06:16:57.831 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:16:57.831 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.831 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.831 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:16:57.831 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.831 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.832 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.832 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6812", 2024-09-14T06:16:57.832 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:16:57.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.832 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.832 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.832 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6813", 2024-09-14T06:16:57.832 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:16:57.832 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.832 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.832 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6809/1157559201", 2024-09-14T06:16:57.832 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6811/1157559201", 2024-09-14T06:16:57.833 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6815/1157559201", 2024-09-14T06:16:57.833 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6813/1157559201", 2024-09-14T06:16:57.833 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:16:57.833 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:16:57.833 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:16:57.833 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.833 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 1, 2024-09-14T06:16:57.833 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b", 2024-09-14T06:16:57.833 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:16:57.833 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:16:57.833 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:16:57.833 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:16:57.834 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:16:57.834 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:16:57.834 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:16:57.834 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 37, 2024-09-14T06:16:57.834 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:16:57.834 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:16:57.834 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:16:57.834 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.835 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.835 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6816", 2024-09-14T06:16:57.835 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:16:57.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.835 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.835 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6817", 2024-09-14T06:16:57.835 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:16:57.835 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.835 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.836 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:16:57.836 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.836 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.836 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.836 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6818", 2024-09-14T06:16:57.836 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:16:57.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.836 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.836 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.836 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6819", 2024-09-14T06:16:57.836 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:16:57.837 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.837 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.837 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:16:57.837 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.837 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.837 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.837 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6822", 2024-09-14T06:16:57.837 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:16:57.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.837 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.837 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.837 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6823", 2024-09-14T06:16:57.837 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:16:57.838 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.838 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.838 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:16:57.838 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.838 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.838 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.838 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6820", 2024-09-14T06:16:57.838 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:16:57.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.838 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.838 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.838 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6821", 2024-09-14T06:16:57.839 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:16:57.839 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.839 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.839 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6817/4062089212", 2024-09-14T06:16:57.839 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6819/4062089212", 2024-09-14T06:16:57.839 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6823/4062089212", 2024-09-14T06:16:57.839 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6821/4062089212", 2024-09-14T06:16:57.839 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:16:57.839 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:16:57.839 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:16:57.839 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.840 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.840 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 2, 2024-09-14T06:16:57.840 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "9bda988f-e615-4545-849b-318c408c4c42", 2024-09-14T06:16:57.840 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:16:57.840 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:16:57.840 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:16:57.840 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 0, 2024-09-14T06:16:57.840 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:16:57.841 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:16:57.841 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:16:57.841 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 29, 2024-09-14T06:16:57.841 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:16:57.841 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:16:57.841 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:16:57.841 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.841 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.841 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.841 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6824", 2024-09-14T06:16:57.841 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:16:57.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.841 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.842 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.842 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6825", 2024-09-14T06:16:57.842 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:16:57.842 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.842 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.842 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:16:57.842 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.842 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.842 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.842 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6826", 2024-09-14T06:16:57.842 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:16:57.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.842 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.843 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.843 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6827", 2024-09-14T06:16:57.843 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:16:57.843 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.843 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.843 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:16:57.843 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.843 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.843 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.843 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6830", 2024-09-14T06:16:57.843 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:16:57.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.843 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.844 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.844 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6831", 2024-09-14T06:16:57.844 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:16:57.844 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.844 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.844 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:16:57.844 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.844 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.844 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.844 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6828", 2024-09-14T06:16:57.844 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:16:57.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.845 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.845 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.845 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6829", 2024-09-14T06:16:57.845 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:16:57.845 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.845 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.845 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6825/1724773882", 2024-09-14T06:16:57.845 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6827/1724773882", 2024-09-14T06:16:57.845 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6831/1724773882", 2024-09-14T06:16:57.845 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6829/1724773882", 2024-09-14T06:16:57.845 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:16:57.845 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:16:57.845 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:16:57.846 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.846 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.846 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 3, 2024-09-14T06:16:57.846 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "8ea6cd69-2ef1-42ea-b339-7394bf8f4a87", 2024-09-14T06:16:57.846 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:16:57.846 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:16:57.846 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:16:57.846 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:16:57.846 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:16:57.846 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:16:57.846 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:16:57.846 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 34, 2024-09-14T06:16:57.846 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:16:57.847 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:16:57.847 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:16:57.847 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.847 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.847 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.847 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6800", 2024-09-14T06:16:57.847 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:16:57.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.847 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.847 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.847 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6801", 2024-09-14T06:16:57.847 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:16:57.847 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.848 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.848 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:16:57.848 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.848 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.848 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.848 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6802", 2024-09-14T06:16:57.848 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:16:57.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.848 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.848 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.849 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6803", 2024-09-14T06:16:57.849 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:16:57.849 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.849 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.849 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:16:57.849 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.849 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.849 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.849 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6806", 2024-09-14T06:16:57.849 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:16:57.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.854 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.854 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.854 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6807", 2024-09-14T06:16:57.854 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:16:57.854 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.854 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.854 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.854 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:16:57.854 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.854 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.854 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.854 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6804", 2024-09-14T06:16:57.855 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:16:57.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.855 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.855 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.855 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6805", 2024-09-14T06:16:57.855 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:16:57.855 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.855 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.855 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6801/3775031293", 2024-09-14T06:16:57.855 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6803/3775031293", 2024-09-14T06:16:57.855 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6807/3775031293", 2024-09-14T06:16:57.856 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6805/3775031293", 2024-09-14T06:16:57.856 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:16:57.856 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:16:57.856 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:16:57.856 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.856 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.856 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 4, 2024-09-14T06:16:57.856 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "fea45e8a-e8ea-4963-b93f-0651b84c9e74", 2024-09-14T06:16:57.856 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:16:57.856 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:16:57.856 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:16:57.856 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:16:57.856 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:16:57.857 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:16:57.857 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:16:57.857 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 29, 2024-09-14T06:16:57.857 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:16:57.857 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:16:57.857 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:16:57.857 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.857 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.857 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.857 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6824", 2024-09-14T06:16:57.857 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:16:57.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.857 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.858 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.858 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6825", 2024-09-14T06:16:57.858 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:16:57.858 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.858 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.858 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:16:57.858 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.858 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.858 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.858 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6826", 2024-09-14T06:16:57.858 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:16:57.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.859 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.859 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.859 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6827", 2024-09-14T06:16:57.859 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:16:57.859 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.859 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.859 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:16:57.859 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.860 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.860 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.860 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6830", 2024-09-14T06:16:57.860 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:16:57.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.860 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.860 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.860 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6831", 2024-09-14T06:16:57.860 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:16:57.860 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.860 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.860 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:16:57.861 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.861 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.861 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.861 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6828", 2024-09-14T06:16:57.861 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:16:57.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.861 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.861 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.861 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6829", 2024-09-14T06:16:57.861 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:16:57.861 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.861 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.862 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6825/2632317380", 2024-09-14T06:16:57.862 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6827/2632317380", 2024-09-14T06:16:57.862 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6831/2632317380", 2024-09-14T06:16:57.862 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6829/2632317380", 2024-09-14T06:16:57.862 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:16:57.862 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:16:57.862 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:16:57.862 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.862 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.862 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 5, 2024-09-14T06:16:57.863 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "b601e852-a180-477f-86fc-6a213fa866f0", 2024-09-14T06:16:57.863 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:16:57.863 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:16:57.863 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:16:57.863 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:16:57.863 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:16:57.863 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:16:57.863 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:16:57.863 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 29, 2024-09-14T06:16:57.863 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:16:57.863 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:16:57.863 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:16:57.863 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.863 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.864 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.864 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6816", 2024-09-14T06:16:57.864 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:16:57.864 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.864 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.864 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.864 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6817", 2024-09-14T06:16:57.864 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:16:57.864 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.864 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.864 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.864 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:16:57.864 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.865 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.865 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.865 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6818", 2024-09-14T06:16:57.865 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:16:57.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.865 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.865 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.865 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6819", 2024-09-14T06:16:57.865 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:16:57.865 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.865 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.865 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:16:57.865 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.866 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.866 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.866 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6822", 2024-09-14T06:16:57.866 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:16:57.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.866 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.866 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.866 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6823", 2024-09-14T06:16:57.866 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:16:57.866 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.866 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.866 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:16:57.866 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.866 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.867 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.867 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6820", 2024-09-14T06:16:57.867 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:16:57.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.867 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.867 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.867 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6821", 2024-09-14T06:16:57.867 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:16:57.867 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.867 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.868 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6817/3501664149", 2024-09-14T06:16:57.868 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6819/3501664149", 2024-09-14T06:16:57.868 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6823/3501664149", 2024-09-14T06:16:57.868 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6821/3501664149", 2024-09-14T06:16:57.868 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:16:57.868 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:16:57.868 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:16:57.868 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.868 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.868 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.868 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 6, 2024-09-14T06:16:57.868 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "1113afe0-f524-41b1-8e51-eb5164602d38", 2024-09-14T06:16:57.868 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:16:57.868 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:16:57.868 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:16:57.869 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:16:57.869 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:16:57.869 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:16:57.869 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:16:57.869 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 34, 2024-09-14T06:16:57.869 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:16:57.869 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:16:57.869 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:16:57.869 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.869 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.870 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.870 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6800", 2024-09-14T06:16:57.870 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:16:57.870 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.870 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.870 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.870 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6801", 2024-09-14T06:16:57.870 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:16:57.870 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.870 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.870 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.870 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:16:57.870 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.870 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.870 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.871 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6802", 2024-09-14T06:16:57.871 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:16:57.871 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.871 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.871 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.871 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6803", 2024-09-14T06:16:57.871 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:16:57.871 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.871 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.871 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.871 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:16:57.871 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.872 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.872 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.872 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6806", 2024-09-14T06:16:57.872 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:16:57.872 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.872 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.872 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.872 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6807", 2024-09-14T06:16:57.872 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:16:57.872 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.872 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.872 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.872 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:16:57.872 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.872 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.873 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.873 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6804", 2024-09-14T06:16:57.873 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:16:57.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.873 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.873 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.873 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6805", 2024-09-14T06:16:57.873 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:16:57.873 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.873 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.873 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6801/1395936437", 2024-09-14T06:16:57.873 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6803/1395936437", 2024-09-14T06:16:57.873 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6807/1395936437", 2024-09-14T06:16:57.873 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6805/1395936437", 2024-09-14T06:16:57.874 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:16:57.874 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:16:57.874 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:16:57.874 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.874 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.874 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.874 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 7, 2024-09-14T06:16:57.874 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "c9162545-318b-4591-be64-cc8ace9097fe", 2024-09-14T06:16:57.874 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:16:57.874 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:16:57.874 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:16:57.874 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:16:57.874 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:16:57.874 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:16:57.874 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:16:57.875 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 23, 2024-09-14T06:16:57.875 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:16:57.875 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:16:57.875 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:16:57.875 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.875 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.875 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.875 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6808", 2024-09-14T06:16:57.875 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:16:57.875 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.875 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.875 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.875 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6809", 2024-09-14T06:16:57.875 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:16:57.875 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.876 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.876 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.876 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:16:57.876 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.876 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.876 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.876 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6810", 2024-09-14T06:16:57.876 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:16:57.876 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.876 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.876 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.876 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6811", 2024-09-14T06:16:57.876 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:16:57.876 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.877 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.877 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.877 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:16:57.877 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.877 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.877 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.877 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6814", 2024-09-14T06:16:57.877 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:16:57.877 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.877 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.877 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.878 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6815", 2024-09-14T06:16:57.878 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:16:57.878 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.878 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.878 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:16:57.878 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:16:57.878 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.878 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:16:57.878 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6812", 2024-09-14T06:16:57.878 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:16:57.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.878 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.878 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:16:57.878 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6813", 2024-09-14T06:16:57.879 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:16:57.879 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.879 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.879 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.879 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6809/711112216", 2024-09-14T06:16:57.879 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6811/711112216", 2024-09-14T06:16:57.879 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6815/711112216", 2024-09-14T06:16:57.879 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6813/711112216", 2024-09-14T06:16:57.879 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:16:57.879 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:16:57.879 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:16:57.879 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.879 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.879 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:16:57.879 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_xinfo": [ 2024-09-14T06:16:57.880 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.880 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 0, 2024-09-14T06:16:57.880 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:16:57.880 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:16:57.880 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:16:57.880 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:16:57.880 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:16:57.880 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.705575+0000", 2024-09-14T06:16:57.880 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:16:57.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.880 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.880 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 1, 2024-09-14T06:16:57.880 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:16:57.880 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:16:57.881 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:16:57.881 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:16:57.881 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:16:57.881 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.806946+0000", 2024-09-14T06:16:57.881 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:16:57.881 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.881 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.881 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 2, 2024-09-14T06:16:57.881 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:16:57.881 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:16:57.881 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:16:57.881 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:16:57.881 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:16:57.881 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.853831+0000", 2024-09-14T06:16:57.882 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:16:57.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.882 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.882 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 3, 2024-09-14T06:16:57.882 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:16:57.882 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:16:57.882 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:16:57.882 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:16:57.882 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:16:57.882 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.690315+0000", 2024-09-14T06:16:57.882 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:16:57.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.882 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.882 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 4, 2024-09-14T06:16:57.882 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:16:57.882 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:16:57.883 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:16:57.883 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:16:57.883 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:16:57.883 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.494526+0000", 2024-09-14T06:16:57.883 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:16:57.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.883 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.883 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 5, 2024-09-14T06:16:57.883 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:16:57.883 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:16:57.883 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:16:57.883 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:16:57.883 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:16:57.883 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.546122+0000", 2024-09-14T06:16:57.883 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:16:57.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.884 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.884 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 6, 2024-09-14T06:16:57.884 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:16:57.884 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:16:57.884 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:16:57.884 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:16:57.884 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:16:57.884 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.173691+0000", 2024-09-14T06:16:57.884 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:16:57.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.884 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.884 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 7, 2024-09-14T06:16:57.884 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:16:57.885 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:16:57.885 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:16:57.885 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:16:57.885 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:16:57.885 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.148509+0000", 2024-09-14T06:16:57.885 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:16:57.885 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.885 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:16:57.885 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap": [], 2024-09-14T06:16:57.885 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap_items": [ 2024-09-14T06:16:57.885 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.885 INFO:teuthology.orchestra.run.smithi039.stdout: "pgid": "3.14", 2024-09-14T06:16:57.885 INFO:teuthology.orchestra.run.smithi039.stdout: "mappings": [ 2024-09-14T06:16:57.885 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:57.885 INFO:teuthology.orchestra.run.smithi039.stdout: "from": 3, 2024-09-14T06:16:57.886 INFO:teuthology.orchestra.run.smithi039.stdout: "to": 1 2024-09-14T06:16:57.886 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.886 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:57.886 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.886 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:16:57.886 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap_primaries": [], 2024-09-14T06:16:57.886 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_temp": [], 2024-09-14T06:16:57.886 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_temp": [], 2024-09-14T06:16:57.886 INFO:teuthology.orchestra.run.smithi039.stdout: "blocklist": {}, 2024-09-14T06:16:57.886 INFO:teuthology.orchestra.run.smithi039.stdout: "range_blocklist": {}, 2024-09-14T06:16:57.886 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profiles": { 2024-09-14T06:16:57.886 INFO:teuthology.orchestra.run.smithi039.stdout: "default": { 2024-09-14T06:16:57.886 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-failure-domain": "osd", 2024-09-14T06:16:57.886 INFO:teuthology.orchestra.run.smithi039.stdout: "k": "2", 2024-09-14T06:16:57.886 INFO:teuthology.orchestra.run.smithi039.stdout: "m": "1", 2024-09-14T06:16:57.886 INFO:teuthology.orchestra.run.smithi039.stdout: "plugin": "jerasure", 2024-09-14T06:16:57.887 INFO:teuthology.orchestra.run.smithi039.stdout: "technique": "reed_sol_van" 2024-09-14T06:16:57.887 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:57.887 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps_queue": [], 2024-09-14T06:16:57.887 INFO:teuthology.orchestra.run.smithi039.stdout: "new_removed_snaps": [], 2024-09-14T06:16:57.887 INFO:teuthology.orchestra.run.smithi039.stdout: "new_purged_snaps": [], 2024-09-14T06:16:57.887 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_node_flags": {}, 2024-09-14T06:16:57.887 INFO:teuthology.orchestra.run.smithi039.stdout: "device_class_flags": {}, 2024-09-14T06:16:57.887 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode": { 2024-09-14T06:16:57.887 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode_enabled": false, 2024-09-14T06:16:57.887 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_bucket_count": 0, 2024-09-14T06:16:57.887 INFO:teuthology.orchestra.run.smithi039.stdout: "degraded_stretch_mode": 0, 2024-09-14T06:16:57.887 INFO:teuthology.orchestra.run.smithi039.stdout: "recovering_stretch_mode": 0, 2024-09-14T06:16:57.888 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode_bucket": 0 2024-09-14T06:16:57.888 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:57.888 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:57.889 INFO:tasks.thrashosds.thrasher:j is {'epoch': 51, 'fsid': '038563b1-268f-461f-b6a6-4b4b41ad0236', 'created': '2024-09-14T06:15:49.493035+0000', 'modified': '2024-09-14T06:16:57.037907+0000', 'last_up_change': '2024-09-14T06:16:02.014989+0000', 'last_in_change': '2024-09-14T06:16:26.221200+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:16:02.429225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:16:08.129454+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.380000114440918, 'score_stable': 4.380000114440918, 'optimal_score': 1, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:16:18.928556+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 51, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'c669ea76-ed19-4c12-9e56-d79d62846f95', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6808', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6809', 'nonce': 1157559201}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6810', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6811', 'nonce': 1157559201}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6814', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6815', 'nonce': 1157559201}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6812', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6813', 'nonce': 1157559201}]}, 'public_addr': '172.21.15.39:6809/1157559201', 'cluster_addr': '172.21.15.39:6811/1157559201', 'heartbeat_back_addr': '172.21.15.39:6815/1157559201', 'heartbeat_front_addr': '172.21.15.39:6813/1157559201', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6816', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6817', 'nonce': 4062089212}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6818', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6819', 'nonce': 4062089212}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6822', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6823', 'nonce': 4062089212}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6820', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6821', 'nonce': 4062089212}]}, 'public_addr': '172.21.15.39:6817/4062089212', 'cluster_addr': '172.21.15.39:6819/4062089212', 'heartbeat_back_addr': '172.21.15.39:6823/4062089212', 'heartbeat_front_addr': '172.21.15.39:6821/4062089212', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9bda988f-e615-4545-849b-318c408c4c42', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 29, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6824', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6825', 'nonce': 1724773882}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6826', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6827', 'nonce': 1724773882}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6830', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6831', 'nonce': 1724773882}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6828', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6829', 'nonce': 1724773882}]}, 'public_addr': '172.21.15.39:6825/1724773882', 'cluster_addr': '172.21.15.39:6827/1724773882', 'heartbeat_back_addr': '172.21.15.39:6831/1724773882', 'heartbeat_front_addr': '172.21.15.39:6829/1724773882', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8ea6cd69-2ef1-42ea-b339-7394bf8f4a87', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6800', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6801', 'nonce': 3775031293}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6802', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6803', 'nonce': 3775031293}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6806', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6807', 'nonce': 3775031293}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6804', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6805', 'nonce': 3775031293}]}, 'public_addr': '172.21.15.39:6801/3775031293', 'cluster_addr': '172.21.15.39:6803/3775031293', 'heartbeat_back_addr': '172.21.15.39:6807/3775031293', 'heartbeat_front_addr': '172.21.15.39:6805/3775031293', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fea45e8a-e8ea-4963-b93f-0651b84c9e74', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 29, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6824', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6825', 'nonce': 2632317380}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6826', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6827', 'nonce': 2632317380}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6830', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6831', 'nonce': 2632317380}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6828', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6829', 'nonce': 2632317380}]}, 'public_addr': '172.21.15.87:6825/2632317380', 'cluster_addr': '172.21.15.87:6827/2632317380', 'heartbeat_back_addr': '172.21.15.87:6831/2632317380', 'heartbeat_front_addr': '172.21.15.87:6829/2632317380', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b601e852-a180-477f-86fc-6a213fa866f0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 29, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6816', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6817', 'nonce': 3501664149}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6818', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6819', 'nonce': 3501664149}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6822', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6823', 'nonce': 3501664149}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6820', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6821', 'nonce': 3501664149}]}, 'public_addr': '172.21.15.87:6817/3501664149', 'cluster_addr': '172.21.15.87:6819/3501664149', 'heartbeat_back_addr': '172.21.15.87:6823/3501664149', 'heartbeat_front_addr': '172.21.15.87:6821/3501664149', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '1113afe0-f524-41b1-8e51-eb5164602d38', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6800', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6801', 'nonce': 1395936437}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6802', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6803', 'nonce': 1395936437}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6806', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6807', 'nonce': 1395936437}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6804', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6805', 'nonce': 1395936437}]}, 'public_addr': '172.21.15.87:6801/1395936437', 'cluster_addr': '172.21.15.87:6803/1395936437', 'heartbeat_back_addr': '172.21.15.87:6807/1395936437', 'heartbeat_front_addr': '172.21.15.87:6805/1395936437', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c9162545-318b-4591-be64-cc8ace9097fe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6808', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6809', 'nonce': 711112216}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6810', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6811', 'nonce': 711112216}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6814', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6815', 'nonce': 711112216}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6812', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6813', 'nonce': 711112216}]}, 'public_addr': '172.21.15.87:6809/711112216', 'cluster_addr': '172.21.15.87:6811/711112216', 'heartbeat_back_addr': '172.21.15.87:6815/711112216', 'heartbeat_front_addr': '172.21.15.87:6813/711112216', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.705575+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.806946+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.853831+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.690315+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.494526+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.546122+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.173691+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.148509+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.14', 'mappings': [{'from': 3, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T06:16:57.890 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:16:57.891 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:57.891 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:57.891 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:16:57.891 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:57.890+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:57.990 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:57.986+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:58.054 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:16:58.055 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:58.055 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:58.071 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:16:58.119 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:58.119+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:58.189 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone15 from 0xab20aa7 (overlap 0x193b820) to 0x285f36 2024-09-14T06:16:58.192 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:58.190+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:58.251 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:58.251 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:58.267 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:16:58.292 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:16:58.290+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:16:58.298 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-14T06:16:58.348 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:16:58.349 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:16:58.372 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-09-14T06:16:58.083042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":561213893,"num_objects":688,"num_object_clones":111,"num_object_copies":1376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":688,"num_whiteouts":90,"num_read":45030,"num_read_kb":319241,"num_write":8127,"num_write_kb":204474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":11113572,"num_keys_recovered":9,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6471,"ondisk_log_size":6471,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":330968,"kb_used_data":39820,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754643752,"statfs":{"total":773094113280,"available":772755202048,"internally_reserved":0,"allocated":40775680,"data_stored":346278000,"data_compressed":11189724,"data_compressed_allocated":23891968,"data_compressed_original":333669465,"omap_allocated":12752,"internal_metadata":297913904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":114482639,"num_objects":96,"num_object_clones":10,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":1,"num_read":9465,"num_read_kb":89078,"num_write":2379,"num_write_kb":42435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.536050"},"pg_stats":[{"pgid":"3.1f","version":"50'262","reported_seq":1224,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.144711+0000","last_change":"2024-09-14T06:16:22.204741+0000","last_active":"2024-09-14T06:16:56.144711+0000","last_peered":"2024-09-14T06:16:56.144711+0000","last_clean":"2024-09-14T06:16:56.144711+0000","last_became_active":"2024-09-14T06:16:20.192649+0000","last_became_peered":"2024-09-14T06:16:20.192649+0000","last_unstale":"2024-09-14T06:16:56.144711+0000","last_undegraded":"2024-09-14T06:16:56.144711+0000","last_fullsized":"2024-09-14T06:16:56.144711+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:12:15.418446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010466430000000001,"stat_sum":{"num_bytes":20708312,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":1,"num_read":2065,"num_read_kb":12448,"num_write":341,"num_write_kb":6342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"50'116","reported_seq":545,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.185278+0000","last_change":"2024-09-14T06:16:27.310639+0000","last_active":"2024-09-14T06:16:56.185278+0000","last_peered":"2024-09-14T06:16:56.185278+0000","last_clean":"2024-09-14T06:16:56.185278+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:16:56.185278+0000","last_undegraded":"2024-09-14T06:16:56.185278+0000","last_fullsized":"2024-09-14T06:16:56.185278+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:24:52.700244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095849699999999999,"stat_sum":{"num_bytes":18598149,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":482,"num_read_kb":7005,"num_write":128,"num_write_kb":5015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"49'107","reported_seq":643,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.049148+0000","last_change":"2024-09-14T06:16:22.322492+0000","last_active":"2024-09-14T06:16:57.049148+0000","last_peered":"2024-09-14T06:16:57.049148+0000","last_clean":"2024-09-14T06:16:57.049148+0000","last_became_active":"2024-09-14T06:16:20.192807+0000","last_became_peered":"2024-09-14T06:16:20.192807+0000","last_unstale":"2024-09-14T06:16:57.049148+0000","last_undegraded":"2024-09-14T06:16:57.049148+0000","last_fullsized":"2024-09-14T06:16:57.049148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:52:04.855274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001319401,"stat_sum":{"num_bytes":22444329,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":2,"num_read":484,"num_read_kb":9487,"num_write":109,"num_write_kb":5035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"51'715","reported_seq":1569,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.595768+0000","last_change":"2024-09-14T06:16:22.201834+0000","last_active":"2024-09-14T06:16:57.595768+0000","last_peered":"2024-09-14T06:16:57.595768+0000","last_clean":"2024-09-14T06:16:57.595768+0000","last_became_active":"2024-09-14T06:16:20.190215+0000","last_became_peered":"2024-09-14T06:16:20.190215+0000","last_unstale":"2024-09-14T06:16:57.595768+0000","last_undegraded":"2024-09-14T06:16:57.595768+0000","last_fullsized":"2024-09-14T06:16:57.595768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:28:43.856800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026065160000000001,"stat_sum":{"num_bytes":3976989,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":2,"num_read":4747,"num_read_kb":4825,"num_write":1129,"num_write_kb":2241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"50'110","reported_seq":566,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.181021+0000","last_change":"2024-09-14T06:16:22.193116+0000","last_active":"2024-09-14T06:16:56.181021+0000","last_peered":"2024-09-14T06:16:56.181021+0000","last_clean":"2024-09-14T06:16:56.181021+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:16:56.181021+0000","last_undegraded":"2024-09-14T06:16:56.181021+0000","last_fullsized":"2024-09-14T06:16:56.181021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:58.093891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014745120000000001,"stat_sum":{"num_bytes":16217472,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":3,"num_read":414,"num_read_kb":11914,"num_write":97,"num_write_kb":9660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"50'388","reported_seq":1650,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.629763+0000","last_change":"2024-09-14T06:16:22.201461+0000","last_active":"2024-09-14T06:16:57.629763+0000","last_peered":"2024-09-14T06:16:57.629763+0000","last_clean":"2024-09-14T06:16:57.629763+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:16:57.629763+0000","last_undegraded":"2024-09-14T06:16:57.629763+0000","last_fullsized":"2024-09-14T06:16:57.629763+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:14:19.980571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002267542,"stat_sum":{"num_bytes":9541739,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":4,"num_read":2681,"num_read_kb":9841,"num_write":547,"num_write_kb":6183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"50'363","reported_seq":1326,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.186224+0000","last_change":"2024-09-14T06:16:22.191773+0000","last_active":"2024-09-14T06:16:56.186224+0000","last_peered":"2024-09-14T06:16:56.186224+0000","last_clean":"2024-09-14T06:16:56.186224+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:16:56.186224+0000","last_undegraded":"2024-09-14T06:16:56.186224+0000","last_fullsized":"2024-09-14T06:16:56.186224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:45:49.456951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073833899999999997,"stat_sum":{"num_bytes":10887289,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":4,"num_read":2335,"num_read_kb":7442,"num_write":566,"num_write_kb":6460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"50'170","reported_seq":950,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.197339+0000","last_change":"2024-09-14T06:16:22.205296+0000","last_active":"2024-09-14T06:16:56.197339+0000","last_peered":"2024-09-14T06:16:56.197339+0000","last_clean":"2024-09-14T06:16:56.197339+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:16:56.197339+0000","last_undegraded":"2024-09-14T06:16:56.197339+0000","last_fullsized":"2024-09-14T06:16:56.197339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:49:31.662259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012474280000000001,"stat_sum":{"num_bytes":17285695,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":2,"num_read":1655,"num_read_kb":17752,"num_write":163,"num_write_kb":11782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"50'135","reported_seq":501,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.043500+0000","last_change":"2024-09-14T06:16:22.204961+0000","last_active":"2024-09-14T06:16:56.043500+0000","last_peered":"2024-09-14T06:16:56.043500+0000","last_clean":"2024-09-14T06:16:56.043500+0000","last_became_active":"2024-09-14T06:16:20.187444+0000","last_became_peered":"2024-09-14T06:16:20.187444+0000","last_unstale":"2024-09-14T06:16:56.043500+0000","last_undegraded":"2024-09-14T06:16:56.043500+0000","last_fullsized":"2024-09-14T06:16:56.043500+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:00:26.533067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001106627,"stat_sum":{"num_bytes":14192326,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":519,"num_read_kb":14570,"num_write":154,"num_write_kb":11095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"50'130","reported_seq":615,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.594643+0000","last_change":"2024-09-14T06:16:33.544175+0000","last_active":"2024-09-14T06:16:57.594643+0000","last_peered":"2024-09-14T06:16:57.594643+0000","last_clean":"2024-09-14T06:16:57.594643+0000","last_became_active":"2024-09-14T06:16:33.543480+0000","last_became_peered":"2024-09-14T06:16:33.543480+0000","last_unstale":"2024-09-14T06:16:57.594643+0000","last_undegraded":"2024-09-14T06:16:57.594643+0000","last_fullsized":"2024-09-14T06:16:57.594643+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:12:23.207113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001557389,"stat_sum":{"num_bytes":7817922,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":4,"num_read":439,"num_read_kb":6090,"num_write":117,"num_write_kb":2905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"51'259","reported_seq":897,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.597913+0000","last_change":"2024-09-14T06:16:22.194304+0000","last_active":"2024-09-14T06:16:57.597913+0000","last_peered":"2024-09-14T06:16:57.597913+0000","last_clean":"2024-09-14T06:16:57.597913+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:16:57.597913+0000","last_undegraded":"2024-09-14T06:16:57.597913+0000","last_fullsized":"2024-09-14T06:16:57.597913+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:10.096640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019733379999999998,"stat_sum":{"num_bytes":13601260,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":2,"num_read":1537,"num_read_kb":11215,"num_write":363,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"50'133","reported_seq":319,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.596202+0000","last_change":"2024-09-14T06:16:40.063145+0000","last_active":"2024-09-14T06:16:57.596202+0000","last_peered":"2024-09-14T06:16:57.596202+0000","last_clean":"2024-09-14T06:16:57.596202+0000","last_became_active":"2024-09-14T06:16:39.975752+0000","last_became_peered":"2024-09-14T06:16:39.975752+0000","last_unstale":"2024-09-14T06:16:57.596202+0000","last_undegraded":"2024-09-14T06:16:57.596202+0000","last_fullsized":"2024-09-14T06:16:57.596202+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:30:51.600175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001465247,"stat_sum":{"num_bytes":24408554,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1125,"num_read_kb":20995,"num_write":138,"num_write_kb":11501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9287382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":78,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.049395+0000","last_change":"2024-09-14T06:16:12.179717+0000","last_active":"2024-09-14T06:16:57.049395+0000","last_peered":"2024-09-14T06:16:57.049395+0000","last_clean":"2024-09-14T06:16:57.049395+0000","last_became_active":"2024-09-14T06:16:10.150426+0000","last_became_peered":"2024-09-14T06:16:10.150426+0000","last_unstale":"2024-09-14T06:16:57.049395+0000","last_undegraded":"2024-09-14T06:16:57.049395+0000","last_fullsized":"2024-09-14T06:16:57.049395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:29:55.403337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010688049999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"50'266","reported_seq":639,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.184415+0000","last_change":"2024-09-14T06:16:22.192972+0000","last_active":"2024-09-14T06:16:56.184415+0000","last_peered":"2024-09-14T06:16:56.184415+0000","last_clean":"2024-09-14T06:16:56.184415+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:16:56.184415+0000","last_undegraded":"2024-09-14T06:16:56.184415+0000","last_fullsized":"2024-09-14T06:16:56.184415+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:59:42.007141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086504599999999996,"stat_sum":{"num_bytes":22141482,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":3,"num_read":1126,"num_read_kb":7518,"num_write":327,"num_write_kb":6876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":76,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.042767+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:16:56.042767+0000","last_peered":"2024-09-14T06:16:56.042767+0000","last_clean":"2024-09-14T06:16:56.042767+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:16:56.042767+0000","last_undegraded":"2024-09-14T06:16:56.042767+0000","last_fullsized":"2024-09-14T06:16:56.042767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"50'169","reported_seq":1923,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.050184+0000","last_change":"2024-09-14T06:16:22.192683+0000","last_active":"2024-09-14T06:16:56.050184+0000","last_peered":"2024-09-14T06:16:56.050184+0000","last_clean":"2024-09-14T06:16:56.050184+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:16:56.050184+0000","last_undegraded":"2024-09-14T06:16:56.050184+0000","last_fullsized":"2024-09-14T06:16:56.050184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:14:14.300816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001328716,"stat_sum":{"num_bytes":21871443,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":4,"num_read":2264,"num_read_kb":15484,"num_write":160,"num_write_kb":4259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":78,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.049623+0000","last_change":"2024-09-14T06:16:12.180261+0000","last_active":"2024-09-14T06:16:57.049623+0000","last_peered":"2024-09-14T06:16:57.049623+0000","last_clean":"2024-09-14T06:16:57.049623+0000","last_became_active":"2024-09-14T06:16:10.147337+0000","last_became_peered":"2024-09-14T06:16:10.147337+0000","last_unstale":"2024-09-14T06:16:57.049623+0000","last_undegraded":"2024-09-14T06:16:57.049623+0000","last_fullsized":"2024-09-14T06:16:57.049623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:54:30.677581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001722187,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"49'146","reported_seq":1150,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.036280+0000","last_change":"2024-09-14T06:16:22.213152+0000","last_active":"2024-09-14T06:16:56.036280+0000","last_peered":"2024-09-14T06:16:56.036280+0000","last_clean":"2024-09-14T06:16:56.036280+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:16:56.036280+0000","last_undegraded":"2024-09-14T06:16:56.036280+0000","last_fullsized":"2024-09-14T06:16:56.036280+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:16:42.614154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057590699999999996,"stat_sum":{"num_bytes":36266956,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":3,"num_read":964,"num_read_kb":10897,"num_write":141,"num_write_kb":5884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":76,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.042479+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:16:56.042479+0000","last_peered":"2024-09-14T06:16:56.042479+0000","last_clean":"2024-09-14T06:16:56.042479+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:16:56.042479+0000","last_undegraded":"2024-09-14T06:16:56.042479+0000","last_fullsized":"2024-09-14T06:16:56.042479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"50'98","reported_seq":999,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.130666+0000","last_change":"2024-09-14T06:16:33.548583+0000","last_active":"2024-09-14T06:16:56.130666+0000","last_peered":"2024-09-14T06:16:56.130666+0000","last_clean":"2024-09-14T06:16:56.130666+0000","last_became_active":"2024-09-14T06:16:33.547925+0000","last_became_peered":"2024-09-14T06:16:33.547925+0000","last_unstale":"2024-09-14T06:16:56.130666+0000","last_undegraded":"2024-09-14T06:16:56.130666+0000","last_fullsized":"2024-09-14T06:16:56.130666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:33:32.379146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089329700000000004,"stat_sum":{"num_bytes":30292586,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":911,"num_read_kb":15977,"num_write":115,"num_write_kb":8085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"51'64","reported_seq":369,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.599324+0000","last_change":"2024-09-14T06:16:27.282745+0000","last_active":"2024-09-14T06:16:57.599324+0000","last_peered":"2024-09-14T06:16:57.599324+0000","last_clean":"2024-09-14T06:16:57.599324+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:16:57.599324+0000","last_undegraded":"2024-09-14T06:16:57.599324+0000","last_fullsized":"2024-09-14T06:16:57.599324+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:00.587122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002354358,"stat_sum":{"num_bytes":14609202,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":225,"num_read_kb":3498,"num_write":61,"num_write_kb":2055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.048656+0000","last_change":"2024-09-14T06:16:12.182029+0000","last_active":"2024-09-14T06:16:57.048656+0000","last_peered":"2024-09-14T06:16:57.048656+0000","last_clean":"2024-09-14T06:16:57.048656+0000","last_became_active":"2024-09-14T06:16:10.196390+0000","last_became_peered":"2024-09-14T06:16:10.196390+0000","last_unstale":"2024-09-14T06:16:57.048656+0000","last_undegraded":"2024-09-14T06:16:57.048656+0000","last_fullsized":"2024-09-14T06:16:57.048656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:21.237087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034648000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"50'161","reported_seq":1194,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.197041+0000","last_change":"2024-09-14T06:16:22.205051+0000","last_active":"2024-09-14T06:16:56.197041+0000","last_peered":"2024-09-14T06:16:56.197041+0000","last_clean":"2024-09-14T06:16:56.197041+0000","last_became_active":"2024-09-14T06:16:20.196215+0000","last_became_peered":"2024-09-14T06:16:20.196215+0000","last_unstale":"2024-09-14T06:16:56.197041+0000","last_undegraded":"2024-09-14T06:16:56.197041+0000","last_fullsized":"2024-09-14T06:16:56.197041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:51:02.113439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001116559,"stat_sum":{"num_bytes":31348078,"num_objects":23,"num_object_clones":2,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":2,"num_read":1277,"num_read_kb":13371,"num_write":214,"num_write_kb":9918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":76,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.092447+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:16:56.092447+0000","last_peered":"2024-09-14T06:16:56.092447+0000","last_clean":"2024-09-14T06:16:56.092447+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:16:56.092447+0000","last_undegraded":"2024-09-14T06:16:56.092447+0000","last_fullsized":"2024-09-14T06:16:56.092447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"50'299","reported_seq":487,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.184363+0000","last_change":"2024-09-14T06:16:27.316499+0000","last_active":"2024-09-14T06:16:56.184363+0000","last_peered":"2024-09-14T06:16:56.184363+0000","last_clean":"2024-09-14T06:16:56.184363+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:16:56.184363+0000","last_undegraded":"2024-09-14T06:16:56.184363+0000","last_fullsized":"2024-09-14T06:16:56.184363+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:19:52.949868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052773999999999996,"stat_sum":{"num_bytes":9782492,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":2,"num_read":1145,"num_read_kb":3513,"num_write":441,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":78,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.048293+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:16:57.048293+0000","last_peered":"2024-09-14T06:16:57.048293+0000","last_clean":"2024-09-14T06:16:57.048293+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:16:57.048293+0000","last_undegraded":"2024-09-14T06:16:57.048293+0000","last_fullsized":"2024-09-14T06:16:57.048293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"50'151","reported_seq":587,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.196942+0000","last_change":"2024-09-14T06:16:27.302045+0000","last_active":"2024-09-14T06:16:56.196942+0000","last_peered":"2024-09-14T06:16:56.196942+0000","last_clean":"2024-09-14T06:16:56.196942+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:16:56.196942+0000","last_undegraded":"2024-09-14T06:16:56.196942+0000","last_fullsized":"2024-09-14T06:16:56.196942+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:47:50.881789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060103200000000004,"stat_sum":{"num_bytes":10468025,"num_objects":26,"num_object_clones":3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":3,"num_read":426,"num_read_kb":5318,"num_write":151,"num_write_kb":3796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":99,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.048993+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:16:57.048993+0000","last_peered":"2024-09-14T06:16:57.048993+0000","last_clean":"2024-09-14T06:16:57.048993+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:16:57.048993+0000","last_undegraded":"2024-09-14T06:16:57.048993+0000","last_fullsized":"2024-09-14T06:16:57.048993+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":169,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.062799+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:16:57.062799+0000","last_peered":"2024-09-14T06:16:57.062799+0000","last_clean":"2024-09-14T06:16:57.062799+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:16:57.062799+0000","last_undegraded":"2024-09-14T06:16:57.062799+0000","last_fullsized":"2024-09-14T06:16:57.062799+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"49'86","reported_seq":240,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.042760+0000","last_change":"2024-09-14T06:16:22.191260+0000","last_active":"2024-09-14T06:16:56.042760+0000","last_peered":"2024-09-14T06:16:56.042760+0000","last_clean":"2024-09-14T06:16:56.042760+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:16:56.042760+0000","last_undegraded":"2024-09-14T06:16:56.042760+0000","last_fullsized":"2024-09-14T06:16:56.042760+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:36:01.717785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047498199999999999,"stat_sum":{"num_bytes":9165280,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":104,"num_read_kb":3165,"num_write":78,"num_write_kb":2954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"51'76","reported_seq":337,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.618589+0000","last_change":"2024-09-14T06:16:22.322878+0000","last_active":"2024-09-14T06:16:57.618589+0000","last_peered":"2024-09-14T06:16:57.618589+0000","last_clean":"2024-09-14T06:16:57.618589+0000","last_became_active":"2024-09-14T06:16:20.196837+0000","last_became_peered":"2024-09-14T06:16:20.196837+0000","last_unstale":"2024-09-14T06:16:57.618589+0000","last_undegraded":"2024-09-14T06:16:57.618589+0000","last_fullsized":"2024-09-14T06:16:57.618589+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:47:07.994348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418299,"stat_sum":{"num_bytes":22818137,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":2,"num_read":206,"num_read_kb":3414,"num_write":66,"num_write_kb":4039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"51'145","reported_seq":1064,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.406463+0000","last_change":"2024-09-14T06:16:22.323725+0000","last_active":"2024-09-14T06:16:57.406463+0000","last_peered":"2024-09-14T06:16:57.406463+0000","last_clean":"2024-09-14T06:16:57.406463+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:16:57.406463+0000","last_undegraded":"2024-09-14T06:16:57.406463+0000","last_fullsized":"2024-09-14T06:16:57.406463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:23:09.013334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024221899999999998,"stat_sum":{"num_bytes":22218337,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1403,"num_read_kb":7277,"num_write":141,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"50'72","reported_seq":865,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.186629+0000","last_change":"2024-09-14T06:16:22.219322+0000","last_active":"2024-09-14T06:16:56.186629+0000","last_peered":"2024-09-14T06:16:56.186629+0000","last_clean":"2024-09-14T06:16:56.186629+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:16:56.186629+0000","last_undegraded":"2024-09-14T06:16:56.186629+0000","last_fullsized":"2024-09-14T06:16:56.186629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:17:30.268584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045017199999999999,"stat_sum":{"num_bytes":11504344,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":2,"num_read":754,"num_read_kb":9718,"num_write":75,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"50'56","reported_seq":1066,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.598704+0000","last_change":"2024-09-14T06:16:22.195705+0000","last_active":"2024-09-14T06:16:57.598704+0000","last_peered":"2024-09-14T06:16:57.598704+0000","last_clean":"2024-09-14T06:16:57.598704+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:16:57.598704+0000","last_undegraded":"2024-09-14T06:16:57.598704+0000","last_fullsized":"2024-09-14T06:16:57.598704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:11:00.846479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025578329999999998,"stat_sum":{"num_bytes":22004370,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":1,"num_read":949,"num_read_kb":13880,"num_write":80,"num_write_kb":5029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"51'489","reported_seq":1215,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.595063+0000","last_change":"2024-09-14T06:16:22.323126+0000","last_active":"2024-09-14T06:16:57.595063+0000","last_peered":"2024-09-14T06:16:57.595063+0000","last_clean":"2024-09-14T06:16:57.595063+0000","last_became_active":"2024-09-14T06:16:20.190580+0000","last_became_peered":"2024-09-14T06:16:20.190580+0000","last_unstale":"2024-09-14T06:16:57.595063+0000","last_undegraded":"2024-09-14T06:16:57.595063+0000","last_fullsized":"2024-09-14T06:16:57.595063+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:31:47.329967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001627671,"stat_sum":{"num_bytes":25742740,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":3,"num_read":2878,"num_read_kb":17911,"num_write":819,"num_write_kb":16790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"51'256","reported_seq":1520,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.597490+0000","last_change":"2024-09-14T06:16:33.547557+0000","last_active":"2024-09-14T06:16:57.597490+0000","last_peered":"2024-09-14T06:16:57.597490+0000","last_clean":"2024-09-14T06:16:57.597490+0000","last_became_active":"2024-09-14T06:16:33.546796+0000","last_became_peered":"2024-09-14T06:16:33.546796+0000","last_unstale":"2024-09-14T06:16:57.597490+0000","last_undegraded":"2024-09-14T06:16:57.597490+0000","last_fullsized":"2024-09-14T06:16:57.597490+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:48:00.113397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026669160000000001,"stat_sum":{"num_bytes":13460745,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":1,"num_read":3400,"num_read_kb":9975,"num_write":243,"num_write_kb":6854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"51'153","reported_seq":1364,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.110749+0000","last_change":"2024-09-14T06:16:22.195247+0000","last_active":"2024-09-14T06:16:57.110749+0000","last_peered":"2024-09-14T06:16:57.110749+0000","last_clean":"2024-09-14T06:16:57.110749+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:16:57.110749+0000","last_undegraded":"2024-09-14T06:16:57.110749+0000","last_fullsized":"2024-09-14T06:16:57.110749+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:20:49.252773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022870590000000001,"stat_sum":{"num_bytes":28493429,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":3,"num_read":1232,"num_read_kb":13134,"num_write":147,"num_write_kb":9843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"51'144","reported_seq":765,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.599582+0000","last_change":"2024-09-14T06:16:22.200094+0000","last_active":"2024-09-14T06:16:57.599582+0000","last_peered":"2024-09-14T06:16:57.599582+0000","last_clean":"2024-09-14T06:16:57.599582+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:16:57.599582+0000","last_undegraded":"2024-09-14T06:16:57.599582+0000","last_fullsized":"2024-09-14T06:16:57.599582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:17:20.881203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011343239999999999,"stat_sum":{"num_bytes":10378866,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":840,"num_read_kb":6231,"num_write":134,"num_write_kb":3518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"51'181","reported_seq":1168,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.048878+0000","last_change":"2024-09-14T06:16:27.331344+0000","last_active":"2024-09-14T06:16:57.048878+0000","last_peered":"2024-09-14T06:16:57.048878+0000","last_clean":"2024-09-14T06:16:57.048878+0000","last_became_active":"2024-09-14T06:16:27.251848+0000","last_became_peered":"2024-09-14T06:16:27.251848+0000","last_unstale":"2024-09-14T06:16:57.048878+0000","last_undegraded":"2024-09-14T06:16:57.048878+0000","last_fullsized":"2024-09-14T06:16:57.048878+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:23:55.923585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226324,"stat_sum":{"num_bytes":14705776,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":6,"num_read":1662,"num_read_kb":9112,"num_write":220,"num_write_kb":5966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":66525,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"50'242","reported_seq":624,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T06:16:56.065138+0000","last_change":"2024-09-14T06:16:22.213466+0000","last_active":"2024-09-14T06:16:56.065138+0000","last_peered":"2024-09-14T06:16:56.065138+0000","last_clean":"2024-09-14T06:16:56.065138+0000","last_became_active":"2024-09-14T06:16:20.197825+0000","last_became_peered":"2024-09-14T06:16:20.197825+0000","last_unstale":"2024-09-14T06:16:56.065138+0000","last_undegraded":"2024-09-14T06:16:56.065138+0000","last_fullsized":"2024-09-14T06:16:56.065138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:27:50.618432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082016299999999995,"stat_sum":{"num_bytes":14275709,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":4,"num_read":977,"num_read_kb":3804,"num_write":314,"num_write_kb":5483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"51'254","reported_seq":2234,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T06:16:57.593410+0000","last_change":"2024-09-14T06:16:22.322160+0000","last_active":"2024-09-14T06:16:57.593410+0000","last_peered":"2024-09-14T06:16:57.593410+0000","last_clean":"2024-09-14T06:16:57.593410+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:16:57.593410+0000","last_undegraded":"2024-09-14T06:16:57.593410+0000","last_fullsized":"2024-09-14T06:16:57.593410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:32:36.015187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011502319999999999,"stat_sum":{"num_bytes":9395473,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":5,"num_read":3738,"num_read_kb":12405,"num_write":245,"num_write_kb":4176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":560623506,"num_objects":684,"num_object_clones":111,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":684,"num_whiteouts":90,"num_read":44964,"num_read_kb":319186,"num_write":8024,"num_write_kb":202844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":11113572,"num_keys_recovered":9,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36978688,"data_stored":353171859,"data_compressed":11396874,"data_compressed_allocated":24637440,"data_compressed_original":342533209,"omap_allocated":0,"internal_metadata":0},"log_size":6396,"ondisk_log_size":6396,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45332,"kb_used_data":5820,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326508,"statfs":{"total":96636764160,"available":96590344192,"internally_reserved":0,"allocated":5959680,"data_stored":53335190,"data_compressed":1943249,"data_compressed_allocated":3948544,"data_compressed_original":51829536,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40600,"kb_used_data":5200,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331240,"statfs":{"total":96636764160,"available":96595189760,"internally_reserved":0,"allocated":5324800,"data_stored":40799635,"data_compressed":1106545,"data_compressed_allocated":2789376,"data_compressed_original":38903851,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48928,"kb_used_data":5260,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94322912,"statfs":{"total":96636764160,"available":96586661888,"internally_reserved":0,"allocated":5386240,"data_stored":49134753,"data_compressed":1400092,"data_compressed_allocated":3469312,"data_compressed_original":47743580,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44980,"kb_used_data":5476,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326860,"statfs":{"total":96636764160,"available":96590704640,"internally_reserved":0,"allocated":5607424,"data_stored":47296039,"data_compressed":1366200,"data_compressed_allocated":3252224,"data_compressed_original":45383893,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34612,"kb_used_data":3428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337228,"statfs":{"total":96636764160,"available":96601321472,"internally_reserved":0,"allocated":3510272,"data_stored":28351970,"data_compressed":1094950,"data_compressed_allocated":1847296,"data_compressed_original":27138039,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40776,"kb_used_data":5428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331064,"statfs":{"total":96636764160,"available":96595009536,"internally_reserved":0,"allocated":5558272,"data_stored":43290270,"data_compressed":1532565,"data_compressed_allocated":3088384,"data_compressed_original":41461320,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542157,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40940,"kb_used_data":5596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330900,"statfs":{"total":96636764160,"available":96594841600,"internally_reserved":0,"allocated":5730304,"data_stored":52044990,"data_compressed":1886521,"data_compressed_allocated":3608576,"data_compressed_original":50520970,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34800,"kb_used_data":3612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337040,"statfs":{"total":96636764160,"available":96601128960,"internally_reserved":0,"allocated":3698688,"data_stored":32025153,"data_compressed":859602,"data_compressed_allocated":1888256,"data_compressed_original":30688276,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5046272,"data_stored":47051258,"data_compressed":1366200,"data_compressed_allocated":3252224,"data_compressed_original":45383893,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5107712,"data_stored":52210622,"data_compressed":1510860,"data_compressed_allocated":3780608,"data_compressed_original":51081820,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4988928,"data_stored":43038463,"data_compressed":1117488,"data_compressed_allocated":2953216,"data_compressed_original":41328683,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5754880,"data_stored":56823341,"data_compressed":2038788,"data_compressed_allocated":4259840,"data_compressed_original":55507744,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3133440,"data_stored":31780276,"data_compressed":859602,"data_compressed_allocated":1888256,"data_compressed_original":30688276,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5087232,"data_stored":51185263,"data_compressed":1876421,"data_compressed_allocated":3567616,"data_compressed_original":49943434,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4988928,"data_stored":43045374,"data_compressed":1532565,"data_compressed_allocated":3088384,"data_compressed_original":41461320,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2871296,"data_stored":28037262,"data_compressed":1094950,"data_compressed_allocated":1847296,"data_compressed_original":27138039,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:16:58.375 INFO:tasks.thrashosds.thrasher:Setting 3.1f to [6, 0, 1, 2] 2024-09-14T06:16:58.375 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1f', '6', '0', '1', '2'] 2024-09-14T06:16:58.375 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1f 6 0 1 2 2024-09-14T06:16:58.393 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:58.393 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:58.394 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:58.390+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:58.414 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:16:58.494 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:58.494+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:58.589 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:58.589 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:58.597 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:58.595+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:58.604 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:16:58.698 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:58.695+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:58.731 INFO:teuthology.orchestra.run.smithi039.stdout:2930 write 0x155945a thru 0x1566e61 (0xda08 bytes) 2024-09-14T06:16:58.756 INFO:teuthology.orchestra.run.smithi039.stdout:2933 read 0x5a33930 thru 0x5a3fb80 (0xc251 bytes) 2024-09-14T06:16:58.765 INFO:teuthology.orchestra.run.smithi039.stdout:2935 trunc from 0xab20aa7 to 0x5003abc 2024-09-14T06:16:58.775 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:58.775 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:58.794 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:16:58.835 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:58.835+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:58.849 INFO:teuthology.orchestra.run.smithi039.stdout:2936 trunc from 0x5003abc to 0x5f2616 2024-09-14T06:16:58.900 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:58.899+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:58.909 INFO:teuthology.orchestra.run.smithi039.stdout:2937 write 0x11c2356 thru 0x11cf036 (0xcce1 bytes) 2024-09-14T06:16:58.918 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:16:58.927 INFO:teuthology.orchestra.run.smithi039.stdout:2938 trunc from 0x11cf037 to 0xb59d9ba 2024-09-14T06:16:58.939 INFO:teuthology.orchestra.run.smithi039.stdout:2939 read 0x131fd84 thru 0x132ba0f (0xbc8c bytes) 2024-09-14T06:16:58.939 INFO:teuthology.orchestra.run.smithi039.stdout:2940 read 0x7ff1dcc thru 0x7ff6c79 (0x4eae bytes) 2024-09-14T06:16:58.939 INFO:teuthology.orchestra.run.smithi039.stdout:2941 write 0x7396d50 thru 0x73a1298 (0xa549 bytes) 2024-09-14T06:16:58.946 INFO:teuthology.orchestra.run.smithi039.stdout:2943 write 0x74fbdab thru 0x750267d (0x68d3 bytes) 2024-09-14T06:16:58.948 INFO:teuthology.orchestra.run.smithi039.stdout:2944 read 0x7abd15f thru 0x7abe3d9 (0x127b bytes) 2024-09-14T06:16:58.958 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:16:58.958 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:16:58.960 INFO:teuthology.orchestra.run.smithi039.stdout:2945 write 0xc4067e1 thru 0xc4116e1 (0xaf01 bytes) 2024-09-14T06:16:58.963 INFO:teuthology.orchestra.run.smithi039.stdout:2946 punch from 0x844e361 to 0x8453607, (0x52a6 bytes) 2024-09-14T06:16:58.963 INFO:teuthology.orchestra.run.smithi039.stdout:2947 writesame 0x409718d thru 0x40a074d data_size 0xd(0x95c1 bytes) 2024-09-14T06:16:58.970 INFO:teuthology.orchestra.run.smithi039.stdout:2948 write 0xbc5da1 thru 0xbc5eac (0x10c bytes) 2024-09-14T06:16:58.973 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:16:58.976 INFO:teuthology.orchestra.run.smithi039.stdout:2949 write 0x42517a8 thru 0x425a581 (0x8dda bytes) 2024-09-14T06:16:58.979 INFO:teuthology.orchestra.run.smithi039.stdout:2950 write 0x4b5c008 thru 0x4b6adc8 (0xedc1 bytes) 2024-09-14T06:16:58.986 INFO:teuthology.orchestra.run.smithi039.stdout:2952 write 0x1a8782c thru 0x1a902f1 (0x8ac6 bytes) 2024-09-14T06:16:58.992 INFO:teuthology.orchestra.run.smithi039.stdout:2954 read 0x183a912 thru 0x184a4be (0xfbad bytes) 2024-09-14T06:16:58.993 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:58.993 INFO:teuthology.orchestra.run.smithi039.stdout:2957 punch from 0x45f51b4 to 0x45f6745, (0x1591 bytes) 2024-09-14T06:16:58.993 INFO:teuthology.orchestra.run.smithi039.stdout:2958 write 0x616bd09 thru 0x6176858 (0xab50 bytes) 2024-09-14T06:16:58.995 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:58.995 INFO:teuthology.orchestra.run.smithi039.stdout:2961 write 0x568f10e thru 0x569e761 (0xf654 bytes) 2024-09-14T06:16:58.998 INFO:teuthology.orchestra.run.smithi039.stdout:2962 punch from 0x5605b7 to 0x56ccbb, (0xc704 bytes) 2024-09-14T06:16:58.998 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:58.998 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:58.998 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:58.998 INFO:teuthology.orchestra.run.smithi039.stdout:2967 writesame 0x46570df thru 0x4660bde data_size 0x9b(0x9b00 bytes) 2024-09-14T06:16:59.000 INFO:teuthology.orchestra.run.smithi039.stdout:2968 writesame 0x4be439e thru 0x4be7716 data_size 0x1(0x3379 bytes) 2024-09-14T06:16:59.000 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:58.999+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:59.016 INFO:teuthology.orchestra.run.smithi039.stdout:2969 read 0xde3243 thru 0xde34e1 (0x29f bytes) 2024-09-14T06:16:59.017 INFO:teuthology.orchestra.run.smithi039.stdout:2970 read 0x279e1f0 thru 0x27a0ab3 (0x28c4 bytes) 2024-09-14T06:16:59.017 INFO:teuthology.orchestra.run.smithi039.stdout:2971 punch from 0xa0257f to 0xa1119a, (0xec1b bytes) 2024-09-14T06:16:59.017 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:59.030 INFO:teuthology.orchestra.run.smithi039.stdout:2974 trunc from 0xc4116e2 to 0x286d3ff 2024-09-14T06:16:59.057 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:16:59.067 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:16:59.067 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:16:59.079 INFO:teuthology.orchestra.run.smithi039.stdout:2976 writesame 0xa370d16 thru 0xa380822 data_size 0x6f(0xfb0d bytes) 2024-09-14T06:16:59.083 INFO:teuthology.orchestra.run.smithi039.stdout:2977 trunc from 0xa380823 to 0xbc0bb0f 2024-09-14T06:16:59.084 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:16:59.085 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:16:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:set 3.1f pg_upmap_items mapping to [6->0,1->2] 2024-09-14T06:16:59.095 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-09-14T06:16:59.101 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:59.098+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:59.106 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:16:59.106 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:16:59.110 INFO:teuthology.orchestra.run.smithi039.stdout:2978 trunc from 0xbc0bb0f to 0x7faf021 2024-09-14T06:16:59.113 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:59.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:59.114 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:59.114 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:59.114 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:59.128 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:16:59.129 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:16:59.132 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:16:59.146 INFO:teuthology.orchestra.run.smithi039.stdout:2980 write 0xc245516 thru 0xc24b8d6 (0x63c1 bytes) 2024-09-14T06:16:59.147 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:16:59.147 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:16:59.160 INFO:teuthology.orchestra.run.smithi039.stdout:2981 writesame 0xc2ea4e8 thru 0xc2eaba1 data_size 0x7(0x6ba bytes) 2024-09-14T06:16:59.162 INFO:teuthology.orchestra.run.smithi039.stdout:2983 punch from 0x6b71e8a to 0x6b7e422, (0xc598 bytes) 2024-09-14T06:16:59.165 INFO:teuthology.orchestra.run.smithi039.stdout:2985 trunc from 0xc2eaba2 to 0xa2ff793 2024-09-14T06:16:59.172 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:16:59.172 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:16:59.184 INFO:teuthology.orchestra.run.smithi039.stdout:2986 punch from 0x953b72b to 0x954335b, (0x7c30 bytes) 2024-09-14T06:16:59.184 INFO:teuthology.orchestra.run.smithi039.stdout:2988 punch from 0x529e1ef to 0x529f402, (0x1213 bytes) 2024-09-14T06:16:59.184 INFO:teuthology.orchestra.run.smithi039.stdout:2989 write 0x97798e5 thru 0x9785eb6 (0xc5d2 bytes) 2024-09-14T06:16:59.194 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:16:59.194 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:16:59.202 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:16:59.202+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:16:59.214 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:16:59.214 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:16:59.262 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:59.263 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:59.263 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:59.263 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:59.263 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.263 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.5 30/29 [PushOp(3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head, version: 28'5, data_included: [0~212992,360448~311296], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a0c56f00:::rbd_data.1152bd92e04f.000000000000000f:head@28'5, size: 1032192, copy_subset: [(0, 1032192)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1032192, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:16:59.263 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:59.263 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 31.985770996999999, 2024-09-14T06:16:59.263 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041630249999999999, 2024-09-14T06:16:59.263 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.263 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.264 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.264 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.264 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.264 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.264 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:59.264 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.264 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.264 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.264 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275886+0000", 2024-09-14T06:16:59.264 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.264 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.264 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.265 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.275888+0000", 2024-09-14T06:16:59.265 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.367e-06 2024-09-14T06:16:59.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.265 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.265 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.265 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277098+0000", 2024-09-14T06:16:59.265 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001210661 2024-09-14T06:16:59.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.265 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.265 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.265 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277122+0000", 2024-09-14T06:16:59.265 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3413000000000001e-05 2024-09-14T06:16:59.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.266 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.266 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277141+0000", 2024-09-14T06:16:59.266 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9133000000000001e-05 2024-09-14T06:16:59.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.266 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.266 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277364+0000", 2024-09-14T06:16:59.266 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000223365 2024-09-14T06:16:59.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.266 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.266 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.277405+0000", 2024-09-14T06:16:59.266 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1053000000000001e-05 2024-09-14T06:16:59.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.267 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.267 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.267 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.280049+0000", 2024-09-14T06:16:59.267 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026440330000000001 2024-09-14T06:16:59.267 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.267 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.267 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.267 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.267 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3868 3.16 3:6d40cb04:::rbd_data.1152bd92e04f.0000000000000012:head [stat,truncate 1998848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:59.267 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:59.267 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 31.759175627000001, 2024-09-14T06:16:59.267 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034445919999999998, 2024-09-14T06:16:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3868 2024-09-14T06:16:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502482+0000", 2024-09-14T06:16:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1900000000000002e-07 2024-09-14T06:16:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502501+0000", 2024-09-14T06:16:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8340999999999999e-05 2024-09-14T06:16:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502512+0000", 2024-09-14T06:16:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1347e-05 2024-09-14T06:16:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502518+0000", 2024-09-14T06:16:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.947e-06 2024-09-14T06:16:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502570+0000", 2024-09-14T06:16:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1858999999999999e-05 2024-09-14T06:16:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502633+0000", 2024-09-14T06:16:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2974999999999999e-05 2024-09-14T06:16:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:16:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.502795+0000", 2024-09-14T06:16:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016262899999999999 2024-09-14T06:16:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505179+0000", 2024-09-14T06:16:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002383844 2024-09-14T06:16:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505676+0000", 2024-09-14T06:16:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00049634899999999999 2024-09-14T06:16:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505816+0000", 2024-09-14T06:16:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00014026300000000001 2024-09-14T06:16:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.505926+0000", 2024-09-14T06:16:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000110319 2024-09-14T06:16:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3916 3.16 3:6ab8b1d8:::rbd_data.1152bd92e04f.0000000000000002:head [stat,truncate 1590169] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 31.682456196, 2024-09-14T06:16:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022958078999999999, 2024-09-14T06:16:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3916 2024-09-14T06:16:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:16:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579203+0000", 2024-09-14T06:16:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.748e-06 2024-09-14T06:16:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579208+0000", 2024-09-14T06:16:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:16:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579223+0000", 2024-09-14T06:16:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.508e-05 2024-09-14T06:16:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579232+0000", 2024-09-14T06:16:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9719999999999998e-06 2024-09-14T06:16:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579307+0000", 2024-09-14T06:16:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5028000000000001e-05 2024-09-14T06:16:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579412+0000", 2024-09-14T06:16:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010525400000000001 2024-09-14T06:16:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:16:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579535+0000", 2024-09-14T06:16:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122726 2024-09-14T06:16:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.581415+0000", 2024-09-14T06:16:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018800360000000001 2024-09-14T06:16:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602017+0000", 2024-09-14T06:16:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.020602425000000001 2024-09-14T06:16:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602096+0000", 2024-09-14T06:16:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8968999999999996e-05 2024-09-14T06:16:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.279 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602159+0000", 2024-09-14T06:16:59.279 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2824999999999996e-05 2024-09-14T06:16:59.279 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.279 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.279 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.279 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4873 3.e 3:76656e66:::rbd_data.1152bd92e04f.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=2b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:16:59.279 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:59.279 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 31.068009657000001, 2024-09-14T06:16:59.279 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048796120000000002, 2024-09-14T06:16:59.279 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.279 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.279 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.280 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.280 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.280 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.280 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4873 2024-09-14T06:16:59.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.280 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.280 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.280 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.280 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:59.280 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.280 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.280 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.280 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:16:59.281 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.281 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.281 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.281 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193651+0000", 2024-09-14T06:16:59.281 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:16:59.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.281 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.281 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.281 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193663+0000", 2024-09-14T06:16:59.281 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2047e-05 2024-09-14T06:16:59.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.281 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.281 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193692+0000", 2024-09-14T06:16:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9346e-05 2024-09-14T06:16:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193708+0000", 2024-09-14T06:16:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6123000000000001e-05 2024-09-14T06:16:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193798+0000", 2024-09-14T06:16:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9909999999999998e-05 2024-09-14T06:16:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193863+0000", 2024-09-14T06:16:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4583000000000002e-05 2024-09-14T06:16:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.194717+0000", 2024-09-14T06:16:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00085473299999999999 2024-09-14T06:16:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198400+0000", 2024-09-14T06:16:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036830890000000001 2024-09-14T06:16:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198461+0000", 2024-09-14T06:16:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0844000000000002e-05 2024-09-14T06:16:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198496+0000", 2024-09-14T06:16:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4820000000000002e-05 2024-09-14T06:16:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.285 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198527+0000", 2024-09-14T06:16:59.285 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1229999999999997e-05 2024-09-14T06:16:59.285 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.285 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.285 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.285 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.285 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:6925 3.5 3:a15e9af1:::rbd_data.1152f87cbac6.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:16:59.285 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:59.285 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.146878119, 2024-09-14T06:16:59.285 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0049096490000000003, 2024-09-14T06:16:59.285 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.285 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.285 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.286 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.286 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.286 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.286 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 6925 2024-09-14T06:16:59.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.286 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.286 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.286 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:59.286 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.286 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.286 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:16:59.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.287 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.287 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114780+0000", 2024-09-14T06:16:59.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.09e-07 2024-09-14T06:16:59.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.287 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.287 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114784+0000", 2024-09-14T06:16:59.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4669999999999998e-06 2024-09-14T06:16:59.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.287 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.287 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114799+0000", 2024-09-14T06:16:59.288 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4491000000000001e-05 2024-09-14T06:16:59.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.288 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.288 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.288 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114810+0000", 2024-09-14T06:16:59.288 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1243000000000001e-05 2024-09-14T06:16:59.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.288 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.288 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.288 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115515+0000", 2024-09-14T06:16:59.288 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00070521899999999998 2024-09-14T06:16:59.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.288 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.288 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.289 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115610+0000", 2024-09-14T06:16:59.289 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4260999999999998e-05 2024-09-14T06:16:59.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.289 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.289 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:16:59.289 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115742+0000", 2024-09-14T06:16:59.289 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000132414 2024-09-14T06:16:59.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.292 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.293 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:59.293 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.116650+0000", 2024-09-14T06:16:59.293 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00090814900000000002 2024-09-14T06:16:59.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.293 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.293 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:59.293 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119568+0000", 2024-09-14T06:16:59.293 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029178440000000002 2024-09-14T06:16:59.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.293 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.293 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.293 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119636+0000", 2024-09-14T06:16:59.293 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8365e-05 2024-09-14T06:16:59.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.294 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.294 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.294 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119689+0000", 2024-09-14T06:16:59.294 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2386999999999998e-05 2024-09-14T06:16:59.294 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.294 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.294 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.294 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.294 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:7151 3.e 3:7793b17d:::rbd_data.1152f87cbac6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:16:59.294 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:59.294 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.932060328999999, 2024-09-14T06:16:59.294 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.029556783, 2024-09-14T06:16:59.295 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.295 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.295 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.295 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.295 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.295 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.295 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 7151 2024-09-14T06:16:59.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.295 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.295 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.295 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.295 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:59.295 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.296 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.296 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.296 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:16:59.296 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.296 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.296 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.296 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329599+0000", 2024-09-14T06:16:59.296 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5270000000000001e-06 2024-09-14T06:16:59.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.296 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.296 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.296 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329610+0000", 2024-09-14T06:16:59.297 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0431000000000001e-05 2024-09-14T06:16:59.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.297 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.297 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.297 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329634+0000", 2024-09-14T06:16:59.297 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3617000000000001e-05 2024-09-14T06:16:59.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.297 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.297 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.297 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329644+0000", 2024-09-14T06:16:59.297 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.095e-05 2024-09-14T06:16:59.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.297 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.297 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.298 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329712+0000", 2024-09-14T06:16:59.298 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7067e-05 2024-09-14T06:16:59.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.298 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.298 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.298 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329827+0000", 2024-09-14T06:16:59.298 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011498 2024-09-14T06:16:59.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.298 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.298 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:16:59.298 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.334271+0000", 2024-09-14T06:16:59.298 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044445839999999997 2024-09-14T06:16:59.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.298 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.299 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:16:59.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.358967+0000", 2024-09-14T06:16:59.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024696356999999999 2024-09-14T06:16:59.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.299 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.299 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:16:59.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359027+0000", 2024-09-14T06:16:59.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0006000000000001e-05 2024-09-14T06:16:59.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.299 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.299 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359099+0000", 2024-09-14T06:16:59.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1052999999999999e-05 2024-09-14T06:16:59.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.300 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.300 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.300 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359154+0000", 2024-09-14T06:16:59.300 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5210999999999997e-05 2024-09-14T06:16:59.300 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.300 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.300 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.301 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.301 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:7151 3.e e32/29 ondisk, result = 0)", 2024-09-14T06:16:59.301 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:59.301 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.912115894999999, 2024-09-14T06:16:59.301 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0096251340000000005, 2024-09-14T06:16:59.301 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.301 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.301 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.301 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.301 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.301 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.301 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:59.301 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.302 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.302 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.302 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:16:59.302 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.302 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.302 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.302 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349543+0000", 2024-09-14T06:16:59.302 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:16:59.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.302 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.302 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.302 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349552+0000", 2024-09-14T06:16:59.302 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1039999999999996e-06 2024-09-14T06:16:59.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.303 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.303 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349579+0000", 2024-09-14T06:16:59.303 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6449999999999999e-05 2024-09-14T06:16:59.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.303 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.303 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349593+0000", 2024-09-14T06:16:59.303 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4099e-05 2024-09-14T06:16:59.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.303 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.303 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359003+0000", 2024-09-14T06:16:59.304 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0094096119999999995 2024-09-14T06:16:59.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.304 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.304 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.304 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359014+0000", 2024-09-14T06:16:59.304 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0871e-05 2024-09-14T06:16:59.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.304 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.304 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.304 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359167+0000", 2024-09-14T06:16:59.304 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015300700000000001 2024-09-14T06:16:59.304 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.304 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.304 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.305 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15516 3.1 e36/29 3:860a8b0d:::rbd_data.115223d7013d.00000000000000af:head v 36'26, mlcod=36'24)", 2024-09-14T06:16:59.305 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:59.305 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.004361544999998, 2024-09-14T06:16:59.305 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035459770000000001, 2024-09-14T06:16:59.305 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.305 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.305 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.305 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.305 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.305 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:59.305 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.306 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.306 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.306 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257296+0000", 2024-09-14T06:16:59.306 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.306 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.306 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.306 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257297+0000", 2024-09-14T06:16:59.306 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4900000000000004e-07 2024-09-14T06:16:59.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.306 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.306 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.306 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257304+0000", 2024-09-14T06:16:59.306 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:16:59.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.307 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.307 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.307 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257321+0000", 2024-09-14T06:16:59.307 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6311e-05 2024-09-14T06:16:59.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.307 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.307 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.307 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257329+0000", 2024-09-14T06:16:59.307 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0399999999999993e-06 2024-09-14T06:16:59.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.307 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.307 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257414+0000", 2024-09-14T06:16:59.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4846999999999996e-05 2024-09-14T06:16:59.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.308 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.308 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.257436+0000", 2024-09-14T06:16:59.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2048999999999998e-05 2024-09-14T06:16:59.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.308 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.309 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.309 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.260796+0000", 2024-09-14T06:16:59.309 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033607559999999999 2024-09-14T06:16:59.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.309 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.309 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.309 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.260842+0000", 2024-09-14T06:16:59.309 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5226999999999999e-05 2024-09-14T06:16:59.309 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.309 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.309 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.309 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.309 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15613 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'121, mlcod=36'116)", 2024-09-14T06:16:59.310 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:59.310 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.968753039999999, 2024-09-14T06:16:59.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053619389999999996, 2024-09-14T06:16:59.310 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.310 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.310 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.310 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.310 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.310 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:59.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.311 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.311 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:16:59.311 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.311 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.311 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292905+0000", 2024-09-14T06:16:59.311 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6800000000000009e-07 2024-09-14T06:16:59.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.311 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.311 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292915+0000", 2024-09-14T06:16:59.311 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:16:59.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.312 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292927+0000", 2024-09-14T06:16:59.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-09-14T06:16:59.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.312 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292935+0000", 2024-09-14T06:16:59.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2459999999999997e-06 2024-09-14T06:16:59.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.312 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292992+0000", 2024-09-14T06:16:59.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6873999999999998e-05 2024-09-14T06:16:59.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.313 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.313 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293016+0000", 2024-09-14T06:16:59.313 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4037e-05 2024-09-14T06:16:59.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.313 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.313 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298217+0000", 2024-09-14T06:16:59.313 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052008710000000001 2024-09-14T06:16:59.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.313 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.313 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298266+0000", 2024-09-14T06:16:59.314 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9691000000000002e-05 2024-09-14T06:16:59.314 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.314 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.314 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.314 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.314 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15615 3.11 e36/29 3:8d07d636:::rbd_data.115223d7013d.0000000000000002:head v 36'122, mlcod=36'116)", 2024-09-14T06:16:59.314 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:59.314 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.967856681000001, 2024-09-14T06:16:59.314 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053069229999999998, 2024-09-14T06:16:59.314 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.314 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.314 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.314 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.315 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.315 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:59.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.315 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.315 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:59.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.315 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.315 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:16:59.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5799999999999998e-07 2024-09-14T06:16:59.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.316 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.316 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294003+0000", 2024-09-14T06:16:59.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020149600000000001 2024-09-14T06:16:59.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.316 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.316 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294016+0000", 2024-09-14T06:16:59.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3424999999999999e-05 2024-09-14T06:16:59.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.316 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.316 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294024+0000", 2024-09-14T06:16:59.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5240000000000003e-06 2024-09-14T06:16:59.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294673+0000", 2024-09-14T06:16:59.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064906700000000003 2024-09-14T06:16:59.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294698+0000", 2024-09-14T06:16:59.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5301999999999999e-05 2024-09-14T06:16:59.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299064+0000", 2024-09-14T06:16:59.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043652170000000002 2024-09-14T06:16:59.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299108+0000", 2024-09-14T06:16:59.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4033999999999999e-05 2024-09-14T06:16:59.318 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.318 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.318 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.318 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15619 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'124, mlcod=36'119)", 2024-09-14T06:16:59.318 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:59.319 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.963802733000001, 2024-09-14T06:16:59.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061545389999999997, 2024-09-14T06:16:59.319 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.319 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.319 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.319 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.319 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.319 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:59.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.319 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.319 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:59.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:16:59.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3399999999999998e-07 2024-09-14T06:16:59.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297861+0000", 2024-09-14T06:16:59.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7640000000000002e-06 2024-09-14T06:16:59.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.321 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297876+0000", 2024-09-14T06:16:59.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4881e-05 2024-09-14T06:16:59.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.321 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.321 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297884+0000", 2024-09-14T06:16:59.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.588e-06 2024-09-14T06:16:59.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.321 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.321 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298294+0000", 2024-09-14T06:16:59.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00041011200000000002 2024-09-14T06:16:59.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298317+0000", 2024-09-14T06:16:59.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3417999999999998e-05 2024-09-14T06:16:59.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303972+0000", 2024-09-14T06:16:59.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056548680000000004 2024-09-14T06:16:59.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304009+0000", 2024-09-14T06:16:59.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7073999999999998e-05 2024-09-14T06:16:59.323 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.323 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.323 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.323 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15621 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'125, mlcod=36'119)", 2024-09-14T06:16:59.323 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:59.323 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.962291715999999, 2024-09-14T06:16:59.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0069496169999999999, 2024-09-14T06:16:59.323 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.323 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.323 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.323 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.324 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:59.324 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.324 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.324 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:59.324 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.324 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.324 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:16:59.324 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2200000000000003e-07 2024-09-14T06:16:59.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.325 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.325 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299372+0000", 2024-09-14T06:16:59.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3850000000000001e-06 2024-09-14T06:16:59.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.325 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.325 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299387+0000", 2024-09-14T06:16:59.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4905e-05 2024-09-14T06:16:59.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.325 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.325 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299393+0000", 2024-09-14T06:16:59.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9090000000000003e-06 2024-09-14T06:16:59.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303591+0000", 2024-09-14T06:16:59.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041976640000000003 2024-09-14T06:16:59.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303619+0000", 2024-09-14T06:16:59.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7651e-05 2024-09-14T06:16:59.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.327 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306287+0000", 2024-09-14T06:16:59.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026686689999999998 2024-09-14T06:16:59.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.327 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.327 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306315+0000", 2024-09-14T06:16:59.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7712000000000002e-05 2024-09-14T06:16:59.327 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.327 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.327 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.327 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15623 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'126, mlcod=36'119)", 2024-09-14T06:16:59.327 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:59.327 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.958984903000001, 2024-09-14T06:16:59.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003658859, 2024-09-14T06:16:59.328 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.328 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.328 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.328 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:59.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.329 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.329 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302672+0000", 2024-09-14T06:16:59.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.329 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.329 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302673+0000", 2024-09-14T06:16:59.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4199999999999995e-07 2024-09-14T06:16:59.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.329 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302679+0000", 2024-09-14T06:16:59.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7599999999999999e-06 2024-09-14T06:16:59.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302689+0000", 2024-09-14T06:16:59.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0335e-05 2024-09-14T06:16:59.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.302697+0000", 2024-09-14T06:16:59.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3420000000000002e-06 2024-09-14T06:16:59.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304047+0000", 2024-09-14T06:16:59.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0013503530000000001 2024-09-14T06:16:59.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304059+0000", 2024-09-14T06:16:59.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2285e-05 2024-09-14T06:16:59.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306318+0000", 2024-09-14T06:16:59.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022586580000000002 2024-09-14T06:16:59.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306331+0000", 2024-09-14T06:16:59.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3383999999999999e-05 2024-09-14T06:16:59.332 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.332 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.332 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.332 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15626 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'127, mlcod=36'120)", 2024-09-14T06:16:59.332 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:59.332 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.958620507999999, 2024-09-14T06:16:59.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038671249999999999, 2024-09-14T06:16:59.333 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.333 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.333 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.333 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.333 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:59.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.333 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:59.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.334 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303037+0000", 2024-09-14T06:16:59.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0100000000000005e-07 2024-09-14T06:16:59.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.334 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303043+0000", 2024-09-14T06:16:59.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.293e-06 2024-09-14T06:16:59.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.334 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303072+0000", 2024-09-14T06:16:59.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9135000000000001e-05 2024-09-14T06:16:59.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303081+0000", 2024-09-14T06:16:59.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7080000000000003e-06 2024-09-14T06:16:59.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.305965+0000", 2024-09-14T06:16:59.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028841769999999999 2024-09-14T06:16:59.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.305993+0000", 2024-09-14T06:16:59.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8121000000000001e-05 2024-09-14T06:16:59.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306861+0000", 2024-09-14T06:16:59.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00086771099999999996 2024-09-14T06:16:59.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306904+0000", 2024-09-14T06:16:59.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3378999999999999e-05 2024-09-14T06:16:59.336 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.337 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.337 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.337 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:16521 3.e e36/34 3:77c8e3e5:::rbd_data.1152fc09e7e6.0000000000000003:head v 36'83, mlcod=36'81)", 2024-09-14T06:16:59.337 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:59.337 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.335638415999998, 2024-09-14T06:16:59.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044948899999999996, 2024-09-14T06:16:59.337 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.337 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.337 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.337 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:59.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:16:59.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926020+0000", 2024-09-14T06:16:59.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.545e-06 2024-09-14T06:16:59.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.339 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927192+0000", 2024-09-14T06:16:59.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011711919999999999 2024-09-14T06:16:59.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.339 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927215+0000", 2024-09-14T06:16:59.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3677e-05 2024-09-14T06:16:59.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.339 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927226+0000", 2024-09-14T06:16:59.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0665e-05 2024-09-14T06:16:59.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927314+0000", 2024-09-14T06:16:59.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8314999999999997e-05 2024-09-14T06:16:59.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927344+0000", 2024-09-14T06:16:59.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9941e-05 2024-09-14T06:16:59.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.930411+0000", 2024-09-14T06:16:59.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030670649999999999 2024-09-14T06:16:59.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.930514+0000", 2024-09-14T06:16:59.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010249 2024-09-14T06:16:59.341 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.341 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.341 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.341 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:18539 3.1e e40/29 3:7e28f783:::rbd_data.1152227be94f.0000000000000000:head v 40'59, mlcod=40'57)", 2024-09-14T06:16:59.341 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:16:59.341 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.978616365000001, 2024-09-14T06:16:59.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0059097430000000003, 2024-09-14T06:16:59.342 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.342 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.342 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.342 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:16:59.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:16:59.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283043+0000", 2024-09-14T06:16:59.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4499999999999998e-06 2024-09-14T06:16:59.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285878+0000", 2024-09-14T06:16:59.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028348039999999998 2024-09-14T06:16:59.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285908+0000", 2024-09-14T06:16:59.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9756000000000001e-05 2024-09-14T06:16:59.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.344 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.344 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285922+0000", 2024-09-14T06:16:59.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.43e-05 2024-09-14T06:16:59.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.344 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.344 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.286030+0000", 2024-09-14T06:16:59.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000107954 2024-09-14T06:16:59.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.344 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.345 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.286058+0000", 2024-09-14T06:16:59.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7796e-05 2024-09-14T06:16:59.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.345 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.345 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.288888+0000", 2024-09-14T06:16:59.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028295799999999999 2024-09-14T06:16:59.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.345 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.345 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.288951+0000", 2024-09-14T06:16:59.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3102999999999995e-05 2024-09-14T06:16:59.345 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.345 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.346 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.346 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:20514 3.1e e42/29 3:7e502f18:::rbd_data.1152b8cc71.0000000000000003:head v 42'90, mlcod=42'88)", 2024-09-14T06:16:59.346 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:45.814156+0000", 2024-09-14T06:16:59.346 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.447501617, 2024-09-14T06:16:59.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0042868200000000002, 2024-09-14T06:16:59.346 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.346 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.346 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.346 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.814156+0000", 2024-09-14T06:16:59.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.814156+0000", 2024-09-14T06:16:59.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.814158+0000", 2024-09-14T06:16:59.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6599999999999999e-06 2024-09-14T06:16:59.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.816022+0000", 2024-09-14T06:16:59.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018634179999999999 2024-09-14T06:16:59.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.816056+0000", 2024-09-14T06:16:59.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4445e-05 2024-09-14T06:16:59.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.816073+0000", 2024-09-14T06:16:59.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6759e-05 2024-09-14T06:16:59.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.816157+0000", 2024-09-14T06:16:59.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4109000000000004e-05 2024-09-14T06:16:59.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.816183+0000", 2024-09-14T06:16:59.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6296999999999999e-05 2024-09-14T06:16:59.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.818339+0000", 2024-09-14T06:16:59.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0021555879999999999 2024-09-14T06:16:59.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:45.818442+0000", 2024-09-14T06:16:59.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000103544 2024-09-14T06:16:59.350 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.350 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.350 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.350 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:23383 3.2 e45/29 3:416a9e38:::rbd_data.115256cac537.000000000000000e:head v 45'103, mlcod=45'101)", 2024-09-14T06:16:59.351 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:49.995203+0000", 2024-09-14T06:16:59.351 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.2664543659999996, 2024-09-14T06:16:59.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003562169, 2024-09-14T06:16:59.351 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.351 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.351 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.351 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.351 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.351 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.995203+0000", 2024-09-14T06:16:59.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.351 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.995203+0000", 2024-09-14T06:16:59.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.995204+0000", 2024-09-14T06:16:59.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5260000000000001e-06 2024-09-14T06:16:59.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.997471+0000", 2024-09-14T06:16:59.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022666959999999999 2024-09-14T06:16:59.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.997511+0000", 2024-09-14T06:16:59.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0138000000000001e-05 2024-09-14T06:16:59.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.997535+0000", 2024-09-14T06:16:59.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3485999999999998e-05 2024-09-14T06:16:59.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.997629+0000", 2024-09-14T06:16:59.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4246999999999994e-05 2024-09-14T06:16:59.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.997654+0000", 2024-09-14T06:16:59.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4545999999999998e-05 2024-09-14T06:16:59.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.998717+0000", 2024-09-14T06:16:59.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0010629669999999999 2024-09-14T06:16:59.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:49.998765+0000", 2024-09-14T06:16:59.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8563000000000002e-05 2024-09-14T06:16:59.354 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.354 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.355 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.355 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:23716 3.5 e46/34 3:a78aeb3d:::rbd_data.115256cac537.0000000000000006:head v 46'54, mlcod=45'52)", 2024-09-14T06:16:59.355 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:50.679271+0000", 2024-09-14T06:16:59.355 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.5823867029999992, 2024-09-14T06:16:59.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035774600000000002, 2024-09-14T06:16:59.355 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.355 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.355 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.355 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.679271+0000", 2024-09-14T06:16:59.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.356 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.679271+0000", 2024-09-14T06:16:59.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.356 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.679271+0000", 2024-09-14T06:16:59.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8299999999999996e-07 2024-09-14T06:16:59.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.681001+0000", 2024-09-14T06:16:59.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001730075 2024-09-14T06:16:59.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.681013+0000", 2024-09-14T06:16:59.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1821000000000001e-05 2024-09-14T06:16:59.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.681021+0000", 2024-09-14T06:16:59.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1300000000000001e-06 2024-09-14T06:16:59.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.681094+0000", 2024-09-14T06:16:59.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2173000000000002e-05 2024-09-14T06:16:59.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.681119+0000", 2024-09-14T06:16:59.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5595000000000001e-05 2024-09-14T06:16:59.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.682775+0000", 2024-09-14T06:16:59.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0016556419999999999 2024-09-14T06:16:59.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:50.682848+0000", 2024-09-14T06:16:59.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3240999999999999e-05 2024-09-14T06:16:59.359 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.359 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.359 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.359 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:24229 3.5 e47/34 3:a30fd30b:::rbd_data.1152db28e106.0000000000000001:head v 47'55, mlcod=45'53)", 2024-09-14T06:16:59.359 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:16:59.359 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 7.3516461179999997, 2024-09-14T06:16:59.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0092924199999999992, 2024-09-14T06:16:59.360 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.360 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.360 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:16:59.360 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:16:59.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:16:59.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910013+0000", 2024-09-14T06:16:59.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.812e-06 2024-09-14T06:16:59.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915374+0000", 2024-09-14T06:16:59.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053609240000000004 2024-09-14T06:16:59.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915395+0000", 2024-09-14T06:16:59.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1118e-05 2024-09-14T06:16:59.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915406+0000", 2024-09-14T06:16:59.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1303e-05 2024-09-14T06:16:59.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915490+0000", 2024-09-14T06:16:59.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4017000000000006e-05 2024-09-14T06:16:59.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915515+0000", 2024-09-14T06:16:59.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4712e-05 2024-09-14T06:16:59.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:16:59.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919240+0000", 2024-09-14T06:16:59.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037247690000000002 2024-09-14T06:16:59.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919304+0000", 2024-09-14T06:16:59.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3764999999999997e-05 2024-09-14T06:16:59.363 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.363 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.363 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.363 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.364 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.364 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:59.365 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:16:59.365 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:59.363+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:59.466 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:16:59.463+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:16:59.536 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:59.536 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:59.536 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:16:59.536 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:59.551 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:16:59.566 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:16:59.566+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:16:59.668 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:16:59.667+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:16:59.723 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:59.723 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:16:59.723 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:16:59.723 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:16:59.723 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:16:59.738 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:16:59.779 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:16:59.779+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:16:59.869 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:16:59.866+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:16:59.910 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:16:59.911 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:16:59.911 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:16:59.911 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:16:59.911 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.911 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11217 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 925696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.911 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:59.911 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.918386058999999, 2024-09-14T06:16:59.911 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049306313999999997, 2024-09-14T06:16:59.911 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.911 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.911 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.911 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.912 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.912 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.912 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11217 2024-09-14T06:16:59.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.912 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.912 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.912 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.912 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:59.912 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.912 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.912 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.912 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:59.913 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.913 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.913 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.913 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:16:59.913 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.29e-07 2024-09-14T06:16:59.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.914 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.914 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.914 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991345+0000", 2024-09-14T06:16:59.914 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1480000000000001e-06 2024-09-14T06:16:59.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.914 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.914 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.914 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991349+0000", 2024-09-14T06:16:59.914 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9840000000000003e-06 2024-09-14T06:16:59.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.915 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.915 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.915 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991359+0000", 2024-09-14T06:16:59.915 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4050000000000006e-06 2024-09-14T06:16:59.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.915 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.915 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.915 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.030826+0000", 2024-09-14T06:16:59.915 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.039467214 2024-09-14T06:16:59.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.915 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.915 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.915 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.030894+0000", 2024-09-14T06:16:59.916 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8400999999999998e-05 2024-09-14T06:16:59.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.916 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.916 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.916 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040650+0000", 2024-09-14T06:16:59.916 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0097559329999999996 2024-09-14T06:16:59.916 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.916 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.916 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.916 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.916 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11262 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 81920~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.916 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:59.916 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.918200023000001, 2024-09-14T06:16:59.917 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049588879000000002, 2024-09-14T06:16:59.917 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.917 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.917 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.917 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.917 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.917 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.917 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11262 2024-09-14T06:16:59.917 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.917 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.917 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.918 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.918 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:59.918 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.918 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.918 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.918 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:59.918 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.918 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.918 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.918 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:16:59.918 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9599999999999999e-07 2024-09-14T06:16:59.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.919 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.919 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.919 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991533+0000", 2024-09-14T06:16:59.919 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.362e-06 2024-09-14T06:16:59.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.919 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.919 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.919 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991539+0000", 2024-09-14T06:16:59.919 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:16:59.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.919 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.919 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.919 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991544+0000", 2024-09-14T06:16:59.919 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:16:59.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.920 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.920 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.920 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040755+0000", 2024-09-14T06:16:59.920 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049210938000000003 2024-09-14T06:16:59.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.920 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.920 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.920 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040833+0000", 2024-09-14T06:16:59.921 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7812999999999995e-05 2024-09-14T06:16:59.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.921 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.921 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.921 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041119+0000", 2024-09-14T06:16:59.921 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0002855 2024-09-14T06:16:59.921 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.921 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.921 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.922 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.922 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11263 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.922 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:59.922 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.918177244999999, 2024-09-14T06:16:59.922 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049944574999999998, 2024-09-14T06:16:59.922 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.922 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.922 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.922 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.922 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.922 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.922 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11263 2024-09-14T06:16:59.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.922 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.922 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.923 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.923 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:59.923 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.923 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.923 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.923 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:59.923 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.923 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.923 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.923 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:16:59.924 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.51e-07 2024-09-14T06:16:59.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.924 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.924 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.924 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991560+0000", 2024-09-14T06:16:59.924 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6719999999999998e-06 2024-09-14T06:16:59.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.924 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.924 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.924 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991567+0000", 2024-09-14T06:16:59.924 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1470000000000004e-06 2024-09-14T06:16:59.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.924 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.924 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.925 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991572+0000", 2024-09-14T06:16:59.925 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.237e-06 2024-09-14T06:16:59.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.925 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.925 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.925 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041149+0000", 2024-09-14T06:16:59.925 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049576988000000002 2024-09-14T06:16:59.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.925 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.925 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.925 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041277+0000", 2024-09-14T06:16:59.925 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000127687 2024-09-14T06:16:59.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.926 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.926 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.926 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041497+0000", 2024-09-14T06:16:59.926 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022039299999999999 2024-09-14T06:16:59.926 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.926 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.926 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.926 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.926 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11264 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 327680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.926 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:59.926 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.918150904000001, 2024-09-14T06:16:59.926 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050248567000000001, 2024-09-14T06:16:59.926 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.927 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.927 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.927 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.927 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.927 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.927 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11264 2024-09-14T06:16:59.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.927 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.927 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.927 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.927 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:59.927 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.928 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.928 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.928 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:59.928 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.928 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.928 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.928 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:16:59.928 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.41e-07 2024-09-14T06:16:59.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.928 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.928 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.928 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991581+0000", 2024-09-14T06:16:59.929 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3740000000000001e-06 2024-09-14T06:16:59.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.929 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.929 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.929 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991585+0000", 2024-09-14T06:16:59.929 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9400000000000004e-06 2024-09-14T06:16:59.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.929 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.929 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.929 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991587+0000", 2024-09-14T06:16:59.929 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.694e-06 2024-09-14T06:16:59.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.929 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.929 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.930 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041527+0000", 2024-09-14T06:16:59.930 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049939349000000001 2024-09-14T06:16:59.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.930 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.930 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.930 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041598+0000", 2024-09-14T06:16:59.930 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1633999999999999e-05 2024-09-14T06:16:59.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.930 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.930 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.930 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041827+0000", 2024-09-14T06:16:59.930 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000229235 2024-09-14T06:16:59.930 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.930 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.931 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.931 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.931 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.931 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:59.931 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.918137563999998, 2024-09-14T06:16:59.931 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:16:59.931 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.931 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.931 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.931 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.931 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.931 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.931 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:16:59.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.932 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.932 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.932 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.932 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:59.932 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.932 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.932 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.932 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:59.932 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.933 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.933 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.933 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:16:59.933 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:16:59.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.933 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.933 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.933 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:16:59.933 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:16:59.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.934 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.934 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.934 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:16:59.934 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:16:59.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.934 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.934 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.934 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:16:59.934 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:16:59.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.935 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.935 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.935 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:16:59.935 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:16:59.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.935 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.935 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.935 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:16:59.935 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:16:59.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.936 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.936 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.936 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:16:59.936 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:16:59.936 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.936 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.936 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.936 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.936 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.936 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:59.936 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.918124492, 2024-09-14T06:16:59.936 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:16:59.936 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.937 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.937 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.937 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.937 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.937 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.937 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:16:59.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.937 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.937 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.937 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.937 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:59.937 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.937 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.938 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.938 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:16:59.938 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.938 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.938 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.938 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:16:59.938 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:16:59.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.938 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.938 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.938 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:16:59.938 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:16:59.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.939 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.939 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.939 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:16:59.939 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:16:59.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.939 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.939 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.939 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:16:59.939 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:16:59.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.939 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.939 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.939 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:16:59.940 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:16:59.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.940 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.940 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.940 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:16:59.940 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:16:59.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.940 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.940 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.940 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:16:59.941 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:16:59.941 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.941 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.941 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.941 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.941 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.941 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:59.941 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.918107060000001, 2024-09-14T06:16:59.941 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:16:59.941 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.942 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.942 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.942 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.942 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.942 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.942 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:16:59.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.942 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.942 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.942 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.942 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:59.942 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.943 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.943 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.943 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:59.943 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.943 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.943 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.943 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:16:59.943 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:16:59.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.943 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.943 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.943 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:16:59.944 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:16:59.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.944 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.944 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.944 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:16:59.944 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:16:59.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.944 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.944 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.944 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:16:59.944 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:16:59.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.945 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.945 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.945 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:16:59.945 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:16:59.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.945 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.945 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.945 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:16:59.945 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:16:59.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.945 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.945 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.945 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:16:59.946 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:16:59.946 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.946 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.946 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.946 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.946 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.946 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:59.946 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.918085954999999, 2024-09-14T06:16:59.946 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:16:59.946 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.947 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.947 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.947 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.947 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.947 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.947 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:16:59.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.947 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.947 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.947 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.947 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:59.948 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.948 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.948 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.948 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:59.948 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.948 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.948 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.948 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:16:59.948 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:16:59.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.948 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.948 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.949 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:16:59.949 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:16:59.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.949 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.949 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:16:59.949 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:16:59.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.949 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.949 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:16:59.949 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:16:59.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.950 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.950 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:16:59.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:16:59.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.950 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.950 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.950 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:16:59.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:16:59.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.950 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.950 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.950 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:16:59.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:16:59.951 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.951 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.951 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.951 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.951 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.951 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:59.951 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.918064847, 2024-09-14T06:16:59.951 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:16:59.951 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.951 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.951 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.951 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.951 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.952 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.952 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:16:59.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.952 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.952 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.952 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.952 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:59.952 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.952 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.952 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.952 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:59.953 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.953 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.953 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.953 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:16:59.953 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:16:59.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.953 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.953 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.953 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:16:59.953 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:16:59.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.953 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.953 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.954 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:16:59.954 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:16:59.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.954 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.954 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.954 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:16:59.954 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:16:59.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.954 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.954 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.954 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:16:59.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:16:59.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.955 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.955 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:16:59.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:16:59.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.955 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.955 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:16:59.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:16:59.955 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.955 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.955 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.956 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.956 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13590 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 491520~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.956 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:59.956 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.772089964999999, 2024-09-14T06:16:59.956 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042553466999999998, 2024-09-14T06:16:59.956 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.956 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.956 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.956 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.956 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.956 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.956 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13590 2024-09-14T06:16:59.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.957 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.957 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.957 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:59.957 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.957 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.957 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:59.957 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.957 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.957 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:16:59.958 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.34e-07 2024-09-14T06:16:59.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.958 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.958 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137642+0000", 2024-09-14T06:16:59.958 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0150000000000002e-06 2024-09-14T06:16:59.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.958 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.959 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137649+0000", 2024-09-14T06:16:59.959 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4010000000000002e-06 2024-09-14T06:16:59.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.959 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.959 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.959 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137653+0000", 2024-09-14T06:16:59.959 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:16:59.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.959 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.959 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.960 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.179985+0000", 2024-09-14T06:16:59.960 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042331580000000001 2024-09-14T06:16:59.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.960 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.960 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180076+0000", 2024-09-14T06:16:59.960 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0796000000000003e-05 2024-09-14T06:16:59.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.960 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.961 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180193+0000", 2024-09-14T06:16:59.961 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000117471 2024-09-14T06:16:59.961 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.961 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.961 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.961 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.961 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13592 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 606208~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.961 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:59.961 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.772065396999999, 2024-09-14T06:16:59.961 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042709440000000001, 2024-09-14T06:16:59.961 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.962 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.962 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.962 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.962 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.962 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.962 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13592 2024-09-14T06:16:59.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.962 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.962 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.962 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:59.963 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.963 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.963 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:16:59.963 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.963 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.963 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137665+0000", 2024-09-14T06:16:59.963 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4299999999999998e-07 2024-09-14T06:16:59.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.964 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.964 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137667+0000", 2024-09-14T06:16:59.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5780000000000001e-06 2024-09-14T06:16:59.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.964 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.964 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137675+0000", 2024-09-14T06:16:59.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0870000000000001e-06 2024-09-14T06:16:59.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.965 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.965 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137680+0000", 2024-09-14T06:16:59.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4999999999999999e-06 2024-09-14T06:16:59.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.965 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.965 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180223+0000", 2024-09-14T06:16:59.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042542808000000001 2024-09-14T06:16:59.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.965 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.966 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180294+0000", 2024-09-14T06:16:59.966 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1497999999999999e-05 2024-09-14T06:16:59.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.966 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.966 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.966 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180374+0000", 2024-09-14T06:16:59.966 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9525999999999997e-05 2024-09-14T06:16:59.966 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.966 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.966 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.966 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.966 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13594 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 729088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.966 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:59.967 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.772041832999999, 2024-09-14T06:16:59.967 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042830604000000001, 2024-09-14T06:16:59.967 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.967 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.967 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.967 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.967 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.967 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.967 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13594 2024-09-14T06:16:59.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.967 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.967 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.967 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:59.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.968 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.968 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:59.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.968 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.968 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:16:59.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0699999999999998e-07 2024-09-14T06:16:59.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.968 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137690+0000", 2024-09-14T06:16:59.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.012e-06 2024-09-14T06:16:59.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137698+0000", 2024-09-14T06:16:59.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4379999999999998e-06 2024-09-14T06:16:59.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137703+0000", 2024-09-14T06:16:59.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1250000000000001e-06 2024-09-14T06:16:59.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.970 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.970 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180404+0000", 2024-09-14T06:16:59.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042701465000000001 2024-09-14T06:16:59.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.970 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.970 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180456+0000", 2024-09-14T06:16:59.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1227999999999998e-05 2024-09-14T06:16:59.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.970 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.971 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180519+0000", 2024-09-14T06:16:59.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-09-14T06:16:59.971 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.971 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.971 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.971 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13595 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 851968~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.971 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:59.971 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.772019327999999, 2024-09-14T06:16:59.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043012433000000003, 2024-09-14T06:16:59.971 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.972 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.972 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.972 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.972 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.972 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.972 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13595 2024-09-14T06:16:59.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.972 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:59.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.973 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.973 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:16:59.973 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.973 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.973 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137711+0000", 2024-09-14T06:16:59.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:16:59.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137713+0000", 2024-09-14T06:16:59.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5170000000000002e-06 2024-09-14T06:16:59.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137720+0000", 2024-09-14T06:16:59.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-09-14T06:16:59.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137726+0000", 2024-09-14T06:16:59.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9580000000000004e-06 2024-09-14T06:16:59.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180551+0000", 2024-09-14T06:16:59.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042825611999999999 2024-09-14T06:16:59.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180623+0000", 2024-09-14T06:16:59.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1673000000000003e-05 2024-09-14T06:16:59.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180723+0000", 2024-09-14T06:16:59.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.9928999999999998e-05 2024-09-14T06:16:59.976 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.976 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.976 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.977 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13597 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 974848~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.977 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:59.977 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.771996442999999, 2024-09-14T06:16:59.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043180442999999999, 2024-09-14T06:16:59.977 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.977 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.977 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.977 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.977 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.977 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.978 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13597 2024-09-14T06:16:59.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.978 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:59.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:16:59.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.979 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.979 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137734+0000", 2024-09-14T06:16:59.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4499999999999997e-07 2024-09-14T06:16:59.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.979 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.979 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137736+0000", 2024-09-14T06:16:59.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0470000000000001e-06 2024-09-14T06:16:59.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137742+0000", 2024-09-14T06:16:59.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2269999999999998e-06 2024-09-14T06:16:59.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137747+0000", 2024-09-14T06:16:59.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6670000000000004e-06 2024-09-14T06:16:59.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180753+0000", 2024-09-14T06:16:59.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043006132000000002 2024-09-14T06:16:59.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180820+0000", 2024-09-14T06:16:59.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7516999999999997e-05 2024-09-14T06:16:59.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180914+0000", 2024-09-14T06:16:59.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3407999999999997e-05 2024-09-14T06:16:59.981 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.982 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.982 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.982 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13684 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 999424~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.982 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:59.982 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.769763900000001, 2024-09-14T06:16:59.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042643702999999998, 2024-09-14T06:16:59.982 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.982 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.982 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.982 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.982 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.982 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.983 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13684 2024-09-14T06:16:59.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.983 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.983 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.983 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:59.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.983 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.983 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:16:59.984 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.984 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139967+0000", 2024-09-14T06:16:59.984 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1200000000000003e-07 2024-09-14T06:16:59.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.984 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141080+0000", 2024-09-14T06:16:59.984 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011133029999999999 2024-09-14T06:16:59.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141084+0000", 2024-09-14T06:16:59.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.544e-06 2024-09-14T06:16:59.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141088+0000", 2024-09-14T06:16:59.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0280000000000002e-06 2024-09-14T06:16:59.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182457+0000", 2024-09-14T06:16:59.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041368690999999999 2024-09-14T06:16:59.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182528+0000", 2024-09-14T06:16:59.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0881000000000001e-05 2024-09-14T06:16:59.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182610+0000", 2024-09-14T06:16:59.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1644000000000006e-05 2024-09-14T06:16:59.986 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.986 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.986 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.987 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13721 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 532480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.987 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:59.987 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.768463166, 2024-09-14T06:16:59.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042577526999999997, 2024-09-14T06:16:59.987 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.987 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.987 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.987 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.987 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.987 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.987 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13721 2024-09-14T06:16:59.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.987 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:59.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:59.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:16:59.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2999999999999999e-07 2024-09-14T06:16:59.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141268+0000", 2024-09-14T06:16:59.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0130000000000001e-06 2024-09-14T06:16:59.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141271+0000", 2024-09-14T06:16:59.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0970000000000002e-06 2024-09-14T06:16:59.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141274+0000", 2024-09-14T06:16:59.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5809999999999999e-06 2024-09-14T06:16:59.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183522+0000", 2024-09-14T06:16:59.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042248591000000002 2024-09-14T06:16:59.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183691+0000", 2024-09-14T06:16:59.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016885399999999999 2024-09-14T06:16:59.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183844+0000", 2024-09-14T06:16:59.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015316099999999999 2024-09-14T06:16:59.991 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.991 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.991 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.991 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13722 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 655360~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.991 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:59.991 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.768452864, 2024-09-14T06:16:59.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042741610999999999, 2024-09-14T06:16:59.992 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.992 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.992 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.992 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.992 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.992 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.992 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13722 2024-09-14T06:16:59.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.992 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:59.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:59.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:16:59.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:16:59.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141278+0000", 2024-09-14T06:16:59.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.066e-06 2024-09-14T06:16:59.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141282+0000", 2024-09-14T06:16:59.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7100000000000001e-06 2024-09-14T06:16:59.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141285+0000", 2024-09-14T06:16:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.554e-06 2024-09-14T06:16:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:16:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183824+0000", 2024-09-14T06:16:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042539043999999998 2024-09-14T06:16:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:16:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183906+0000", 2024-09-14T06:16:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2298000000000004e-05 2024-09-14T06:16:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:16:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184019+0000", 2024-09-14T06:16:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000112724 2024-09-14T06:16:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:16:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:16:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13723 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 778240~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:16:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.768441894999999, 2024-09-14T06:16:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042995901000000003, 2024-09-14T06:16:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:16:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:16:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:16:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:16:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:16:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:16:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13723 2024-09-14T06:16:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:16:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:16:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:16:59.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:59.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:16:59.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:16:59.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:16:59.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9799999999999999e-07 2024-09-14T06:16:59.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:16:59.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141289+0000", 2024-09-14T06:16:59.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0589999999999999e-06 2024-09-14T06:16:59.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:16:59.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141292+0000", 2024-09-14T06:16:59.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.208e-06 2024-09-14T06:16:59.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:16:59.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141295+0000", 2024-09-14T06:16:59.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5160000000000001e-06 2024-09-14T06:16:59.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:16:59.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:16:59.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:00.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184069+0000", 2024-09-14T06:17:00.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042774176999999997 2024-09-14T06:17:00.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:00.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184169+0000", 2024-09-14T06:17:00.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000100073 2024-09-14T06:17:00.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:00.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184284+0000", 2024-09-14T06:17:00.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011457 2024-09-14T06:17:00.000 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:00.000 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:00.000 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:00.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.001 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13724 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 901120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.001 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:17:00.001 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.768431500999998, 2024-09-14T06:17:00.001 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043197179000000002, 2024-09-14T06:17:00.001 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:00.001 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:00.001 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:00.001 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:00.002 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:00.002 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.002 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13724 2024-09-14T06:17:00.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.002 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:00.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:00.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:17:00.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:00.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:00.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:17:00.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:00.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:00.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141299+0000", 2024-09-14T06:17:00.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.03e-07 2024-09-14T06:17:00.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:00.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141300+0000", 2024-09-14T06:17:00.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0389999999999999e-06 2024-09-14T06:17:00.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:00.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141303+0000", 2024-09-14T06:17:00.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0479999999999999e-06 2024-09-14T06:17:00.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141305+0000", 2024-09-14T06:17:00.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5739999999999998e-06 2024-09-14T06:17:00.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:00.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184317+0000", 2024-09-14T06:17:00.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043011701999999999 2024-09-14T06:17:00.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.005 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:00.005 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184394+0000", 2024-09-14T06:17:00.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7324999999999996e-05 2024-09-14T06:17:00.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.005 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:00.005 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184496+0000", 2024-09-14T06:17:00.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010118799999999999 2024-09-14T06:17:00.005 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:00.006 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:00.006 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:00.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.006 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.006 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13726 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 1024000~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.006 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:17:00.006 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.768421228000001, 2024-09-14T06:17:00.006 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043397185999999997, 2024-09-14T06:17:00.006 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:00.006 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:00.006 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:00.006 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:00.006 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:00.007 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.007 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13726 2024-09-14T06:17:00.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.007 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:00.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:00.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:17:00.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:00.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:00.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:17:00.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:00.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:00.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:17:00.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:17:00.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:00.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141310+0000", 2024-09-14T06:17:00.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:17:00.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:00.009 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141313+0000", 2024-09-14T06:17:00.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0309999999999999e-06 2024-09-14T06:17:00.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.009 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.009 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141315+0000", 2024-09-14T06:17:00.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.52e-06 2024-09-14T06:17:00.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.009 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:00.009 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184527+0000", 2024-09-14T06:17:00.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043211855 2024-09-14T06:17:00.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.009 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:00.010 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184606+0000", 2024-09-14T06:17:00.010 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8287000000000004e-05 2024-09-14T06:17:00.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:00.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:00.010 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:00.010 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184706+0000", 2024-09-14T06:17:00.010 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010021700000000001 2024-09-14T06:17:00.010 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:00.010 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:00.010 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:00.010 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:00.010 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:00.010 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:00.012 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:17:00.012 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:00.010+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:00.114 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:00.111+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:00.144 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:00.145 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:00.145 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:00.145 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:00.158 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:17:00.196 INFO:teuthology.orchestra.run.smithi039.stdout:2992 read 0x5b4abc thru 0x5b5d01 (0x1246 bytes) 2024-09-14T06:17:00.207 INFO:teuthology.orchestra.run.smithi039.stdout:2993 write 0xa5c0e12 thru 0xa5c1fec (0x11db bytes) 2024-09-14T06:17:00.214 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:00.210+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:00.223 INFO:teuthology.orchestra.run.smithi039.stdout:2994 trunc from 0xa5c1fed to 0xe6f2acd 2024-09-14T06:17:00.237 INFO:teuthology.orchestra.run.smithi039.stdout:2995 punch from 0x5e588fc to 0x5e59f12, (0x1616 bytes) 2024-09-14T06:17:00.237 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:00.238 INFO:teuthology.orchestra.run.smithi039.stdout:3000 read 0x182d9c5 thru 0x18399a6 (0xbfe2 bytes) 2024-09-14T06:17:00.241 INFO:teuthology.orchestra.run.smithi039.stdout:3001 write 0x4636dc1 thru 0x4640cfd (0x9f3d bytes) 2024-09-14T06:17:00.250 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:00.250 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:00.250 INFO:teuthology.orchestra.run.smithi039.stdout:3006 writesame 0xdc04fb0 thru 0xdc0f782 data_size 0x1(0xa7d3 bytes) 2024-09-14T06:17:00.283 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:00.283 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:00.283 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:00.283 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:00.284 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:00.297 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:17:00.307 INFO:teuthology.orchestra.run.smithi039.stdout:3008 writesame 0x9bdf229 thru 0x9bece0c data_size 0x6(0xdbe4 bytes) 2024-09-14T06:17:00.316 INFO:teuthology.orchestra.run.smithi039.stdout:3009 read 0x1134d7f thru 0x113a572 (0x57f4 bytes) 2024-09-14T06:17:00.321 INFO:teuthology.orchestra.run.smithi039.stdout:3011 punch from 0x9da6ddd to 0x9db6aba, (0xfcdd bytes) 2024-09-14T06:17:00.322 INFO:teuthology.orchestra.run.smithi039.stdout:3012 writesame 0x39e13f8 thru 0x39f1215 data_size 0xb(0xfe1e bytes) 2024-09-14T06:17:00.329 INFO:teuthology.orchestra.run.smithi039.stdout:3013 writesame 0x50f0e2d thru 0x50fcd5d data_size 0x41(0xbf31 bytes) 2024-09-14T06:17:00.333 INFO:teuthology.orchestra.run.smithi039.stdout:3016 write 0xda904ce thru 0xda9b703 (0xb236 bytes) 2024-09-14T06:17:00.337 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:00.338 INFO:teuthology.orchestra.run.smithi039.stdout:3018 punch from 0x8f769ba to 0x8f7d295, (0x68db bytes) 2024-09-14T06:17:00.338 INFO:teuthology.orchestra.run.smithi039.stdout:3020 punch from 0xcd6720 to 0xcdce53, (0x6733 bytes) 2024-09-14T06:17:00.343 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:00.342+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:00.356 INFO:teuthology.orchestra.run.smithi039.stdout:3021 trunc from 0xe6f2acd to 0x794e40f 2024-09-14T06:17:00.371 INFO:teuthology.orchestra.run.smithi039.stdout:3022 punch from 0x4fb09ad to 0x4fb2557, (0x1baa bytes) 2024-09-14T06:17:00.379 INFO:teuthology.orchestra.run.smithi039.stdout:3023 trunc from 0x794e40f to 0x47eb62a 2024-09-14T06:17:00.394 INFO:teuthology.orchestra.run.smithi039.stdout:3024 write 0x92d6e2 thru 0x92fd20 (0x263f bytes) 2024-09-14T06:17:00.397 INFO:teuthology.orchestra.run.smithi039.stdout:3025 punch from 0x466ab09 to 0x467753f, (0xca36 bytes) 2024-09-14T06:17:00.398 INFO:teuthology.orchestra.run.smithi039.stdout:3026 flatten 2024-09-14T06:17:00.408 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:00.416 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:00.414+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:00.448 INFO:teuthology.orchestra.run.smithi039.stdout:3028 writesame 0xe0ebc3d thru 0xe0ee079 data_size 0x1(0x243d bytes) 2024-09-14T06:17:00.450 INFO:teuthology.orchestra.run.smithi039.stdout:3029 read 0x5ac9533 thru 0x5ad59f3 (0xc4c1 bytes) 2024-09-14T06:17:00.451 INFO:teuthology.orchestra.run.smithi039.stdout:3030 write 0x551cad5 thru 0x551d4f9 (0xa25 bytes) 2024-09-14T06:17:00.455 INFO:teuthology.orchestra.run.smithi039.stdout:3032 read 0x7c0d6a7 thru 0x7c114eb (0x3e45 bytes) 2024-09-14T06:17:00.455 INFO:teuthology.orchestra.run.smithi039.stdout:3033 writesame 0x6e4bd58 thru 0x6e50d2d data_size 0xb(0x4fd6 bytes) 2024-09-14T06:17:00.469 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:00.469 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:00.469 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:00.469 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:00.469 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.469 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13229 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.469 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:00.470 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.369732217999999, 2024-09-14T06:17:00.470 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081798868999999996, 2024-09-14T06:17:00.470 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.470 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.470 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.470 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.470 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.470 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.470 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13229 2024-09-14T06:17:00.470 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.470 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.470 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.470 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.470 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:00.471 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.471 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.471 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.471 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.471 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:00.471 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.471 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.471 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.471 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.471 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098710+0000", 2024-09-14T06:17:00.471 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8199999999999999e-07 2024-09-14T06:17:00.471 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.471 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.471 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.471 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098714+0000", 2024-09-14T06:17:00.472 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:17:00.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.472 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.472 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.472 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098727+0000", 2024-09-14T06:17:00.472 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2520999999999999e-05 2024-09-14T06:17:00.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.472 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.472 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.472 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098749+0000", 2024-09-14T06:17:00.472 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2773999999999999e-05 2024-09-14T06:17:00.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.472 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.472 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.473 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160234+0000", 2024-09-14T06:17:00.473 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.061484430999999999 2024-09-14T06:17:00.473 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.473 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.473 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.473 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160281+0000", 2024-09-14T06:17:00.473 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7437e-05 2024-09-14T06:17:00.473 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.473 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.473 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.473 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160569+0000", 2024-09-14T06:17:00.473 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028788700000000002 2024-09-14T06:17:00.473 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.473 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.474 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.474 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163121+0000", 2024-09-14T06:17:00.474 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025514090000000001 2024-09-14T06:17:00.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.474 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.474 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.474 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180410+0000", 2024-09-14T06:17:00.474 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.017289098999999999 2024-09-14T06:17:00.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.474 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.474 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.474 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180472+0000", 2024-09-14T06:17:00.474 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2547999999999998e-05 2024-09-14T06:17:00.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.475 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.475 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.475 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180508+0000", 2024-09-14T06:17:00.475 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5534000000000002e-05 2024-09-14T06:17:00.475 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.475 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.475 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.475 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.475 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13230 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.475 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:00.475 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.369651015999999, 2024-09-14T06:17:00.475 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081851558000000005, 2024-09-14T06:17:00.475 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.475 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.476 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.476 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.476 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.476 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.476 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13230 2024-09-14T06:17:00.476 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.476 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.476 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.476 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.476 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:00.476 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.476 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.477 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.477 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.477 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:00.477 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.477 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.477 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.477 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.477 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098791+0000", 2024-09-14T06:17:00.477 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8800000000000001e-07 2024-09-14T06:17:00.477 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.477 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.477 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.477 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098795+0000", 2024-09-14T06:17:00.478 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.391e-06 2024-09-14T06:17:00.478 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.478 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.478 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.478 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098804+0000", 2024-09-14T06:17:00.478 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3440000000000001e-06 2024-09-14T06:17:00.478 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.478 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.478 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.478 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098810+0000", 2024-09-14T06:17:00.478 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2339999999999999e-06 2024-09-14T06:17:00.478 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.478 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.478 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.478 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160930+0000", 2024-09-14T06:17:00.479 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062120018999999999 2024-09-14T06:17:00.479 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.479 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.479 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.479 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161017+0000", 2024-09-14T06:17:00.479 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6868999999999998e-05 2024-09-14T06:17:00.479 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.479 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.479 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.479 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161300+0000", 2024-09-14T06:17:00.479 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028349499999999998 2024-09-14T06:17:00.479 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.479 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.479 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.479 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164612+0000", 2024-09-14T06:17:00.479 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033119680000000002 2024-09-14T06:17:00.480 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.480 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.480 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.480 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180559+0000", 2024-09-14T06:17:00.480 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015946612999999998 2024-09-14T06:17:00.480 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.480 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.480 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.480 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180605+0000", 2024-09-14T06:17:00.480 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5890000000000003e-05 2024-09-14T06:17:00.480 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.480 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.480 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.481 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180642+0000", 2024-09-14T06:17:00.481 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:17:00.481 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.481 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.481 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.481 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.481 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.481 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13231 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.481 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:00.481 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.369388602000001, 2024-09-14T06:17:00.481 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081750685000000003, 2024-09-14T06:17:00.481 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.481 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.482 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.482 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.482 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.482 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.482 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13231 2024-09-14T06:17:00.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.482 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.482 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.482 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.482 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:00.482 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.482 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.482 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.483 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:00.483 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.483 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.483 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.483 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.483 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099053+0000", 2024-09-14T06:17:00.483 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:17:00.483 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.483 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.483 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.483 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099059+0000", 2024-09-14T06:17:00.483 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1789999999999999e-06 2024-09-14T06:17:00.483 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.483 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.484 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.484 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099072+0000", 2024-09-14T06:17:00.484 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3179999999999999e-05 2024-09-14T06:17:00.484 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.484 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.484 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.484 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099080+0000", 2024-09-14T06:17:00.484 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9030000000000004e-06 2024-09-14T06:17:00.484 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.484 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.484 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.484 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161681+0000", 2024-09-14T06:17:00.484 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062601641999999999 2024-09-14T06:17:00.484 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.485 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.485 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.485 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161766+0000", 2024-09-14T06:17:00.485 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4909999999999998e-05 2024-09-14T06:17:00.485 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.485 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.485 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.485 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162109+0000", 2024-09-14T06:17:00.485 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00034264899999999999 2024-09-14T06:17:00.485 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.485 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.485 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.485 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164615+0000", 2024-09-14T06:17:00.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025061979999999998 2024-09-14T06:17:00.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180718+0000", 2024-09-14T06:17:00.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016103240000000001 2024-09-14T06:17:00.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180768+0000", 2024-09-14T06:17:00.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9781999999999999e-05 2024-09-14T06:17:00.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.487 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.487 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180803+0000", 2024-09-14T06:17:00.487 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5021999999999998e-05 2024-09-14T06:17:00.487 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.487 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.487 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.487 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.487 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.487 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13232 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.487 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:00.487 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.369346828000001, 2024-09-14T06:17:00.488 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081861981, 2024-09-14T06:17:00.488 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.488 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.488 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.488 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.488 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.488 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.488 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13232 2024-09-14T06:17:00.488 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.488 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.488 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.488 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.489 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:00.489 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.489 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.489 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.489 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:00.489 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.489 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.489 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.489 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099095+0000", 2024-09-14T06:17:00.489 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:17:00.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.489 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.490 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.490 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099098+0000", 2024-09-14T06:17:00.490 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5420000000000001e-06 2024-09-14T06:17:00.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.490 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.490 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099106+0000", 2024-09-14T06:17:00.490 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.824e-06 2024-09-14T06:17:00.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.490 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.490 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099112+0000", 2024-09-14T06:17:00.490 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9789999999999998e-06 2024-09-14T06:17:00.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.491 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.491 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162576+0000", 2024-09-14T06:17:00.491 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.063463723999999999 2024-09-14T06:17:00.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.491 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.491 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.491 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162627+0000", 2024-09-14T06:17:00.491 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1530000000000003e-05 2024-09-14T06:17:00.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.491 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.491 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.491 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162854+0000", 2024-09-14T06:17:00.491 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000226159 2024-09-14T06:17:00.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.491 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.492 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.492 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166090+0000", 2024-09-14T06:17:00.492 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032362319999999999 2024-09-14T06:17:00.492 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.492 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.492 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.492 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180877+0000", 2024-09-14T06:17:00.492 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014786726 2024-09-14T06:17:00.492 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.492 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.492 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.492 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180920+0000", 2024-09-14T06:17:00.492 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3427000000000003e-05 2024-09-14T06:17:00.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.493 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.493 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.493 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180956+0000", 2024-09-14T06:17:00.493 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6195999999999997e-05 2024-09-14T06:17:00.493 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.493 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.493 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.493 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.493 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13233 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.493 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:00.493 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.369321565, 2024-09-14T06:17:00.493 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081924860000000002, 2024-09-14T06:17:00.494 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.494 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.494 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.494 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.494 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.494 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.494 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13233 2024-09-14T06:17:00.494 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.494 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.494 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.494 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.494 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:00.494 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.494 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.494 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:00.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099120+0000", 2024-09-14T06:17:00.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:17:00.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099122+0000", 2024-09-14T06:17:00.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:17:00.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.496 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.496 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099126+0000", 2024-09-14T06:17:00.496 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2540000000000003e-06 2024-09-14T06:17:00.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.496 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.496 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099130+0000", 2024-09-14T06:17:00.496 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4060000000000001e-06 2024-09-14T06:17:00.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.496 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.496 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163169+0000", 2024-09-14T06:17:00.496 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064039415000000002 2024-09-14T06:17:00.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.497 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.497 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163260+0000", 2024-09-14T06:17:00.497 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1038000000000005e-05 2024-09-14T06:17:00.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.497 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.497 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163640+0000", 2024-09-14T06:17:00.497 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037954699999999999 2024-09-14T06:17:00.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.497 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.497 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166093+0000", 2024-09-14T06:17:00.497 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024533229999999999 2024-09-14T06:17:00.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.498 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180993+0000", 2024-09-14T06:17:00.498 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01489997 2024-09-14T06:17:00.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.498 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181020+0000", 2024-09-14T06:17:00.498 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6806999999999999e-05 2024-09-14T06:17:00.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.498 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181044+0000", 2024-09-14T06:17:00.498 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4405000000000001e-05 2024-09-14T06:17:00.499 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.499 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.499 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.499 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13234 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.499 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:00.499 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.36927936, 2024-09-14T06:17:00.499 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082015597999999995, 2024-09-14T06:17:00.499 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.499 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.499 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.499 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.499 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.499 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.500 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13234 2024-09-14T06:17:00.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.500 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.500 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.500 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:00.500 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.500 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.500 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:00.500 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.500 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.501 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:00.501 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0999999999999998e-07 2024-09-14T06:17:00.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.501 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.501 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099166+0000", 2024-09-14T06:17:00.501 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:17:00.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.501 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.501 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099171+0000", 2024-09-14T06:17:00.501 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6949999999999997e-06 2024-09-14T06:17:00.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099175+0000", 2024-09-14T06:17:00.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9029999999999997e-06 2024-09-14T06:17:00.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164063+0000", 2024-09-14T06:17:00.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064887608999999999 2024-09-14T06:17:00.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164111+0000", 2024-09-14T06:17:00.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7648999999999997e-05 2024-09-14T06:17:00.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.503 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.503 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164351+0000", 2024-09-14T06:17:00.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024072899999999999 2024-09-14T06:17:00.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.503 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.503 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167561+0000", 2024-09-14T06:17:00.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032094179999999999 2024-09-14T06:17:00.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.503 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.503 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181093+0000", 2024-09-14T06:17:00.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013532245 2024-09-14T06:17:00.504 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.504 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.504 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.504 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181139+0000", 2024-09-14T06:17:00.504 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6483999999999997e-05 2024-09-14T06:17:00.504 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.504 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.504 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.504 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181177+0000", 2024-09-14T06:17:00.504 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7861000000000002e-05 2024-09-14T06:17:00.504 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.504 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.504 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.504 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.504 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.505 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13235 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.505 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:00.505 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.369172085999999, 2024-09-14T06:17:00.505 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081988088000000001, 2024-09-14T06:17:00.505 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.505 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.505 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.505 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.505 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.505 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.505 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13235 2024-09-14T06:17:00.505 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.505 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.506 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.506 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.506 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:00.506 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.506 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.506 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.506 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.506 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:00.506 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.506 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.506 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.506 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.506 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099270+0000", 2024-09-14T06:17:00.506 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7400000000000003e-07 2024-09-14T06:17:00.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.507 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.507 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.507 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099276+0000", 2024-09-14T06:17:00.507 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:17:00.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.507 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.507 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.507 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099284+0000", 2024-09-14T06:17:00.507 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7979999999999995e-06 2024-09-14T06:17:00.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.507 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.507 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.507 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099289+0000", 2024-09-14T06:17:00.507 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:17:00.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164664+0000", 2024-09-14T06:17:00.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065374901999999999 2024-09-14T06:17:00.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164755+0000", 2024-09-14T06:17:00.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0790000000000003e-05 2024-09-14T06:17:00.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.509 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165143+0000", 2024-09-14T06:17:00.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000387904 2024-09-14T06:17:00.509 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.509 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.509 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167564+0000", 2024-09-14T06:17:00.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024208989999999998 2024-09-14T06:17:00.509 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.509 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.509 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181212+0000", 2024-09-14T06:17:00.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013648664 2024-09-14T06:17:00.509 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.510 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181237+0000", 2024-09-14T06:17:00.510 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5177999999999998e-05 2024-09-14T06:17:00.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.510 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181257+0000", 2024-09-14T06:17:00.510 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9660000000000002e-05 2024-09-14T06:17:00.510 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.510 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.510 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.510 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13236 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.511 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:00.511 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.369065267, 2024-09-14T06:17:00.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082010939000000005, 2024-09-14T06:17:00.511 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.511 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.511 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.511 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.511 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.511 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.511 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13236 2024-09-14T06:17:00.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.511 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.511 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.512 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:00.512 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.512 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.512 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:00.512 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.512 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.512 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:00.512 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.61e-07 2024-09-14T06:17:00.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.513 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.513 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099382+0000", 2024-09-14T06:17:00.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3589999999999999e-06 2024-09-14T06:17:00.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.513 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.513 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099388+0000", 2024-09-14T06:17:00.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9619999999999998e-06 2024-09-14T06:17:00.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.513 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.513 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099393+0000", 2024-09-14T06:17:00.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3410000000000002e-06 2024-09-14T06:17:00.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.514 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.514 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165553+0000", 2024-09-14T06:17:00.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066159604999999996 2024-09-14T06:17:00.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.514 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.514 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165603+0000", 2024-09-14T06:17:00.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0427999999999999e-05 2024-09-14T06:17:00.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.514 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165827+0000", 2024-09-14T06:17:00.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022358299999999999 2024-09-14T06:17:00.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177005+0000", 2024-09-14T06:17:00.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011178044 2024-09-14T06:17:00.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181302+0000", 2024-09-14T06:17:00.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042970669999999999 2024-09-14T06:17:00.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.516 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.516 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181349+0000", 2024-09-14T06:17:00.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7287999999999999e-05 2024-09-14T06:17:00.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.516 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.516 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181387+0000", 2024-09-14T06:17:00.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7601000000000001e-05 2024-09-14T06:17:00.516 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.516 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.516 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.516 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13237 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.517 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:00.517 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.368907119999999, 2024-09-14T06:17:00.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081927299999999995, 2024-09-14T06:17:00.517 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.517 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.517 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.517 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.517 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.517 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.517 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13237 2024-09-14T06:17:00.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.517 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.517 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.517 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.518 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:00.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.518 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.518 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:00.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.518 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.518 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099535+0000", 2024-09-14T06:17:00.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:17:00.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099538+0000", 2024-09-14T06:17:00.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:17:00.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099545+0000", 2024-09-14T06:17:00.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:17:00.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099551+0000", 2024-09-14T06:17:00.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1770000000000003e-06 2024-09-14T06:17:00.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.520 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.520 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.520 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166131+0000", 2024-09-14T06:17:00.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066580499000000001 2024-09-14T06:17:00.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.520 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.520 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.520 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166221+0000", 2024-09-14T06:17:00.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0000000000000006e-05 2024-09-14T06:17:00.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.520 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.520 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166622+0000", 2024-09-14T06:17:00.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040029799999999999 2024-09-14T06:17:00.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177008+0000", 2024-09-14T06:17:00.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010386772000000001 2024-09-14T06:17:00.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181420+0000", 2024-09-14T06:17:00.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044119190000000003 2024-09-14T06:17:00.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181443+0000", 2024-09-14T06:17:00.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2603999999999999e-05 2024-09-14T06:17:00.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181461+0000", 2024-09-14T06:17:00.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8405000000000001e-05 2024-09-14T06:17:00.522 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.522 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.522 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.523 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13238 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.523 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:00.523 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.368870812000001, 2024-09-14T06:17:00.523 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082014927000000001, 2024-09-14T06:17:00.523 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.523 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.523 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.523 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.523 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.523 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.524 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13238 2024-09-14T06:17:00.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.524 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.524 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.524 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:00.524 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.524 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.524 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:00.524 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099571+0000", 2024-09-14T06:17:00.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9699999999999996e-07 2024-09-14T06:17:00.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099577+0000", 2024-09-14T06:17:00.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2890000000000003e-06 2024-09-14T06:17:00.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099587+0000", 2024-09-14T06:17:00.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0002e-05 2024-09-14T06:17:00.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099593+0000", 2024-09-14T06:17:00.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4539999999999998e-06 2024-09-14T06:17:00.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167024+0000", 2024-09-14T06:17:00.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067431480000000002 2024-09-14T06:17:00.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.527 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167074+0000", 2024-09-14T06:17:00.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9996e-05 2024-09-14T06:17:00.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.527 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167288+0000", 2024-09-14T06:17:00.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021410900000000001 2024-09-14T06:17:00.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.527 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177997+0000", 2024-09-14T06:17:00.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010708756 2024-09-14T06:17:00.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181508+0000", 2024-09-14T06:17:00.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003511363 2024-09-14T06:17:00.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181552+0000", 2024-09-14T06:17:00.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3609999999999998e-05 2024-09-14T06:17:00.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.529 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.529 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181585+0000", 2024-09-14T06:17:00.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3271e-05 2024-09-14T06:17:00.529 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.529 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.529 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.529 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13239 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.529 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:00.529 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.368778993999999, 2024-09-14T06:17:00.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081998772999999997, 2024-09-14T06:17:00.529 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.530 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.530 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.530 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.530 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.530 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.530 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13239 2024-09-14T06:17:00.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.530 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.530 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.530 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.530 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:00.530 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:00.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099663+0000", 2024-09-14T06:17:00.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6100000000000001e-07 2024-09-14T06:17:00.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099666+0000", 2024-09-14T06:17:00.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.056e-06 2024-09-14T06:17:00.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099672+0000", 2024-09-14T06:17:00.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8120000000000002e-06 2024-09-14T06:17:00.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099676+0000", 2024-09-14T06:17:00.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:17:00.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167602+0000", 2024-09-14T06:17:00.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067925867000000001 2024-09-14T06:17:00.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.533 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.533 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.533 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167693+0000", 2024-09-14T06:17:00.533 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0736000000000007e-05 2024-09-14T06:17:00.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.533 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.533 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.533 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168065+0000", 2024-09-14T06:17:00.533 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037245399999999998 2024-09-14T06:17:00.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.533 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.533 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.533 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178003+0000", 2024-09-14T06:17:00.534 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0099372360000000003 2024-09-14T06:17:00.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.534 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.534 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181621+0000", 2024-09-14T06:17:00.534 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036178489999999998 2024-09-14T06:17:00.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.534 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.534 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181643+0000", 2024-09-14T06:17:00.534 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2155e-05 2024-09-14T06:17:00.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.534 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.535 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181661+0000", 2024-09-14T06:17:00.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8147000000000001e-05 2024-09-14T06:17:00.535 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.535 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.535 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.535 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13240 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.535 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:00.535 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.368659901000001, 2024-09-14T06:17:00.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081942607000000001, 2024-09-14T06:17:00.535 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.535 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.535 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.536 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.536 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.536 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.536 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13240 2024-09-14T06:17:00.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.536 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:00.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:00.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099782+0000", 2024-09-14T06:17:00.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3900000000000004e-07 2024-09-14T06:17:00.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099787+0000", 2024-09-14T06:17:00.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6189999999999999e-06 2024-09-14T06:17:00.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.538 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099793+0000", 2024-09-14T06:17:00.538 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.998e-06 2024-09-14T06:17:00.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.538 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.538 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099797+0000", 2024-09-14T06:17:00.538 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8449999999999996e-06 2024-09-14T06:17:00.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.538 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.538 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168494+0000", 2024-09-14T06:17:00.538 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.068696500999999993 2024-09-14T06:17:00.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.539 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.539 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168590+0000", 2024-09-14T06:17:00.539 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6733999999999999e-05 2024-09-14T06:17:00.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.539 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.539 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.539 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168973+0000", 2024-09-14T06:17:00.539 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038229399999999998 2024-09-14T06:17:00.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.539 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.539 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.539 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178013+0000", 2024-09-14T06:17:00.539 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0090397199999999994 2024-09-14T06:17:00.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.539 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181683+0000", 2024-09-14T06:17:00.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036705819999999999 2024-09-14T06:17:00.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181705+0000", 2024-09-14T06:17:00.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1551000000000001e-05 2024-09-14T06:17:00.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181724+0000", 2024-09-14T06:17:00.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9123999999999999e-05 2024-09-14T06:17:00.540 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.541 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.541 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.541 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.541 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13243 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.541 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:00.541 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.368126171, 2024-09-14T06:17:00.541 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081767815999999993, 2024-09-14T06:17:00.541 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.541 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.541 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.541 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.541 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.541 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.542 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13243 2024-09-14T06:17:00.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.542 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.542 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.542 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:00.542 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.542 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.542 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:00.542 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:00.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1e-07 2024-09-14T06:17:00.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100317+0000", 2024-09-14T06:17:00.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.092e-06 2024-09-14T06:17:00.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100321+0000", 2024-09-14T06:17:00.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7809999999999999e-06 2024-09-14T06:17:00.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.544 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.544 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100324+0000", 2024-09-14T06:17:00.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.117e-06 2024-09-14T06:17:00.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.544 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.544 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171113+0000", 2024-09-14T06:17:00.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.070788816000000004 2024-09-14T06:17:00.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.544 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.544 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171202+0000", 2024-09-14T06:17:00.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8843000000000003e-05 2024-09-14T06:17:00.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.545 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.545 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171584+0000", 2024-09-14T06:17:00.545 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038231099999999998 2024-09-14T06:17:00.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.545 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.545 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178101+0000", 2024-09-14T06:17:00.545 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0065168329999999997 2024-09-14T06:17:00.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.545 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.546 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182041+0000", 2024-09-14T06:17:00.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039401810000000001 2024-09-14T06:17:00.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.546 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.546 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182065+0000", 2024-09-14T06:17:00.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3623000000000001e-05 2024-09-14T06:17:00.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.546 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.546 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182083+0000", 2024-09-14T06:17:00.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7909000000000001e-05 2024-09-14T06:17:00.546 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.546 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.547 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.547 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.547 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.547 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13244 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.547 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:00.547 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.368111351, 2024-09-14T06:17:00.547 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081837800000000002, 2024-09-14T06:17:00.547 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.547 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.547 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.547 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.547 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.547 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.547 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13244 2024-09-14T06:17:00.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.548 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.548 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.548 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.548 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:00.548 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.548 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.548 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.548 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:00.548 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.548 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.548 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:00.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:17:00.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100337+0000", 2024-09-14T06:17:00.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.939e-06 2024-09-14T06:17:00.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100341+0000", 2024-09-14T06:17:00.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2320000000000003e-06 2024-09-14T06:17:00.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100344+0000", 2024-09-14T06:17:00.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0759999999999999e-06 2024-09-14T06:17:00.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171996+0000", 2024-09-14T06:17:00.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.071651919999999994 2024-09-14T06:17:00.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172079+0000", 2024-09-14T06:17:00.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2579999999999999e-05 2024-09-14T06:17:00.551 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.551 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.551 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.551 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172457+0000", 2024-09-14T06:17:00.551 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037785100000000001 2024-09-14T06:17:00.551 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.551 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.551 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.551 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178104+0000", 2024-09-14T06:17:00.551 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056471730000000001 2024-09-14T06:17:00.551 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.551 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.551 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.551 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182123+0000", 2024-09-14T06:17:00.552 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040190019999999998 2024-09-14T06:17:00.552 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.552 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.552 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.552 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182147+0000", 2024-09-14T06:17:00.552 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4241000000000001e-05 2024-09-14T06:17:00.552 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.552 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.552 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.552 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182168+0000", 2024-09-14T06:17:00.552 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0492999999999998e-05 2024-09-14T06:17:00.552 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.552 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.553 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.553 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.553 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.553 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13245 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.553 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:00.553 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.368091843999999, 2024-09-14T06:17:00.553 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081893899000000006, 2024-09-14T06:17:00.553 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.553 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.553 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.553 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.553 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.553 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.554 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13245 2024-09-14T06:17:00.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.554 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:00.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:00.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.555 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.555 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100350+0000", 2024-09-14T06:17:00.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5499999999999999e-07 2024-09-14T06:17:00.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.555 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.555 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.555 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100353+0000", 2024-09-14T06:17:00.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:17:00.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.555 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.555 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.555 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100357+0000", 2024-09-14T06:17:00.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8500000000000004e-06 2024-09-14T06:17:00.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.556 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.556 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100360+0000", 2024-09-14T06:17:00.556 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8700000000000001e-06 2024-09-14T06:17:00.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.556 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.556 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172862+0000", 2024-09-14T06:17:00.556 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.072501992000000001 2024-09-14T06:17:00.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.556 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.557 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172950+0000", 2024-09-14T06:17:00.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8988999999999998e-05 2024-09-14T06:17:00.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.557 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.557 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.557 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173324+0000", 2024-09-14T06:17:00.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037366199999999997 2024-09-14T06:17:00.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.557 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.557 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.557 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178107+0000", 2024-09-14T06:17:00.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047830520000000003 2024-09-14T06:17:00.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.557 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.558 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.558 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182202+0000", 2024-09-14T06:17:00.558 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040949280000000003 2024-09-14T06:17:00.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.558 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.558 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182226+0000", 2024-09-14T06:17:00.558 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3535000000000001e-05 2024-09-14T06:17:00.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.558 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.558 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182243+0000", 2024-09-14T06:17:00.558 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7445e-05 2024-09-14T06:17:00.558 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.558 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.559 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.559 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.559 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13246 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.559 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:00.559 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.368077522, 2024-09-14T06:17:00.559 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081962276000000001, 2024-09-14T06:17:00.559 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.559 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.559 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.559 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.559 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.559 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.560 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13246 2024-09-14T06:17:00.560 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.560 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.560 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.560 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:00.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.560 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.560 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.560 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:00.561 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.561 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.561 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:00.561 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0800000000000001e-07 2024-09-14T06:17:00.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.561 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.561 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100367+0000", 2024-09-14T06:17:00.561 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8439999999999998e-06 2024-09-14T06:17:00.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.561 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.562 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100372+0000", 2024-09-14T06:17:00.562 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4009999999999996e-06 2024-09-14T06:17:00.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.562 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.562 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100378+0000", 2024-09-14T06:17:00.562 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7180000000000001e-06 2024-09-14T06:17:00.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.562 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.562 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173780+0000", 2024-09-14T06:17:00.562 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.073401991 2024-09-14T06:17:00.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.563 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173879+0000", 2024-09-14T06:17:00.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9449000000000003e-05 2024-09-14T06:17:00.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.563 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.563 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174240+0000", 2024-09-14T06:17:00.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036118100000000002 2024-09-14T06:17:00.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.563 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.563 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178109+0000", 2024-09-14T06:17:00.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038690619999999999 2024-09-14T06:17:00.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.564 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.564 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182281+0000", 2024-09-14T06:17:00.564 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041715900000000002 2024-09-14T06:17:00.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.564 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.564 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182306+0000", 2024-09-14T06:17:00.564 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4477e-05 2024-09-14T06:17:00.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.564 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.564 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182326+0000", 2024-09-14T06:17:00.564 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0254999999999998e-05 2024-09-14T06:17:00.565 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.565 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.565 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.565 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.565 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.565 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13247 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.565 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:00.565 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.367980344999999, 2024-09-14T06:17:00.565 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966593000000004, 2024-09-14T06:17:00.565 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.565 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.565 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.565 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.566 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.566 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.566 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13247 2024-09-14T06:17:00.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.566 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.566 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.566 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:00.566 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.566 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.566 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:00.566 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100462+0000", 2024-09-14T06:17:00.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9800000000000002e-07 2024-09-14T06:17:00.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100466+0000", 2024-09-14T06:17:00.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4039999999999997e-06 2024-09-14T06:17:00.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.568 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100475+0000", 2024-09-14T06:17:00.568 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3680000000000008e-06 2024-09-14T06:17:00.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.568 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.568 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.568 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100482+0000", 2024-09-14T06:17:00.568 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8630000000000002e-06 2024-09-14T06:17:00.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.568 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.568 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.568 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174669+0000", 2024-09-14T06:17:00.568 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074186849999999999 2024-09-14T06:17:00.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.569 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.569 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174758+0000", 2024-09-14T06:17:00.569 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9021e-05 2024-09-14T06:17:00.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.569 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.569 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175137+0000", 2024-09-14T06:17:00.569 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378511 2024-09-14T06:17:00.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.569 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.569 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178158+0000", 2024-09-14T06:17:00.569 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003020956 2024-09-14T06:17:00.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.570 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.570 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182386+0000", 2024-09-14T06:17:00.570 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042278840000000003 2024-09-14T06:17:00.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.570 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.570 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182410+0000", 2024-09-14T06:17:00.570 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4026e-05 2024-09-14T06:17:00.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.570 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.570 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182427+0000", 2024-09-14T06:17:00.571 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7711999999999999e-05 2024-09-14T06:17:00.571 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.571 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.571 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.571 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.571 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13248 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.571 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:00.571 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.367898264000001, 2024-09-14T06:17:00.571 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966527999999997, 2024-09-14T06:17:00.571 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.571 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.571 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.571 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.572 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.572 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.572 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13248 2024-09-14T06:17:00.572 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.572 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.572 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.572 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:00.572 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.572 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.572 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.572 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:00.573 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.573 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.573 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.573 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:00.573 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:17:00.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.573 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.573 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.573 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100548+0000", 2024-09-14T06:17:00.573 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:17:00.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.573 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.573 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100557+0000", 2024-09-14T06:17:00.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:17:00.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.574 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100563+0000", 2024-09-14T06:17:00.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2519999999999996e-06 2024-09-14T06:17:00.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.574 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175572+0000", 2024-09-14T06:17:00.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075008459 2024-09-14T06:17:00.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175656+0000", 2024-09-14T06:17:00.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3906999999999995e-05 2024-09-14T06:17:00.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176010+0000", 2024-09-14T06:17:00.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00035447499999999998 2024-09-14T06:17:00.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178304+0000", 2024-09-14T06:17:00.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0022941379999999998 2024-09-14T06:17:00.576 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.576 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.576 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.576 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182467+0000", 2024-09-14T06:17:00.576 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041632020000000004 2024-09-14T06:17:00.576 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.576 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.576 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.576 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182491+0000", 2024-09-14T06:17:00.576 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3733000000000001e-05 2024-09-14T06:17:00.576 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.576 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.576 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.577 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182509+0000", 2024-09-14T06:17:00.577 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8113000000000001e-05 2024-09-14T06:17:00.577 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.577 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.577 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.577 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.577 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13249 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.577 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:00.577 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.367863329999999, 2024-09-14T06:17:00.577 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082037225000000005, 2024-09-14T06:17:00.577 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.577 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.577 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.578 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.578 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.578 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.578 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13249 2024-09-14T06:17:00.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.578 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.578 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.578 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.578 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:00.578 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.578 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.578 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.579 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:00.579 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.579 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.579 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.579 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.579 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:00.579 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4600000000000005e-07 2024-09-14T06:17:00.579 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.579 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.579 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.579 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100584+0000", 2024-09-14T06:17:00.579 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.947e-06 2024-09-14T06:17:00.579 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.579 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.580 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.580 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100593+0000", 2024-09-14T06:17:00.580 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9760000000000001e-06 2024-09-14T06:17:00.580 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.580 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.580 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.580 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100597+0000", 2024-09-14T06:17:00.580 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9689999999999996e-06 2024-09-14T06:17:00.580 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.580 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.580 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.580 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176461+0000", 2024-09-14T06:17:00.580 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075864132000000001 2024-09-14T06:17:00.580 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.581 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.581 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176516+0000", 2024-09-14T06:17:00.581 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4416999999999998e-05 2024-09-14T06:17:00.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.581 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.581 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176734+0000", 2024-09-14T06:17:00.581 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021859399999999999 2024-09-14T06:17:00.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.581 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.581 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178308+0000", 2024-09-14T06:17:00.582 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015733660000000001 2024-09-14T06:17:00.582 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.582 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.582 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.582 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182573+0000", 2024-09-14T06:17:00.582 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042649200000000002 2024-09-14T06:17:00.582 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.582 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.582 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.582 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182597+0000", 2024-09-14T06:17:00.582 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4355e-05 2024-09-14T06:17:00.582 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.582 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.582 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.583 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182615+0000", 2024-09-14T06:17:00.583 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8003e-05 2024-09-14T06:17:00.583 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.583 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.583 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.583 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.583 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.583 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13250 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:00.583 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:00.583 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.367698725, 2024-09-14T06:17:00.583 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081983890000000004, 2024-09-14T06:17:00.583 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:00.583 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:00.583 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:00.584 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:00.584 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:00.584 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:00.584 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13250 2024-09-14T06:17:00.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.584 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:00.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.584 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:00.584 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:00.584 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.584 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:00.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:00.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:00.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.585 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:00.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100743+0000", 2024-09-14T06:17:00.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:17:00.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.585 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:00.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100753+0000", 2024-09-14T06:17:00.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9979999999999998e-06 2024-09-14T06:17:00.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.586 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:00.586 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100772+0000", 2024-09-14T06:17:00.586 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8664e-05 2024-09-14T06:17:00.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.586 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.586 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:00.586 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100779+0000", 2024-09-14T06:17:00.586 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.046e-06 2024-09-14T06:17:00.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.586 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.586 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:00.586 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177049+0000", 2024-09-14T06:17:00.586 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076270819000000004 2024-09-14T06:17:00.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.587 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.587 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:00.587 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177140+0000", 2024-09-14T06:17:00.587 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0746000000000002e-05 2024-09-14T06:17:00.587 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.587 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.587 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:00.587 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177519+0000", 2024-09-14T06:17:00.587 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378656 2024-09-14T06:17:00.587 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.587 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.587 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:00.587 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178349+0000", 2024-09-14T06:17:00.587 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00083061599999999995 2024-09-14T06:17:00.588 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.588 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.588 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:00.588 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182679+0000", 2024-09-14T06:17:00.588 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043298379999999999 2024-09-14T06:17:00.588 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.588 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.588 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:00.588 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182705+0000", 2024-09-14T06:17:00.588 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5602999999999999e-05 2024-09-14T06:17:00.589 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:00.589 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:00.589 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:00.589 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182726+0000", 2024-09-14T06:17:00.589 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1297e-05 2024-09-14T06:17:00.589 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.589 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.589 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.589 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:00.589 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:00.589 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:00.591 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:17:00.592 INFO:teuthology.orchestra.run.smithi039.stdout:3034 write 0xe543448 thru 0xe550b35 (0xd6ee bytes) 2024-09-14T06:17:00.592 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:00.592 INFO:teuthology.orchestra.run.smithi039.stdout:3038 writesame 0xb561b3c thru 0xb5676bc data_size 0x5(0x5b81 bytes) 2024-09-14T06:17:00.592 INFO:teuthology.orchestra.run.smithi039.stdout:3039 trunc from 0xe550b36 to 0xdc0718c 2024-09-14T06:17:00.592 INFO:teuthology.orchestra.run.smithi039.stdout:3040 write 0x9c3d283 thru 0x9c4a1d0 (0xcf4e bytes) 2024-09-14T06:17:00.592 INFO:teuthology.orchestra.run.smithi039.stdout:3041 writesame 0x5238f3d thru 0x5248e1c data_size 0x10(0xfee0 bytes) 2024-09-14T06:17:00.592 INFO:teuthology.orchestra.run.smithi039.stdout:3042 write 0x5bf71b3 thru 0x5bfb189 (0x3fd7 bytes) 2024-09-14T06:17:00.593 INFO:teuthology.orchestra.run.smithi039.stdout:3043 read 0x5f9967 thru 0x5fafbd (0x1657 bytes) 2024-09-14T06:17:00.593 INFO:teuthology.orchestra.run.smithi039.stdout:3044 write 0xcba0e13 thru 0xcba575b (0x4949 bytes) 2024-09-14T06:17:00.593 INFO:teuthology.orchestra.run.smithi039.stdout:3045 writesame 0x17bb825 thru 0x17c16db data_size 0x1(0x5eb7 bytes) 2024-09-14T06:17:00.593 INFO:teuthology.orchestra.run.smithi039.stdout:3046 trunc from 0xdc0718c to 0x1c190cc 2024-09-14T06:17:00.593 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:00.587+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:00.648 INFO:teuthology.orchestra.run.smithi039.stdout:3047 write 0x42361d0 thru 0x423e982 (0x87b3 bytes) 2024-09-14T06:17:00.655 INFO:teuthology.orchestra.run.smithi039.stdout:3048 read 0x341e76c thru 0x342487a (0x610f bytes) 2024-09-14T06:17:00.655 INFO:teuthology.orchestra.run.smithi039.stdout:3049 read 0x31504c7 thru 0x315fe6b (0xf9a5 bytes) 2024-09-14T06:17:00.676 INFO:teuthology.orchestra.run.smithi039.stdout:3050 write 0x7b2b2f9 thru 0x7b391b0 (0xdeb8 bytes) 2024-09-14T06:17:00.682 INFO:teuthology.orchestra.run.smithi039.stdout:3051 writesame 0x6afe18d thru 0x6b01638 data_size 0x2(0x34ac bytes) 2024-09-14T06:17:00.689 INFO:teuthology.orchestra.run.smithi039.stdout:3053 write 0x30b2230 thru 0x30c1a4a (0xf81b bytes) 2024-09-14T06:17:00.691 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:00.691+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:00.695 INFO:teuthology.orchestra.run.smithi039.stdout:3054 write 0x612dfa8 thru 0x612eaa8 (0xb01 bytes) 2024-09-14T06:17:00.698 INFO:teuthology.orchestra.run.smithi039.stdout:3055 punch from 0x7adb93f to 0x7ade446, (0x2b07 bytes) 2024-09-14T06:17:00.698 INFO:teuthology.orchestra.run.smithi039.stdout:3056 write 0x5d6438 thru 0x5e15e6 (0xb1af bytes) 2024-09-14T06:17:00.701 INFO:teuthology.orchestra.run.smithi039.stdout:3057 read 0x2d6eddc thru 0x2d70aa1 (0x1cc6 bytes) 2024-09-14T06:17:00.716 INFO:teuthology.orchestra.run.smithi039.stdout:3058 write 0x889f4d4 thru 0x889f640 (0x16d bytes) 2024-09-14T06:17:00.722 INFO:teuthology.orchestra.run.smithi039.stdout:3059 read 0x1b0baf5 thru 0x1b1b5af (0xfabb bytes) 2024-09-14T06:17:00.725 INFO:teuthology.orchestra.run.smithi039.stdout:3060 trunc from 0x889f641 to 0x7f75268 2024-09-14T06:17:00.743 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:00.744 INFO:teuthology.orchestra.run.smithi039.stdout:3063 writesame 0x1ec336a thru 0x1ecd9dc data_size 0x1(0xa673 bytes) 2024-09-14T06:17:00.769 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:00.770 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:00.770 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:00.770 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:00.778 INFO:teuthology.orchestra.run.smithi039.stdout:3064 write 0x1d0807f thru 0x1d0b9c6 (0x3948 bytes) 2024-09-14T06:17:00.781 INFO:teuthology.orchestra.run.smithi039.stdout:3068 read 0x74f2239 thru 0x74f90e8 (0x6eb0 bytes) 2024-09-14T06:17:00.782 INFO:teuthology.orchestra.run.smithi039.stdout:3069 punch from 0x5201f03 to 0x520a64e, (0x874b bytes) 2024-09-14T06:17:00.782 INFO:teuthology.orchestra.run.smithi039.stdout:3071 punch from 0x44affb to 0x450265, (0x526a bytes) 2024-09-14T06:17:00.782 INFO:teuthology.orchestra.run.smithi039.stdout:3072 write 0xb09b55 thru 0xb0f584 (0x5a30 bytes) 2024-09-14T06:17:00.785 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:17:00.786 INFO:teuthology.orchestra.run.smithi039.stdout:3073 read 0xfa1ef2 thru 0xfa3be4 (0x1cf3 bytes) 2024-09-14T06:17:00.786 INFO:teuthology.orchestra.run.smithi039.stdout:3074 read 0x3df2284 thru 0x3df5710 (0x348d bytes) 2024-09-14T06:17:00.811 INFO:teuthology.orchestra.run.smithi039.stdout:3075 write 0xc98e823 thru 0xc996f27 (0x8705 bytes) 2024-09-14T06:17:00.814 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:00.815 INFO:teuthology.orchestra.run.smithi039.stdout:3077 read 0x6970acc thru 0x69722cb (0x1800 bytes) 2024-09-14T06:17:00.815 INFO:teuthology.orchestra.run.smithi039.stdout:3081 read 0x5b3f3bc thru 0x5b3fab6 (0x6fb bytes) 2024-09-14T06:17:00.816 INFO:teuthology.orchestra.run.smithi039.stdout:3082 write 0x28a3893 thru 0x28af960 (0xc0ce bytes) 2024-09-14T06:17:00.819 INFO:teuthology.orchestra.run.smithi039.stdout:3083 punch from 0xc3bc475 to 0xc3c1b70, (0x56fb bytes) 2024-09-14T06:17:00.820 INFO:teuthology.orchestra.run.smithi039.stdout:3087 read 0x432a39e thru 0x4333dbc (0x9a1f bytes) 2024-09-14T06:17:00.820 INFO:teuthology.orchestra.run.smithi039.stdout:3088 read 0x32979a3 thru 0x32a6c06 (0xf264 bytes) 2024-09-14T06:17:00.821 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:00.822 INFO:teuthology.orchestra.run.smithi039.stdout:3092 write 0x68a715a thru 0x68a9f35 (0x2ddc bytes) 2024-09-14T06:17:00.824 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:00.824 INFO:teuthology.orchestra.run.smithi039.stdout:3094 punch from 0x2425206 to 0x2428336, (0x3130 bytes) 2024-09-14T06:17:00.834 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:00.834+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:00.842 INFO:teuthology.orchestra.run.smithi039.stdout:3095 trunc from 0xc996f28 to 0x5e5aafb 2024-09-14T06:17:00.858 INFO:teuthology.orchestra.run.smithi039.stdout:3096 read 0x1e1c5dc thru 0x1e20b50 (0x4575 bytes) 2024-09-14T06:17:00.859 INFO:teuthology.orchestra.run.smithi039.stdout:3097 punch from 0x41ec21c to 0x41faa41, (0xe825 bytes) 2024-09-14T06:17:00.868 INFO:teuthology.orchestra.run.smithi039.stdout:3098 trunc from 0x5e5aafb to 0x28359a6 2024-09-14T06:17:00.893 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:00.890+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:00.894 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:00.894 INFO:teuthology.orchestra.run.smithi039.stdout:3100 punch from 0x265b992 to 0x2663a3d, (0x80ab bytes) 2024-09-14T06:17:00.911 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:00.911 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:00.912 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:00.912 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:00.912 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:00.921 INFO:teuthology.orchestra.run.smithi039.stdout:3101 write 0x7a6a834 thru 0x7a6fd75 (0x5542 bytes) 2024-09-14T06:17:00.926 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:17:00.932 INFO:teuthology.orchestra.run.smithi039.stdout:3103 trunc from 0x7a6fd76 to 0xa0ecbf2 2024-09-14T06:17:00.958 INFO:teuthology.orchestra.run.smithi039.stdout:3104 writesame 0xc3c80ad thru 0xc3cbff8 data_size 0x2(0x3f4c bytes) 2024-09-14T06:17:00.959 INFO:teuthology.orchestra.run.smithi039.stdout:3105 punch from 0x87a9b07 to 0x87af987, (0x5e80 bytes) 2024-09-14T06:17:00.984 INFO:teuthology.orchestra.run.smithi039.stdout:3106 trunc from 0xc3cbff9 to 0x2913021 2024-09-14T06:17:00.998 INFO:teuthology.orchestra.run.smithi039.stdout:3108 read 0x20349ea thru 0x2035665 (0xc7c bytes) 2024-09-14T06:17:00.998 INFO:teuthology.orchestra.run.smithi039.stdout:3109 write 0x78907f thru 0x7894d4 (0x456 bytes) 2024-09-14T06:17:01.009 INFO:teuthology.orchestra.run.smithi039.stdout:3110 trunc from 0x2913021 to 0x6014269 2024-09-14T06:17:01.011 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:01.010+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:01.020 INFO:teuthology.orchestra.run.smithi039.stdout:3111 read 0x54999d2 thru 0x549da66 (0x4095 bytes) 2024-09-14T06:17:01.021 INFO:teuthology.orchestra.run.smithi039.stdout:3112 read 0x1dc0f98 thru 0x1dc2e0d (0x1e76 bytes) 2024-09-14T06:17:01.035 INFO:teuthology.orchestra.run.smithi039.stdout:3113 writesame 0x72d871e thru 0x72dace6 data_size 0x1(0x25c9 bytes) 2024-09-14T06:17:01.035 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:01.036 INFO:teuthology.orchestra.run.smithi039.stdout:3117 read 0x1ffc35a thru 0x2001f46 (0x5bed bytes) 2024-09-14T06:17:01.036 INFO:teuthology.orchestra.run.smithi039.stdout:3118 writesame 0x48bd5e6 thru 0x48bf24d data_size 0x24(0x1c68 bytes) 2024-09-14T06:17:01.040 INFO:teuthology.orchestra.run.smithi039.stdout:3119 punch from 0x70face3 to 0x70fc453, (0x1770 bytes) 2024-09-14T06:17:01.040 INFO:teuthology.orchestra.run.smithi039.stdout:3120 punch from 0x2718746 to 0x2722c3e, (0xa4f8 bytes) 2024-09-14T06:17:01.059 INFO:teuthology.orchestra.run.smithi039.stdout:3121 write 0xac575f0 thru 0xac5f08f (0x7aa0 bytes) 2024-09-14T06:17:01.062 INFO:teuthology.orchestra.run.smithi039.stdout:3122 read 0x1f16fdb thru 0x1f1d6aa (0x66d0 bytes) 2024-09-14T06:17:01.063 INFO:teuthology.orchestra.run.smithi039.stdout:3123 write 0x5206ff9 thru 0x5207cca (0xcd2 bytes) 2024-09-14T06:17:01.066 INFO:teuthology.orchestra.run.smithi039.stdout:3124 punch from 0x8a51cc5 to 0x8a5f963, (0xdc9e bytes) 2024-09-14T06:17:01.066 INFO:teuthology.orchestra.run.smithi039.stdout:3125 writesame 0x16a7813 thru 0x16b77e8 data_size 0x8f(0xffd6 bytes) 2024-09-14T06:17:01.081 INFO:teuthology.orchestra.run.smithi039.stdout:3126 trunc from 0xac5f090 to 0x6ece1bc 2024-09-14T06:17:01.094 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:01.094+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:01.097 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:01.097 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:01.097 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:01.097 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:01.097 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.097 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16117 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.097 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654259+0000", 2024-09-14T06:17:01.097 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.442512481000001, 2024-09-14T06:17:01.097 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.034316067999999998, 2024-09-14T06:17:01.097 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.097 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.098 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.098 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.098 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.098 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.098 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16117 2024-09-14T06:17:01.098 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.098 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.098 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.098 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.098 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654259+0000", 2024-09-14T06:17:01.098 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.098 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.098 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.099 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.099 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654259+0000", 2024-09-14T06:17:01.099 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.099 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.099 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.099 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.099 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654260+0000", 2024-09-14T06:17:01.099 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1699999999999997e-07 2024-09-14T06:17:01.099 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.099 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.099 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.099 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654263+0000", 2024-09-14T06:17:01.099 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6029999999999999e-06 2024-09-14T06:17:01.099 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.099 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.099 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.100 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654272+0000", 2024-09-14T06:17:01.100 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1600000000000004e-06 2024-09-14T06:17:01.100 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.100 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.100 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.100 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654279+0000", 2024-09-14T06:17:01.100 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9469999999999997e-06 2024-09-14T06:17:01.100 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.100 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.100 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.100 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688248+0000", 2024-09-14T06:17:01.100 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.033969223999999999 2024-09-14T06:17:01.100 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.100 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.101 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.101 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688387+0000", 2024-09-14T06:17:01.101 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013955200000000001 2024-09-14T06:17:01.101 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.101 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.101 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.101 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688575+0000", 2024-09-14T06:17:01.101 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000187865 2024-09-14T06:17:01.101 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.101 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.101 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.101 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.101 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.101 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16118 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.102 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654288+0000", 2024-09-14T06:17:01.102 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.442484095000001, 2024-09-14T06:17:01.102 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.034884971000000001, 2024-09-14T06:17:01.102 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.102 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.102 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.102 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.102 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.102 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.102 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16118 2024-09-14T06:17:01.102 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.102 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.102 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.102 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.103 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654288+0000", 2024-09-14T06:17:01.103 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.103 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.103 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.103 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654288+0000", 2024-09-14T06:17:01.103 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.103 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.103 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.103 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654288+0000", 2024-09-14T06:17:01.103 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1399999999999996e-07 2024-09-14T06:17:01.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.103 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.103 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.104 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654291+0000", 2024-09-14T06:17:01.104 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4700000000000001e-06 2024-09-14T06:17:01.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.104 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.104 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.104 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654300+0000", 2024-09-14T06:17:01.104 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:17:01.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.104 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.104 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.104 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654307+0000", 2024-09-14T06:17:01.104 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1430000000000001e-06 2024-09-14T06:17:01.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.104 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.105 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.105 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688682+0000", 2024-09-14T06:17:01.105 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.034375304000000002 2024-09-14T06:17:01.105 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.105 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.105 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.105 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.688897+0000", 2024-09-14T06:17:01.105 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021480499999999999 2024-09-14T06:17:01.105 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.105 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.105 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.105 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689173+0000", 2024-09-14T06:17:01.105 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00027551499999999999 2024-09-14T06:17:01.106 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.106 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.106 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.106 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.106 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.106 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16119 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.106 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654316+0000", 2024-09-14T06:17:01.106 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.442455632000001, 2024-09-14T06:17:01.106 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035292513999999997, 2024-09-14T06:17:01.106 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.106 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.106 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.106 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.107 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.107 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.107 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16119 2024-09-14T06:17:01.107 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.107 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.107 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.107 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.107 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654316+0000", 2024-09-14T06:17:01.107 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.108 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.108 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.108 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654316+0000", 2024-09-14T06:17:01.108 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.109 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.109 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.109 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654317+0000", 2024-09-14T06:17:01.109 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.37e-07 2024-09-14T06:17:01.109 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.109 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.110 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.110 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654319+0000", 2024-09-14T06:17:01.110 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6469999999999998e-06 2024-09-14T06:17:01.110 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.110 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.110 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.110 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654332+0000", 2024-09-14T06:17:01.111 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2330000000000001e-05 2024-09-14T06:17:01.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.111 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.111 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.111 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654339+0000", 2024-09-14T06:17:01.111 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2130000000000002e-06 2024-09-14T06:17:01.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.112 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.112 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.112 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689282+0000", 2024-09-14T06:17:01.112 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.034942742999999998 2024-09-14T06:17:01.112 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.112 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.112 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.112 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689424+0000", 2024-09-14T06:17:01.113 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014175900000000001 2024-09-14T06:17:01.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.113 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.113 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.113 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689609+0000", 2024-09-14T06:17:01.113 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00018508500000000001 2024-09-14T06:17:01.113 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.114 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.114 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.114 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16120 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.114 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654348+0000", 2024-09-14T06:17:01.115 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.442423385000001, 2024-09-14T06:17:01.115 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035856590000000001, 2024-09-14T06:17:01.115 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.115 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.115 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.115 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.115 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.116 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.116 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16120 2024-09-14T06:17:01.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.116 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.116 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.116 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.116 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654348+0000", 2024-09-14T06:17:01.116 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.116 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.116 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.116 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654348+0000", 2024-09-14T06:17:01.116 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.117 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.117 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.117 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654349+0000", 2024-09-14T06:17:01.117 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2699999999999999e-07 2024-09-14T06:17:01.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.117 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.117 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.117 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654352+0000", 2024-09-14T06:17:01.117 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6400000000000001e-06 2024-09-14T06:17:01.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.117 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.117 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.117 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654363+0000", 2024-09-14T06:17:01.118 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1715999999999999e-05 2024-09-14T06:17:01.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.118 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654371+0000", 2024-09-14T06:17:01.118 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5769999999999996e-06 2024-09-14T06:17:01.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.118 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689718+0000", 2024-09-14T06:17:01.118 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035347376999999999 2024-09-14T06:17:01.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.118 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.689941+0000", 2024-09-14T06:17:01.119 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022233499999999999 2024-09-14T06:17:01.119 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.119 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.119 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.119 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.690205+0000", 2024-09-14T06:17:01.119 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00026421799999999998 2024-09-14T06:17:01.119 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.119 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.119 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.119 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.119 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.119 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16121 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 3145728~7288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.119 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654381+0000", 2024-09-14T06:17:01.119 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.442391067999999, 2024-09-14T06:17:01.120 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.036696549000000002, 2024-09-14T06:17:01.120 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.120 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.120 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.120 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.120 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.120 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.120 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16121 2024-09-14T06:17:01.120 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.120 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.120 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.120 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.120 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654381+0000", 2024-09-14T06:17:01.121 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.121 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.121 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654381+0000", 2024-09-14T06:17:01.121 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.121 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.121 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654381+0000", 2024-09-14T06:17:01.121 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9999999999999997e-07 2024-09-14T06:17:01.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.122 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.122 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.122 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654384+0000", 2024-09-14T06:17:01.122 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8499999999999998e-06 2024-09-14T06:17:01.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.122 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.122 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.122 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654404+0000", 2024-09-14T06:17:01.122 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9355999999999999e-05 2024-09-14T06:17:01.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.122 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.122 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.123 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654417+0000", 2024-09-14T06:17:01.123 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.295e-05 2024-09-14T06:17:01.123 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.123 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.123 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.123 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.690327+0000", 2024-09-14T06:17:01.123 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035910899000000003 2024-09-14T06:17:01.123 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.123 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.123 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.123 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.690557+0000", 2024-09-14T06:17:01.123 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022935499999999999 2024-09-14T06:17:01.123 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.124 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.124 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.124 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.691077+0000", 2024-09-14T06:17:01.124 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00052043900000000004 2024-09-14T06:17:01.124 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.124 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.124 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.124 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.124 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.124 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16122 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.124 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654428+0000", 2024-09-14T06:17:01.124 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.442343722, 2024-09-14T06:17:01.124 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.037792064, 2024-09-14T06:17:01.125 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.125 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.125 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.125 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.125 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.125 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.125 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16122 2024-09-14T06:17:01.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.125 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.125 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.125 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.125 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654428+0000", 2024-09-14T06:17:01.125 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.126 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.126 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.126 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654428+0000", 2024-09-14T06:17:01.126 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.126 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.126 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.126 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.126 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654429+0000", 2024-09-14T06:17:01.126 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8999999999999995e-07 2024-09-14T06:17:01.126 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.126 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.126 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.126 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654432+0000", 2024-09-14T06:17:01.126 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0130000000000002e-06 2024-09-14T06:17:01.126 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.126 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.127 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.127 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654442+0000", 2024-09-14T06:17:01.127 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9199999999999999e-06 2024-09-14T06:17:01.127 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.127 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.127 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.127 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654449+0000", 2024-09-14T06:17:01.127 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:17:01.127 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.127 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.127 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.127 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.691192+0000", 2024-09-14T06:17:01.127 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.036742958999999999 2024-09-14T06:17:01.127 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.127 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.128 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.128 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.691445+0000", 2024-09-14T06:17:01.128 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000253608 2024-09-14T06:17:01.128 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.128 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.128 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.128 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692220+0000", 2024-09-14T06:17:01.128 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00077489400000000004 2024-09-14T06:17:01.128 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.128 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.128 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.128 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.128 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.128 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16123 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.128 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654458+0000", 2024-09-14T06:17:01.129 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.442313732999999, 2024-09-14T06:17:01.129 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038257820999999997, 2024-09-14T06:17:01.129 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.129 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.129 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.129 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.129 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.129 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.129 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16123 2024-09-14T06:17:01.129 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.129 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.129 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.129 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.130 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654458+0000", 2024-09-14T06:17:01.130 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.130 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.130 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.130 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654458+0000", 2024-09-14T06:17:01.130 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.130 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.131 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.131 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654459+0000", 2024-09-14T06:17:01.131 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0699999999999996e-07 2024-09-14T06:17:01.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.131 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.131 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.131 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654461+0000", 2024-09-14T06:17:01.131 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.633e-06 2024-09-14T06:17:01.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.131 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.131 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.131 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654470+0000", 2024-09-14T06:17:01.131 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0440000000000001e-06 2024-09-14T06:17:01.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.132 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.132 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.132 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654496+0000", 2024-09-14T06:17:01.132 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5279999999999999e-05 2024-09-14T06:17:01.132 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.132 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.132 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.132 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692393+0000", 2024-09-14T06:17:01.132 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.037897532999999997 2024-09-14T06:17:01.132 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.132 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.132 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.132 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692526+0000", 2024-09-14T06:17:01.133 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013326899999999999 2024-09-14T06:17:01.133 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.133 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.133 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.133 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692716+0000", 2024-09-14T06:17:01.133 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00018935499999999999 2024-09-14T06:17:01.133 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.133 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.133 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.133 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.133 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.133 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16124 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 4521984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.133 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654506+0000", 2024-09-14T06:17:01.133 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.442265396, 2024-09-14T06:17:01.133 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038503725000000003, 2024-09-14T06:17:01.134 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.134 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.134 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.134 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.134 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.134 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.134 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16124 2024-09-14T06:17:01.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.134 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.134 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.134 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.134 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654506+0000", 2024-09-14T06:17:01.134 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.134 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.135 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.135 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654506+0000", 2024-09-14T06:17:01.135 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.135 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.135 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.135 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.135 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654507+0000", 2024-09-14T06:17:01.135 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1299999999999999e-07 2024-09-14T06:17:01.135 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.135 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.135 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.135 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654518+0000", 2024-09-14T06:17:01.135 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0696999999999999e-05 2024-09-14T06:17:01.135 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.135 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.135 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.136 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654531+0000", 2024-09-14T06:17:01.136 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3596e-05 2024-09-14T06:17:01.136 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.136 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.136 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.136 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654539+0000", 2024-09-14T06:17:01.136 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6170000000000001e-06 2024-09-14T06:17:01.136 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.136 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.136 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.136 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692766+0000", 2024-09-14T06:17:01.136 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038227401000000001 2024-09-14T06:17:01.136 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.136 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.136 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.137 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.692869+0000", 2024-09-14T06:17:01.137 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000102216 2024-09-14T06:17:01.137 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.137 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.137 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.137 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693010+0000", 2024-09-14T06:17:01.137 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000141385 2024-09-14T06:17:01.137 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.137 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.137 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.137 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.137 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.137 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16125 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.137 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654549+0000", 2024-09-14T06:17:01.138 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.44222224, 2024-09-14T06:17:01.138 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038734667, 2024-09-14T06:17:01.138 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.138 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.138 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.138 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.138 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.138 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.138 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16125 2024-09-14T06:17:01.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.138 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.138 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.138 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.138 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654549+0000", 2024-09-14T06:17:01.139 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.139 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.139 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.139 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.139 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654549+0000", 2024-09-14T06:17:01.139 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.139 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.139 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.139 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.139 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654550+0000", 2024-09-14T06:17:01.139 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6700000000000003e-07 2024-09-14T06:17:01.139 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.139 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.139 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.140 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654562+0000", 2024-09-14T06:17:01.140 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1926000000000001e-05 2024-09-14T06:17:01.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.140 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.140 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.140 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654575+0000", 2024-09-14T06:17:01.140 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3217000000000001e-05 2024-09-14T06:17:01.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.140 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.140 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.140 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654583+0000", 2024-09-14T06:17:01.140 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3660000000000003e-06 2024-09-14T06:17:01.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.141 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.141 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.141 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693055+0000", 2024-09-14T06:17:01.141 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038471752999999997 2024-09-14T06:17:01.141 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.141 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.141 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.141 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693152+0000", 2024-09-14T06:17:01.141 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7548999999999997e-05 2024-09-14T06:17:01.141 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.141 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.141 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.141 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693284+0000", 2024-09-14T06:17:01.141 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000132089 2024-09-14T06:17:01.141 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.142 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.142 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.142 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.142 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.142 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16126 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 5439488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.142 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654595+0000", 2024-09-14T06:17:01.142 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.442176537000002, 2024-09-14T06:17:01.142 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038961922000000003, 2024-09-14T06:17:01.142 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.142 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.142 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.142 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.142 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.142 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.143 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16126 2024-09-14T06:17:01.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.143 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.143 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.143 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.143 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654595+0000", 2024-09-14T06:17:01.143 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.143 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.143 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.143 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654595+0000", 2024-09-14T06:17:01.143 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.144 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.144 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.144 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654596+0000", 2024-09-14T06:17:01.144 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7700000000000004e-07 2024-09-14T06:17:01.144 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.144 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.144 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.144 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654600+0000", 2024-09-14T06:17:01.144 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.216e-06 2024-09-14T06:17:01.144 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.144 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.144 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.144 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654610+0000", 2024-09-14T06:17:01.144 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7070000000000004e-06 2024-09-14T06:17:01.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.145 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.145 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.145 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654617+0000", 2024-09-14T06:17:01.145 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2200000000000003e-06 2024-09-14T06:17:01.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.145 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.145 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.145 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693327+0000", 2024-09-14T06:17:01.145 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.038709427999999997 2024-09-14T06:17:01.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.145 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.145 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.146 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693426+0000", 2024-09-14T06:17:01.146 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9401999999999995e-05 2024-09-14T06:17:01.146 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.146 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.146 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.146 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693557+0000", 2024-09-14T06:17:01.146 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013117200000000001 2024-09-14T06:17:01.146 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.146 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.146 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.146 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.146 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.146 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16127 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 5898240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.146 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654627+0000", 2024-09-14T06:17:01.147 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.442145104000002, 2024-09-14T06:17:01.147 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.040234709, 2024-09-14T06:17:01.147 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.147 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.147 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.147 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.147 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.147 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.147 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16127 2024-09-14T06:17:01.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.148 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.148 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.148 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.148 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654627+0000", 2024-09-14T06:17:01.148 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.148 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.148 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.148 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.148 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654627+0000", 2024-09-14T06:17:01.148 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.148 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.148 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.148 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.149 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654627+0000", 2024-09-14T06:17:01.149 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4700000000000001e-07 2024-09-14T06:17:01.149 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.149 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.149 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.149 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654630+0000", 2024-09-14T06:17:01.149 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8600000000000001e-06 2024-09-14T06:17:01.149 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.149 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.149 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.150 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654639+0000", 2024-09-14T06:17:01.150 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2599999999999994e-06 2024-09-14T06:17:01.150 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.150 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.150 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.150 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654647+0000", 2024-09-14T06:17:01.150 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2269999999999996e-06 2024-09-14T06:17:01.150 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.150 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.150 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.150 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.693851+0000", 2024-09-14T06:17:01.150 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.039203937000000001 2024-09-14T06:17:01.150 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.150 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.151 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.151 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.694465+0000", 2024-09-14T06:17:01.151 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00061404799999999996 2024-09-14T06:17:01.151 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.151 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.151 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.151 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.694861+0000", 2024-09-14T06:17:01.151 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00039662999999999998 2024-09-14T06:17:01.151 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.151 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.151 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.151 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.151 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.152 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16128 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.152 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654656+0000", 2024-09-14T06:17:01.152 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.442115641000001, 2024-09-14T06:17:01.152 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.040797799000000003, 2024-09-14T06:17:01.152 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.152 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.152 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.152 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.152 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.152 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.152 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16128 2024-09-14T06:17:01.152 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.152 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.153 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.153 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.153 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654656+0000", 2024-09-14T06:17:01.153 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.153 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.153 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.153 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.153 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654656+0000", 2024-09-14T06:17:01.153 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.153 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.153 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.153 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.153 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654657+0000", 2024-09-14T06:17:01.154 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2699999999999999e-07 2024-09-14T06:17:01.154 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.154 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.154 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.154 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654659+0000", 2024-09-14T06:17:01.154 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6400000000000001e-06 2024-09-14T06:17:01.154 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.154 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.154 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.154 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654669+0000", 2024-09-14T06:17:01.154 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1730000000000001e-06 2024-09-14T06:17:01.154 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.154 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.155 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.155 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654685+0000", 2024-09-14T06:17:01.155 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6237000000000001e-05 2024-09-14T06:17:01.155 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.155 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.155 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.155 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.694952+0000", 2024-09-14T06:17:01.155 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.040267078999999997 2024-09-14T06:17:01.155 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.155 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.155 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.155 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695174+0000", 2024-09-14T06:17:01.155 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000221745 2024-09-14T06:17:01.155 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.155 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.156 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.156 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695454+0000", 2024-09-14T06:17:01.156 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000280198 2024-09-14T06:17:01.156 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.156 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.156 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.156 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.156 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.156 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16129 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.156 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654697+0000", 2024-09-14T06:17:01.156 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.442074545000001, 2024-09-14T06:17:01.156 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.041164486, 2024-09-14T06:17:01.156 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.156 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.157 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.157 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.157 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.157 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.157 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16129 2024-09-14T06:17:01.157 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.157 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.157 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.157 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.157 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654697+0000", 2024-09-14T06:17:01.157 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.158 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.158 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.158 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654697+0000", 2024-09-14T06:17:01.158 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.158 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.158 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.158 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654698+0000", 2024-09-14T06:17:01.158 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:17:01.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.158 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.158 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.158 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654701+0000", 2024-09-14T06:17:01.159 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.756e-06 2024-09-14T06:17:01.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.159 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.159 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.159 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654710+0000", 2024-09-14T06:17:01.159 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7669999999999998e-06 2024-09-14T06:17:01.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.159 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.159 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.159 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654718+0000", 2024-09-14T06:17:01.159 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.097e-06 2024-09-14T06:17:01.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.159 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.159 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.160 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695552+0000", 2024-09-14T06:17:01.160 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.040834148000000001 2024-09-14T06:17:01.160 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.160 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.160 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.160 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695690+0000", 2024-09-14T06:17:01.160 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000138479 2024-09-14T06:17:01.160 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.160 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.160 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.160 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695862+0000", 2024-09-14T06:17:01.160 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017150499999999999 2024-09-14T06:17:01.160 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.160 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.161 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.161 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.161 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.161 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16130 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.161 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654727+0000", 2024-09-14T06:17:01.161 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.442044738, 2024-09-14T06:17:01.161 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.041755107999999999, 2024-09-14T06:17:01.161 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.161 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.161 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.162 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.162 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.162 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.162 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16130 2024-09-14T06:17:01.162 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.162 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.162 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.162 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.162 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654727+0000", 2024-09-14T06:17:01.162 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.162 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.163 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.163 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.163 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654727+0000", 2024-09-14T06:17:01.163 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.163 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.163 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.163 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.163 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654728+0000", 2024-09-14T06:17:01.163 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6300000000000004e-07 2024-09-14T06:17:01.163 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.163 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.163 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.163 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654730+0000", 2024-09-14T06:17:01.163 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7439999999999999e-06 2024-09-14T06:17:01.163 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.164 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.164 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.164 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654742+0000", 2024-09-14T06:17:01.164 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1036000000000001e-05 2024-09-14T06:17:01.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.164 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.164 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.164 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654748+0000", 2024-09-14T06:17:01.164 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9199999999999998e-06 2024-09-14T06:17:01.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.164 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.164 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.164 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.695969+0000", 2024-09-14T06:17:01.165 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.041220672 2024-09-14T06:17:01.165 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.165 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.165 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.165 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696192+0000", 2024-09-14T06:17:01.165 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022279799999999999 2024-09-14T06:17:01.165 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.165 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.165 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.165 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696482+0000", 2024-09-14T06:17:01.165 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00029017499999999999 2024-09-14T06:17:01.166 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.166 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.166 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.166 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.166 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16131 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.166 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654758+0000", 2024-09-14T06:17:01.166 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.442013897999999, 2024-09-14T06:17:01.166 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.042069361999999999, 2024-09-14T06:17:01.166 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.166 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.166 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.166 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.167 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.167 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.167 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16131 2024-09-14T06:17:01.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.167 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.167 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.167 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.167 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654758+0000", 2024-09-14T06:17:01.167 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.167 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.167 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.167 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654758+0000", 2024-09-14T06:17:01.167 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.168 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.168 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.168 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654759+0000", 2024-09-14T06:17:01.168 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3e-07 2024-09-14T06:17:01.168 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.168 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.168 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.168 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654761+0000", 2024-09-14T06:17:01.168 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7599999999999998e-06 2024-09-14T06:17:01.168 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.168 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.168 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.168 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654773+0000", 2024-09-14T06:17:01.168 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1216999999999999e-05 2024-09-14T06:17:01.168 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.168 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.169 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.169 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654780+0000", 2024-09-14T06:17:01.169 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3529999999999997e-06 2024-09-14T06:17:01.169 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.169 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.169 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.169 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696524+0000", 2024-09-14T06:17:01.169 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.041744594000000003 2024-09-14T06:17:01.169 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.169 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.169 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.169 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696652+0000", 2024-09-14T06:17:01.169 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000127866 2024-09-14T06:17:01.169 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.170 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.170 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.170 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696827+0000", 2024-09-14T06:17:01.170 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000174842 2024-09-14T06:17:01.170 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.170 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.170 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.170 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.170 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.170 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16132 3.0 3:04ef7a91:::rbd_data.1152c77b28d3.0000000000000004:8 [sparse-read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.170 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.654840+0000", 2024-09-14T06:17:01.170 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.441931368999999, 2024-09-14T06:17:01.170 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.042362179999999999, 2024-09-14T06:17:01.170 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.171 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.171 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.171 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.171 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.171 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.171 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16132 2024-09-14T06:17:01.171 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.171 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.171 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.171 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.171 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654840+0000", 2024-09-14T06:17:01.171 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.171 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.171 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.171 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.172 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654840+0000", 2024-09-14T06:17:01.172 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.172 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.172 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.172 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.172 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654841+0000", 2024-09-14T06:17:01.172 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9300000000000006e-07 2024-09-14T06:17:01.172 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.172 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.172 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.172 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654845+0000", 2024-09-14T06:17:01.172 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.18e-06 2024-09-14T06:17:01.172 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.172 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.173 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.173 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654867+0000", 2024-09-14T06:17:01.173 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.27e-05 2024-09-14T06:17:01.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.173 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.173 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.173 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.654892+0000", 2024-09-14T06:17:01.173 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4284e-05 2024-09-14T06:17:01.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.173 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.173 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.173 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.696907+0000", 2024-09-14T06:17:01.173 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.042015957999999999 2024-09-14T06:17:01.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.174 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.174 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.174 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.697032+0000", 2024-09-14T06:17:01.174 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012415899999999999 2024-09-14T06:17:01.174 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.174 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.174 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.174 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.697203+0000", 2024-09-14T06:17:01.174 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017090599999999999 2024-09-14T06:17:01.174 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.174 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.175 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.175 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.175 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.175 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:16181 3.0 3:067f695c:::rbd_data.1152fc09e7e6.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:01.175 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.757965+0000", 2024-09-14T06:17:01.175 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.338807044999999, 2024-09-14T06:17:01.175 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065959498000000005, 2024-09-14T06:17:01.175 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.175 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.175 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:01.175 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.175 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.175 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.176 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 16181 2024-09-14T06:17:01.176 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.176 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.176 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.176 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.176 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.757965+0000", 2024-09-14T06:17:01.176 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.176 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.176 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.177 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.177 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.757965+0000", 2024-09-14T06:17:01.177 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.177 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.177 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.177 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.177 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.757967+0000", 2024-09-14T06:17:01.177 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6869999999999999e-06 2024-09-14T06:17:01.177 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.177 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.178 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.178 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.757984+0000", 2024-09-14T06:17:01.178 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.658e-05 2024-09-14T06:17:01.178 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.178 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.178 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.178 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.758025+0000", 2024-09-14T06:17:01.178 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1158999999999997e-05 2024-09-14T06:17:01.178 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.178 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.178 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.178 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.758049+0000", 2024-09-14T06:17:01.179 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3442999999999999e-05 2024-09-14T06:17:01.179 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.179 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.179 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.179 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.758200+0000", 2024-09-14T06:17:01.179 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015118900000000001 2024-09-14T06:17:01.180 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.180 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.180 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.180 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.758340+0000", 2024-09-14T06:17:01.180 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000140722 2024-09-14T06:17:01.180 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.181 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.181 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 7", 2024-09-14T06:17:01.181 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.769990+0000", 2024-09-14T06:17:01.181 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011649286999999999 2024-09-14T06:17:01.181 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.181 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.181 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:01.182 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.823608+0000", 2024-09-14T06:17:01.182 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.053617797000000002 2024-09-14T06:17:01.182 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.182 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.182 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:01.182 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.823740+0000", 2024-09-14T06:17:01.182 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000132132 2024-09-14T06:17:01.182 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.182 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.182 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:01.182 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.823843+0000", 2024-09-14T06:17:01.182 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000103712 2024-09-14T06:17:01.182 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.182 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.183 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.183 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.823924+0000", 2024-09-14T06:17:01.183 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0790000000000004e-05 2024-09-14T06:17:01.183 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.183 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.183 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.183 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.184 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.184 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:18155 3.17 3:e91637d8:::rbd_data.115232f4ac2b.0000000000000000:a [sparse-read 5767168~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T06:17:01.184 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:41.255599+0000", 2024-09-14T06:17:01.184 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.841172972999999, 2024-09-14T06:17:01.184 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035557431, 2024-09-14T06:17:01.184 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.184 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.184 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.184 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.184 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.184 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.184 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 18155 2024-09-14T06:17:01.184 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.184 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.185 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.185 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.185 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255599+0000", 2024-09-14T06:17:01.185 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.185 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.185 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.185 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255599+0000", 2024-09-14T06:17:01.185 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.185 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.185 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.185 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255600+0000", 2024-09-14T06:17:01.185 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.37e-06 2024-09-14T06:17:01.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255624+0000", 2024-09-14T06:17:01.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3938e-05 2024-09-14T06:17:01.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255679+0000", 2024-09-14T06:17:01.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4768000000000001e-05 2024-09-14T06:17:01.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255699+0000", 2024-09-14T06:17:01.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0058e-05 2024-09-14T06:17:01.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.289431+0000", 2024-09-14T06:17:01.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.03373259 2024-09-14T06:17:01.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.289640+0000", 2024-09-14T06:17:01.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000208464 2024-09-14T06:17:01.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.291156+0000", 2024-09-14T06:17:01.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001516243 2024-09-14T06:17:01.187 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.188 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.188 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.188 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.188 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:18156 3.17 3:e91637d8:::rbd_data.115232f4ac2b.0000000000000000:a [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T06:17:01.188 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:41.255741+0000", 2024-09-14T06:17:01.188 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.841031199, 2024-09-14T06:17:01.188 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035889028000000003, 2024-09-14T06:17:01.188 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.188 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.188 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:01.188 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.188 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.188 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.189 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 18156 2024-09-14T06:17:01.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.189 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.189 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.189 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.189 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255741+0000", 2024-09-14T06:17:01.189 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.189 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.189 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.189 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255741+0000", 2024-09-14T06:17:01.189 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.190 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.190 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255742+0000", 2024-09-14T06:17:01.190 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3659999999999999e-06 2024-09-14T06:17:01.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.190 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.190 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255760+0000", 2024-09-14T06:17:01.190 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8031999999999999e-05 2024-09-14T06:17:01.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.191 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255784+0000", 2024-09-14T06:17:01.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4454999999999999e-05 2024-09-14T06:17:01.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.191 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.255803+0000", 2024-09-14T06:17:01.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8541000000000001e-05 2024-09-14T06:17:01.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.291231+0000", 2024-09-14T06:17:01.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.035428098999999998 2024-09-14T06:17:01.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.291420+0000", 2024-09-14T06:17:01.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000188786 2024-09-14T06:17:01.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:41.291630+0000", 2024-09-14T06:17:01.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00020974899999999999 2024-09-14T06:17:01.192 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.193 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.193 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.193 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:30547 3.1f 3:f84dfb53:::rbd_data.11529eb229bd.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write 16226533~50642] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T06:17:01.193 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:01.193 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.906962767, 2024-09-14T06:17:01.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0079770020000001, 2024-09-14T06:17:01.193 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:01.193 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:01.193 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:01.193 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:01.193 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:01.194 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:01.194 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 30547 2024-09-14T06:17:01.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.194 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:01.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.194 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:01.194 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:01.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.194 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:01.194 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:01.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:01.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.194 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:01.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189811+0000", 2024-09-14T06:17:01.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2060000000000001e-06 2024-09-14T06:17:01.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:01.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189871+0000", 2024-09-14T06:17:01.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9862999999999999e-05 2024-09-14T06:17:01.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:01.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189899+0000", 2024-09-14T06:17:01.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7849000000000001e-05 2024-09-14T06:17:01.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:01.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189915+0000", 2024-09-14T06:17:01.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:17:01.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.196 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190024+0000", 2024-09-14T06:17:01.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000109202 2024-09-14T06:17:01.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.196 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:17:01.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190041+0000", 2024-09-14T06:17:01.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7665000000000001e-05 2024-09-14T06:17:01.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.196 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188512+0000", 2024-09-14T06:17:01.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.99847100700000002 2024-09-14T06:17:01.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for missing object", 2024-09-14T06:17:01.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188646+0000", 2024-09-14T06:17:01.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000133986 2024-09-14T06:17:01.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:01.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193097+0000", 2024-09-14T06:17:01.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044507059999999996 2024-09-14T06:17:01.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:01.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193183+0000", 2024-09-14T06:17:01.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5496999999999998e-05 2024-09-14T06:17:01.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.198 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:17:01.198 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193307+0000", 2024-09-14T06:17:01.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000124611 2024-09-14T06:17:01.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.198 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:01.198 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.194318+0000", 2024-09-14T06:17:01.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0010106 2024-09-14T06:17:01.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.198 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:01.198 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197670+0000", 2024-09-14T06:17:01.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033523170000000001 2024-09-14T06:17:01.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.199 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:01.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197712+0000", 2024-09-14T06:17:01.199 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1681000000000003e-05 2024-09-14T06:17:01.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:01.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:01.199 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:01.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197786+0000", 2024-09-14T06:17:01.199 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4148999999999998e-05 2024-09-14T06:17:01.199 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.199 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.199 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.199 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:01.199 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:01.199 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:01.200 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:17:01.202 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:17:01.203 INFO:teuthology.orchestra.run.smithi039.stdout:3127 trunc from 0x6ece1bc to 0xc78a7a9 2024-09-14T06:17:01.203 INFO:teuthology.orchestra.run.smithi039.stdout:3128 punch from 0x919003f to 0x91995f7, (0x95b8 bytes) 2024-09-14T06:17:01.203 INFO:teuthology.orchestra.run.smithi039.stdout:3129 writesame 0x9192694 thru 0x919754f data_size 0x2(0x4ebc bytes) 2024-09-14T06:17:01.203 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:01.203 INFO:teuthology.orchestra.run.smithi039.stdout:3132 write 0xd7b8575 thru 0xd7c1de4 (0x9870 bytes) 2024-09-14T06:17:01.203 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:01.203 INFO:teuthology.orchestra.run.smithi039.stdout:3135 trunc from 0xd7c1de5 to 0xab1336b 2024-09-14T06:17:01.203 INFO:teuthology.orchestra.run.smithi039.stdout:3136 read 0xa1f1205 thru 0xa1fed47 (0xdb43 bytes) 2024-09-14T06:17:01.203 INFO:teuthology.orchestra.run.smithi039.stdout:3138 write 0x99da625 thru 0x99e0c03 (0x65df bytes) 2024-09-14T06:17:01.204 INFO:teuthology.orchestra.run.smithi039.stdout:3139 write 0x7dd2fbc thru 0x7dd604d (0x3092 bytes) 2024-09-14T06:17:01.204 INFO:teuthology.orchestra.run.smithi039.stdout:3141 read 0x9c79ea9 thru 0x9c7a8c7 (0xa1f bytes) 2024-09-14T06:17:01.204 INFO:teuthology.orchestra.run.smithi039.stdout:3142 write 0x7e791f0 thru 0x7e7fcfb (0x6b0c bytes) 2024-09-14T06:17:01.204 INFO:teuthology.orchestra.run.smithi039.stdout:3143 punch from 0x651a882 to 0x65200ba, (0x5838 bytes) 2024-09-14T06:17:01.204 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:01.199+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:01.218 INFO:teuthology.orchestra.run.smithi039.stdout:3144 write 0xe864d90 thru 0xe86e27d (0x94ee bytes) 2024-09-14T06:17:01.222 INFO:teuthology.orchestra.run.smithi039.stdout:3146 write 0x74c30c5 thru 0x74c4682 (0x15be bytes) 2024-09-14T06:17:01.230 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:01.230 INFO:teuthology.orchestra.run.smithi039.stdout:3149 write 0xba59029 thru 0xba5ffb0 (0x6f88 bytes) 2024-09-14T06:17:01.231 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:17:01.232 INFO:teuthology.orchestra.run.smithi039.stdout:3150 punch from 0x77c0dda to 0x77c4224, (0x344a bytes) 2024-09-14T06:17:01.232 INFO:teuthology.orchestra.run.smithi039.stdout:3151 read 0x23c7c26 thru 0x23cf353 (0x772e bytes) 2024-09-14T06:17:01.233 INFO:teuthology.orchestra.run.smithi039.stdout:3153 write 0x646cd15 thru 0x6479d82 (0xd06e bytes) 2024-09-14T06:17:01.235 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:01.235 INFO:teuthology.orchestra.run.smithi039.stdout:3157 read 0xe0baa03 thru 0xe0c5956 (0xaf54 bytes) 2024-09-14T06:17:01.302 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:01.302+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:01.373 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:01.373 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:01.373 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:01.374 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:01.386 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:01.386 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:01.387 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:17:01.403 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:01.403 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:01.419 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:01.419 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:01.431 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:01.430+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:01.436 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:01.437 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:01.454 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:01.455 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:01.475 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:01.475 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:01.493 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:01.493 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:01.505 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:01.503+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:01.514 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:01.515 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:01.547 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:01.548 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:01.562 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:17:01.605 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:01.602+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:01.678 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:01.678 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:01.692 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:17:01.706 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:01.702+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:01.807 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:01.806+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:01.815 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:01.815 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:01.829 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:17:01.909 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:01.906+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:01.954 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:01.954 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:01.968 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:17:02.010 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:02.006+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:02.085 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:02.085 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:02.099 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:17:02.110 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:02.106+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:02.186 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-09-14T06:17:02.193 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:02.194 INFO:teuthology.orchestra.run.smithi039.stdout:3162 writesame 0x670c83b thru 0x6714ee7 data_size 0x1(0x86ad bytes) 2024-09-14T06:17:02.211 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:02.210+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:02.214 INFO:teuthology.orchestra.run.smithi039.stdout:3163 write 0xcc21e4e thru 0xcc237d2 (0x1985 bytes) 2024-09-14T06:17:02.218 INFO:teuthology.orchestra.run.smithi039.stdout:3164 read 0x7f31016 thru 0x7f3828e (0x7279 bytes) 2024-09-14T06:17:02.219 INFO:teuthology.orchestra.run.smithi039.stdout:3166 punch from 0x21f00c9 to 0x21fb841, (0xb778 bytes) 2024-09-14T06:17:02.219 INFO:teuthology.orchestra.run.smithi039.stdout:3167 writesame 0x9400269 thru 0x940ea3e data_size 0x19(0xe7d6 bytes) 2024-09-14T06:17:02.224 INFO:teuthology.orchestra.run.smithi039.stdout:3168 punch from 0x86513da to 0x865a1aa, (0x8dd0 bytes) 2024-09-14T06:17:02.224 INFO:teuthology.orchestra.run.smithi039.stdout:3169 write 0xc8a9caa thru 0xc8ac0a9 (0x2400 bytes) 2024-09-14T06:17:02.225 INFO:teuthology.orchestra.run.smithi039.stdout:3170 writesame 0x4ff1aa9 thru 0x4ff2298 data_size 0x8(0x7f0 bytes) 2024-09-14T06:17:02.230 INFO:teuthology.orchestra.run.smithi039.stdout:3171 writesame 0x98f1fb8 thru 0x98f5781 data_size 0x25(0x37ca bytes) 2024-09-14T06:17:02.241 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:02.241 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:02.250 INFO:teuthology.orchestra.run.smithi039.stdout:3172 trunc from 0xe86e27e to 0x3418d6c 2024-09-14T06:17:02.263 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:17:02.313 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:02.310+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:02.313 INFO:teuthology.orchestra.run.smithi039.stdout:3173 writesame 0xe04f352 thru 0xe050c93 data_size 0x35(0x1942 bytes) 2024-09-14T06:17:02.313 INFO:teuthology.orchestra.run.smithi039.stdout:3174 writesame 0x1bae29d thru 0x1bb1359 data_size 0x1(0x30bd bytes) 2024-09-14T06:17:02.330 INFO:teuthology.orchestra.run.smithi039.stdout:3175 writesame 0xbdb094c thru 0xbdbcebb data_size 0xa2(0xc570 bytes) 2024-09-14T06:17:02.334 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:02.334 INFO:teuthology.orchestra.run.smithi039.stdout:3177 write 0x66a93d5 thru 0x66ad966 (0x4592 bytes) 2024-09-14T06:17:02.336 INFO:teuthology.orchestra.run.smithi039.stdout:3179 write 0x213bf3e thru 0x213e487 (0x254a bytes) 2024-09-14T06:17:02.338 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:02.342 INFO:teuthology.orchestra.run.smithi039.stdout:3181 trunc from 0xe050c94 to 0xc894468 2024-09-14T06:17:02.359 INFO:teuthology.orchestra.run.smithi039.stdout:3182 punch from 0x9c00436 to 0x9c0b333, (0xaefd bytes) 2024-09-14T06:17:02.359 INFO:teuthology.orchestra.run.smithi039.stdout:3183 writesame 0x9ab29dc thru 0x9ab4311 data_size 0x7(0x1936 bytes) 2024-09-14T06:17:02.369 INFO:teuthology.orchestra.run.smithi039.stdout:3184 punch from 0x3a85af to 0x3af79f, (0x71f0 bytes) 2024-09-14T06:17:02.369 INFO:teuthology.orchestra.run.smithi039.stdout:3185 writesame 0x8d97268 thru 0x8da25c0 data_size 0x7(0xb359 bytes) 2024-09-14T06:17:02.376 INFO:teuthology.orchestra.run.smithi039.stdout:3186 writesame 0x89da750 thru 0x89df7de data_size 0x1(0x508f bytes) 2024-09-14T06:17:02.389 INFO:teuthology.orchestra.run.smithi039.stdout:3187 read 0x4bd0ad1 thru 0x4bda2ea (0x981a bytes) 2024-09-14T06:17:02.390 INFO:teuthology.orchestra.run.smithi039.stdout:3188 write 0x39ee64a thru 0x39fba79 (0xd430 bytes) 2024-09-14T06:17:02.393 INFO:teuthology.orchestra.run.smithi039.stdout:3192 punch from 0x587a23c to 0x5884cb2, (0xaa76 bytes) 2024-09-14T06:17:02.393 INFO:teuthology.orchestra.run.smithi039.stdout:3194 writesame 0x57cfd9c thru 0x57d0b46 data_size 0x1(0xdab bytes) 2024-09-14T06:17:02.413 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:02.410+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:02.415 INFO:teuthology.orchestra.run.smithi039.stdout:3196 write 0xe9f7942 thru 0xe9f852c (0xbeb bytes) 2024-09-14T06:17:02.419 INFO:teuthology.orchestra.run.smithi039.stdout:3199 punch from 0xcc6b94e to 0xcc78752, (0xce04 bytes) 2024-09-14T06:17:02.419 INFO:teuthology.orchestra.run.smithi039.stdout:3200 write 0xf8757d thru 0xf969bc (0xf440 bytes) 2024-09-14T06:17:02.435 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:02.435 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:02.440 INFO:teuthology.orchestra.run.smithi039.stdout:3201 trunc from 0xe9f852d to 0x7644e2b 2024-09-14T06:17:02.455 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:17:02.457 INFO:teuthology.orchestra.run.smithi039.stdout:3202 write 0x22824b4 thru 0x228f6c9 (0xd216 bytes) 2024-09-14T06:17:02.460 INFO:teuthology.orchestra.run.smithi039.stdout:3203 punch from 0x6648c22 to 0x66576df, (0xeabd bytes) 2024-09-14T06:17:02.460 INFO:teuthology.orchestra.run.smithi039.stdout:3205 punch from 0x2f83f68 to 0x2f87cd9, (0x3d71 bytes) 2024-09-14T06:17:02.460 INFO:teuthology.orchestra.run.smithi039.stdout:3206 read 0x6cf52b3 thru 0x6cfde56 (0x8ba4 bytes) 2024-09-14T06:17:02.461 INFO:teuthology.orchestra.run.smithi039.stdout:3207 write 0x58ef71b thru 0x58fe960 (0xf246 bytes) 2024-09-14T06:17:02.464 INFO:teuthology.orchestra.run.smithi039.stdout:3208 punch from 0x26fd035 to 0x2709659, (0xc624 bytes) 2024-09-14T06:17:02.465 INFO:teuthology.orchestra.run.smithi039.stdout:3209 read 0x74b68f7 thru 0x74baf8a (0x4694 bytes) 2024-09-14T06:17:02.465 INFO:teuthology.orchestra.run.smithi039.stdout:3210 read 0x7305c84 thru 0x731040e (0xa78b bytes) 2024-09-14T06:17:02.466 INFO:teuthology.orchestra.run.smithi039.stdout:3212 read 0x5a7d11c thru 0x5a84d14 (0x7bf9 bytes) 2024-09-14T06:17:02.466 INFO:teuthology.orchestra.run.smithi039.stdout:3213 read 0x486c020 thru 0x48732a1 (0x7282 bytes) 2024-09-14T06:17:02.468 INFO:teuthology.orchestra.run.smithi039.stdout:3214 punch from 0x653d948 to 0x654d4f5, (0xfbad bytes) 2024-09-14T06:17:02.468 INFO:teuthology.orchestra.run.smithi039.stdout:3215 writesame 0x688fde1 thru 0x68988a9 data_size 0xd(0x8ac9 bytes) 2024-09-14T06:17:02.473 INFO:teuthology.orchestra.run.smithi039.stdout:3216 read 0x261e719 thru 0x262252d (0x3e15 bytes) 2024-09-14T06:17:02.474 INFO:teuthology.orchestra.run.smithi039.stdout:3217 read 0x4851891 thru 0x485d903 (0xc073 bytes) 2024-09-14T06:17:02.475 INFO:teuthology.orchestra.run.smithi039.stdout:3219 read 0x1942788 thru 0x1948171 (0x59ea bytes) 2024-09-14T06:17:02.476 INFO:teuthology.orchestra.run.smithi039.stdout:3220 read 0x48adae0 thru 0x48b5902 (0x7e23 bytes) 2024-09-14T06:17:02.499 INFO:teuthology.orchestra.run.smithi039.stdout:3221 write 0xddd41d3 thru 0xddda43a (0x6268 bytes) 2024-09-14T06:17:02.502 INFO:teuthology.orchestra.run.smithi039.stdout:3223 write 0x21fe83a thru 0x22016cc (0x2e93 bytes) 2024-09-14T06:17:02.505 INFO:teuthology.orchestra.run.smithi039.stdout:3224 read 0x1f36c4a thru 0x1f427ca (0xbb81 bytes) 2024-09-14T06:17:02.506 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:02.506 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:02.506 INFO:teuthology.orchestra.run.smithi039.stdout:3227 read 0x4718814 thru 0x4728118 (0xf905 bytes) 2024-09-14T06:17:02.506 INFO:teuthology.orchestra.run.smithi039.stdout:3228 punch from 0xad4488b to 0xad50cef, (0xc464 bytes) 2024-09-14T06:17:02.507 INFO:teuthology.orchestra.run.smithi039.stdout:3231 writesame 0x5e40f93 thru 0x5e49ef5 data_size 0x1(0x8f63 bytes) 2024-09-14T06:17:02.514 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:02.510+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:02.555 INFO:teuthology.orchestra.run.smithi039.stdout:3232 write 0xd26749f thru 0xd26bb90 (0x46f2 bytes) 2024-09-14T06:17:02.565 INFO:teuthology.orchestra.run.smithi039.stdout:3234 trunc from 0xddda43b to 0xac84a7b 2024-09-14T06:17:02.583 INFO:teuthology.orchestra.run.smithi039.stdout:3235 read 0x4f9d009 thru 0x4fa82f8 (0xb2f0 bytes) 2024-09-14T06:17:02.583 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:02.583 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:02.584 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:02.584 INFO:teuthology.orchestra.run.smithi039.stdout:3237 punch from 0x381016c to 0x381dbdf, (0xda73 bytes) 2024-09-14T06:17:02.584 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:02.585 INFO:teuthology.orchestra.run.smithi039.stdout:3239 punch from 0x9447001 to 0x9452ef0, (0xbeef bytes) 2024-09-14T06:17:02.585 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:02.585 INFO:teuthology.orchestra.run.smithi039.stdout:3242 punch from 0x6cc9f7e to 0x6cd46e1, (0xa763 bytes) 2024-09-14T06:17:02.585 INFO:teuthology.orchestra.run.smithi039.stdout:3243 read 0x6540a12 thru 0x654e8c6 (0xdeb5 bytes) 2024-09-14T06:17:02.592 INFO:teuthology.orchestra.run.smithi039.stdout:3244 trunc from 0xac84a7b to 0x7d4f6df 2024-09-14T06:17:02.603 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:17:02.611 INFO:teuthology.orchestra.run.smithi039.stdout:3245 punch from 0xa7a7b0 to 0xa7addf, (0x62f bytes) 2024-09-14T06:17:02.611 INFO:teuthology.orchestra.run.smithi039.stdout:3246 read 0x1f533f8 thru 0x1f57319 (0x3f22 bytes) 2024-09-14T06:17:02.613 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:02.613 INFO:teuthology.orchestra.run.smithi039.stdout:3249 writesame 0x677c6dd thru 0x678388d data_size 0x1(0x71b1 bytes) 2024-09-14T06:17:02.639 INFO:teuthology.orchestra.run.smithi039.stdout:3251 writesame 0x6ccf727 thru 0x6cd788d data_size 0x1(0x8167 bytes) 2024-09-14T06:17:02.640 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:02.638+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:02.676 INFO:teuthology.orchestra.run.smithi039.stdout:3253 writesame 0x7666685 thru 0x7668613 data_size 0x1(0x1f8f bytes) 2024-09-14T06:17:02.690 INFO:teuthology.orchestra.run.smithi039.stdout:3256 trunc from 0x7d4f6df to 0x9c84796 2024-09-14T06:17:02.705 INFO:teuthology.orchestra.run.smithi039.stdout:3258 writesame 0x80733d4 thru 0x80795e9 data_size 0x5d(0x6216 bytes) 2024-09-14T06:17:02.708 INFO:teuthology.orchestra.run.smithi039.stdout:3259 write 0x384ba21 thru 0x384cc7b (0x125b bytes) 2024-09-14T06:17:02.713 INFO:teuthology.orchestra.run.smithi039.stdout:3260 read 0x9648d20 thru 0x96494ca (0x7ab bytes) 2024-09-14T06:17:02.713 INFO:teuthology.orchestra.run.smithi039.stdout:3261 writesame 0x3b9bb50 thru 0x3ba16c1 data_size 0x5(0x5b72 bytes) 2024-09-14T06:17:02.715 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:02.714+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:02.720 INFO:teuthology.orchestra.run.smithi039.stdout:3262 read 0x6e96369 thru 0x6e986be (0x2356 bytes) 2024-09-14T06:17:02.721 INFO:teuthology.orchestra.run.smithi039.stdout:3264 read 0x285d0f3 thru 0x2865f3d (0x8e4b bytes) 2024-09-14T06:17:02.722 INFO:teuthology.orchestra.run.smithi039.stdout:3265 read 0x6ef67e5 thru 0x6efca36 (0x6252 bytes) 2024-09-14T06:17:02.723 INFO:teuthology.orchestra.run.smithi039.stdout:3266 writesame 0x526a24a thru 0x526f670 data_size 0x2b(0x5427 bytes) 2024-09-14T06:17:02.727 INFO:teuthology.orchestra.run.smithi039.stdout:3267 read 0x4d68a5c thru 0x4d72d40 (0xa2e5 bytes) 2024-09-14T06:17:02.729 INFO:teuthology.orchestra.run.smithi039.stdout:3269 trunc from 0x9c84796 to 0x989e9e1 2024-09-14T06:17:02.735 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:02.736 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:02.747 INFO:teuthology.orchestra.run.smithi039.stdout:3270 punch from 0xcfe64d to 0xd00dd8, (0x278b bytes) 2024-09-14T06:17:02.747 INFO:teuthology.orchestra.run.smithi039.stdout:3271 write 0x8deb05e thru 0x8df7e2d (0xcdd0 bytes) 2024-09-14T06:17:02.751 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:17:02.761 INFO:teuthology.orchestra.run.smithi039.stdout:3273 trunc from 0x989e9e1 to 0xe0d34b8 2024-09-14T06:17:02.775 INFO:teuthology.orchestra.run.smithi039.stdout:3274 writesame 0xa34ce8 thru 0xa448ec data_size 0x1(0xfc05 bytes) 2024-09-14T06:17:02.816 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:02.814+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:02.856 INFO:teuthology.orchestra.run.smithi039.stdout:3275 punch from 0xa2ec96b to 0xa2f6e29, (0xa4be bytes) 2024-09-14T06:17:02.856 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:02.856 INFO:teuthology.orchestra.run.smithi039.stdout:3278 read 0x8490c7 thru 0x857566 (0xe4a0 bytes) 2024-09-14T06:17:02.857 INFO:teuthology.orchestra.run.smithi039.stdout:3279 writesame 0xb2c6740 thru 0xb2c77fe data_size 0x3(0x10bf bytes) 2024-09-14T06:17:02.860 INFO:teuthology.orchestra.run.smithi039.stdout:3280 write 0xd919ad1 thru 0xd91b5d2 (0x1b02 bytes) 2024-09-14T06:17:02.863 INFO:teuthology.orchestra.run.smithi039.stdout:3281 read 0x2cf29fb thru 0x2cf2a57 (0x5d bytes) 2024-09-14T06:17:02.864 INFO:teuthology.orchestra.run.smithi039.stdout:3282 read 0x37dab03 thru 0x37e9618 (0xeb16 bytes) 2024-09-14T06:17:02.871 INFO:teuthology.orchestra.run.smithi039.stdout:3284 trunc from 0xe0d34b8 to 0xa77290a 2024-09-14T06:17:02.889 INFO:teuthology.orchestra.run.smithi039.stdout:3286 read 0x99cf915 thru 0x99de082 (0xe76e bytes) 2024-09-14T06:17:02.896 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:02.897 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:02.907 INFO:teuthology.orchestra.run.smithi039.stdout:3288 writesame 0xb4c86ca thru 0xb4ced5e data_size 0x1(0x6695 bytes) 2024-09-14T06:17:02.910 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:02.913 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:17:02.917 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:02.914+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:02.930 INFO:teuthology.orchestra.run.smithi039.stdout:3290 writesame 0xe05e3ab thru 0xe060153 data_size 0x1(0x1da9 bytes) 2024-09-14T06:17:02.930 INFO:teuthology.orchestra.run.smithi039.stdout:3292 read 0x334eacc thru 0x335c34c (0xd881 bytes) 2024-09-14T06:17:02.931 INFO:teuthology.orchestra.run.smithi039.stdout:3293 clone 17 order 20 su 16384 sc 3 2024-09-14T06:17:03.018 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:03.014+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:03.057 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:03.057 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:03.072 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:17:03.119 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:03.118+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:03.201 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:03.201 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:03.219 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:17:03.263 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:03.262+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:03.321 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:03.318+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:03.397 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:03.397 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:03.413 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:17:03.423 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:03.422+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:03.523 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:03.522+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:03.533 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:17:03.549 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-14T06:17:03.571 INFO:teuthology.orchestra.run.smithi039.stdout:3294 write 0x22e4fa8 thru 0x22e8e8f (0x3ee8 bytes) 2024-09-14T06:17:03.589 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:03.589 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:03.611 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:17:03.614 INFO:teuthology.orchestra.run.smithi039.stdout:3295 write 0x3fbb593 thru 0x3fbc143 (0xbb1 bytes) 2024-09-14T06:17:03.625 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:03.622+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:03.631 INFO:teuthology.orchestra.run.smithi039.stdout:3296 punch from 0x541b641 to 0x5426389, (0xad48 bytes) 2024-09-14T06:17:03.650 INFO:teuthology.orchestra.run.smithi039.stdout:3297 trunc from 0xe060154 to 0xf6b474 2024-09-14T06:17:03.698 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:03.698 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:03.716 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:03.716 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:03.725 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:03.722+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:03.732 INFO:teuthology.orchestra.run.smithi039.stdout:3298 write 0xda75444 thru 0xda7fb86 (0xa743 bytes) 2024-09-14T06:17:03.735 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:03.740 INFO:teuthology.orchestra.run.smithi039.stdout:3299 read 0x8e39317 thru 0x8e410fa (0x7de4 bytes) 2024-09-14T06:17:03.740 INFO:teuthology.orchestra.run.smithi039.stdout:3301 read 0xa38e3ad thru 0xa390d94 (0x29e8 bytes) 2024-09-14T06:17:03.740 INFO:teuthology.orchestra.run.smithi039.stdout:3304 punch from 0x629502c to 0x629fbbb, (0xab8f bytes) 2024-09-14T06:17:03.740 INFO:teuthology.orchestra.run.smithi039.stdout:3307 write 0x4f5bb42 thru 0x4f60073 (0x4532 bytes) 2024-09-14T06:17:03.746 INFO:teuthology.orchestra.run.smithi039.stdout:3308 writesame 0x1418815 thru 0x141ad3c data_size 0x1d(0x2528 bytes) 2024-09-14T06:17:03.749 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:03.749 INFO:teuthology.orchestra.run.smithi039.stdout:3312 write 0x85693bf thru 0x8574374 (0xafb6 bytes) 2024-09-14T06:17:03.761 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:03.762 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:03.762 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:03.762 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:03.762 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:03.763 INFO:teuthology.orchestra.run.smithi039.stdout:3314 trunc from 0xda7fb87 to 0x8d16021 2024-09-14T06:17:03.779 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:17:03.781 INFO:teuthology.orchestra.run.smithi039.stdout:3315 read 0x7c6b2d8 thru 0x7c76ec1 (0xbbea bytes) 2024-09-14T06:17:03.781 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:03.781 INFO:teuthology.orchestra.run.smithi039.stdout:3318 writesame 0x1545bb3 thru 0x1550f99 data_size 0x3d(0xb3e7 bytes) 2024-09-14T06:17:03.784 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:03.784 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:03.789 INFO:teuthology.orchestra.run.smithi039.stdout:3319 trunc from 0x8d16021 to 0x969f686 2024-09-14T06:17:03.800 INFO:teuthology.orchestra.run.smithi039.stdout:3320 write 0x1374278 thru 0x137f59f (0xb328 bytes) 2024-09-14T06:17:03.806 INFO:teuthology.orchestra.run.smithi039.stdout:3322 read 0x3910443 thru 0x391fbeb (0xf7a9 bytes) 2024-09-14T06:17:03.806 INFO:teuthology.orchestra.run.smithi039.stdout:3323 read 0x951d957 thru 0x952d40d (0xfab7 bytes) 2024-09-14T06:17:03.806 INFO:teuthology.orchestra.run.smithi039.stdout:3324 read 0x38359f2 thru 0x383a89f (0x4eae bytes) 2024-09-14T06:17:03.809 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:03.809 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:03.809 INFO:teuthology.orchestra.run.smithi039.stdout:3325 trunc from 0x969f686 to 0x7524c4f 2024-09-14T06:17:03.825 INFO:teuthology.orchestra.run.smithi039.stdout:3326 write 0x299f911 thru 0x29a046e (0xb5e bytes) 2024-09-14T06:17:03.826 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:03.822+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:03.831 INFO:teuthology.orchestra.run.smithi039.stdout:3327 read 0x187b94a thru 0x1886ec8 (0xb57f bytes) 2024-09-14T06:17:03.831 INFO:teuthology.orchestra.run.smithi039.stdout:3328 punch from 0x58b951f to 0x58be2f6, (0x4dd7 bytes) 2024-09-14T06:17:03.832 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:03.832 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:03.851 INFO:teuthology.orchestra.run.smithi039.stdout:3329 write 0xe08d47c thru 0xe08d967 (0x4ec bytes) 2024-09-14T06:17:03.855 INFO:teuthology.orchestra.run.smithi039.stdout:3332 read 0x342f0f thru 0x34fd61 (0xce53 bytes) 2024-09-14T06:17:03.859 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:03.859 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:03.862 INFO:teuthology.orchestra.run.smithi039.stdout:3333 write 0x8df2704 thru 0x8dfdd69 (0xb666 bytes) 2024-09-14T06:17:03.870 INFO:teuthology.orchestra.run.smithi039.stdout:3334 writesame 0xd8eedd5 thru 0xd8f2d02 data_size 0x1(0x3f2e bytes) 2024-09-14T06:17:03.870 INFO:teuthology.orchestra.run.smithi039.stdout:3337 punch from 0x9f0f36d to 0x9f125e7, (0x327a bytes) 2024-09-14T06:17:03.870 INFO:teuthology.orchestra.run.smithi039.stdout:3340 writesame 0x6868cd5 thru 0x68707b3 data_size 0x2d(0x7adf bytes) 2024-09-14T06:17:03.912 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:03.912 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:03.912 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:03.912 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:03.912 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:03.926 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:17:03.927 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:03.926+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:03.970 INFO:teuthology.orchestra.run.smithi039.stdout:3341 write 0xd16b55c thru 0xd17749f (0xbf44 bytes) 2024-09-14T06:17:03.978 INFO:teuthology.orchestra.run.smithi039.stdout:3342 trunc from 0xe08d968 to 0xdeaae86 2024-09-14T06:17:03.989 INFO:teuthology.orchestra.run.smithi039.stdout:3343 read 0xd0418be thru 0xd0491c8 (0x790b bytes) 2024-09-14T06:17:03.990 INFO:teuthology.orchestra.run.smithi039.stdout:3344 punch from 0x461d28b to 0x462cb03, (0xf878 bytes) 2024-09-14T06:17:04.002 INFO:teuthology.orchestra.run.smithi039.stdout:3345 write 0xedf792e thru 0xedff297 (0x796a bytes) 2024-09-14T06:17:04.011 INFO:teuthology.orchestra.run.smithi039.stdout:3346 write 0xb76be31 thru 0xb7773cf (0xb59f bytes) 2024-09-14T06:17:04.018 INFO:teuthology.orchestra.run.smithi039.stdout:3347 punch from 0xb61a51a to 0xb61c64d, (0x2133 bytes) 2024-09-14T06:17:04.018 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:04.019 INFO:teuthology.orchestra.run.smithi039.stdout:3350 read 0x64a71ce thru 0x64b1c32 (0xaa65 bytes) 2024-09-14T06:17:04.019 INFO:teuthology.orchestra.run.smithi039.stdout:3355 read 0x841a6df thru 0x841f69f (0x4fc1 bytes) 2024-09-14T06:17:04.019 INFO:teuthology.orchestra.run.smithi039.stdout:3356 clone 18 order 18 su 4096 sc 2 2024-09-14T06:17:04.027 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:04.026+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:04.095 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:04.095 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:04.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:04.095 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:04.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.096 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11217 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 925696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.096 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:17:04.096 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.101284475, 2024-09-14T06:17:04.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049306313999999997, 2024-09-14T06:17:04.096 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.096 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.096 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.096 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.096 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.096 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.096 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11217 2024-09-14T06:17:04.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.096 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.097 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.097 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.097 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:17:04.097 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.097 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.097 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.097 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:17:04.097 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.097 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.097 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.097 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:17:04.097 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.29e-07 2024-09-14T06:17:04.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.098 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.098 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.098 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991345+0000", 2024-09-14T06:17:04.098 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1480000000000001e-06 2024-09-14T06:17:04.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.098 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.098 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.098 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991349+0000", 2024-09-14T06:17:04.098 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9840000000000003e-06 2024-09-14T06:17:04.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991359+0000", 2024-09-14T06:17:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4050000000000006e-06 2024-09-14T06:17:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.030826+0000", 2024-09-14T06:17:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.039467214 2024-09-14T06:17:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.030894+0000", 2024-09-14T06:17:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8400999999999998e-05 2024-09-14T06:17:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040650+0000", 2024-09-14T06:17:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0097559329999999996 2024-09-14T06:17:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11262 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 81920~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:17:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.101098439000001, 2024-09-14T06:17:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049588879000000002, 2024-09-14T06:17:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11262 2024-09-14T06:17:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:17:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:17:04.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:17:04.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9599999999999999e-07 2024-09-14T06:17:04.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991533+0000", 2024-09-14T06:17:04.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.362e-06 2024-09-14T06:17:04.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.103 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.103 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991539+0000", 2024-09-14T06:17:04.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:17:04.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.103 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.103 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991544+0000", 2024-09-14T06:17:04.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:17:04.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.103 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.103 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040755+0000", 2024-09-14T06:17:04.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049210938000000003 2024-09-14T06:17:04.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040833+0000", 2024-09-14T06:17:04.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7812999999999995e-05 2024-09-14T06:17:04.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041119+0000", 2024-09-14T06:17:04.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0002855 2024-09-14T06:17:04.104 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.104 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.104 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.105 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11263 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.105 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:04.105 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.101075660999999, 2024-09-14T06:17:04.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049944574999999998, 2024-09-14T06:17:04.105 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.105 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.105 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.105 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.105 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.105 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.105 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11263 2024-09-14T06:17:04.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.105 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.106 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.106 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:04.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.106 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.106 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:04.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:04.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.51e-07 2024-09-14T06:17:04.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991560+0000", 2024-09-14T06:17:04.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6719999999999998e-06 2024-09-14T06:17:04.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991567+0000", 2024-09-14T06:17:04.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1470000000000004e-06 2024-09-14T06:17:04.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991572+0000", 2024-09-14T06:17:04.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.237e-06 2024-09-14T06:17:04.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041149+0000", 2024-09-14T06:17:04.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049576988000000002 2024-09-14T06:17:04.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041277+0000", 2024-09-14T06:17:04.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000127687 2024-09-14T06:17:04.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041497+0000", 2024-09-14T06:17:04.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022039299999999999 2024-09-14T06:17:04.109 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.110 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.110 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.110 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11264 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 327680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.110 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:04.110 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.101049320000001, 2024-09-14T06:17:04.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050248567000000001, 2024-09-14T06:17:04.110 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.111 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.111 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.111 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.111 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.111 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.111 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11264 2024-09-14T06:17:04.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.111 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.111 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:04.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.111 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.112 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:04.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.112 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.112 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:04.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.41e-07 2024-09-14T06:17:04.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.112 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.112 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991581+0000", 2024-09-14T06:17:04.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3740000000000001e-06 2024-09-14T06:17:04.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.112 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991585+0000", 2024-09-14T06:17:04.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9400000000000004e-06 2024-09-14T06:17:04.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991587+0000", 2024-09-14T06:17:04.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.694e-06 2024-09-14T06:17:04.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041527+0000", 2024-09-14T06:17:04.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049939349000000001 2024-09-14T06:17:04.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041598+0000", 2024-09-14T06:17:04.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1633999999999999e-05 2024-09-14T06:17:04.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041827+0000", 2024-09-14T06:17:04.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000229235 2024-09-14T06:17:04.114 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.114 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.114 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.114 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.115 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:04.115 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.101035979999999, 2024-09-14T06:17:04.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:17:04.115 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.115 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.115 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.115 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.115 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.115 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.115 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:17:04.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.115 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:04.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.116 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:04.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.116 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:04.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:17:04.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.116 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:17:04.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:17:04.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.117 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.117 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:17:04.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:17:04.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.117 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.117 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:17:04.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:17:04.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.118 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.118 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:17:04.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:17:04.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.118 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.118 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:17:04.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:17:04.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.118 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.118 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:17:04.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:17:04.118 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.118 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.118 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.119 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.119 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:04.119 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.101022908000001, 2024-09-14T06:17:04.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:17:04.119 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.119 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.119 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.119 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.119 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.120 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.120 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:17:04.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.120 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:04.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:04.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.121 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.121 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:17:04.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:17:04.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.121 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.121 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:17:04.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:17:04.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.121 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.121 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:17:04.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:17:04.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.122 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.122 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:17:04.122 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:17:04.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.122 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.122 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:17:04.122 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:17:04.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.122 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.122 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:17:04.122 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:17:04.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:17:04.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:17:04.123 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.123 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.123 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.123 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.123 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:04.123 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.101005476000001, 2024-09-14T06:17:04.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:17:04.123 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.123 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.124 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.124 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.124 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.124 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.124 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:17:04.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.124 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:04.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:04.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:04.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:17:04.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:17:04.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:17:04.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:17:04.126 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:17:04.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.126 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.126 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.126 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:17:04.126 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:17:04.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.126 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.126 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.126 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:17:04.126 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:17:04.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.126 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.126 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:17:04.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:17:04.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.127 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:17:04.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:17:04.127 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.127 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.127 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.127 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.127 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:04.127 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.100984370999999, 2024-09-14T06:17:04.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:17:04.128 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.128 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.128 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.128 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.128 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.128 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.128 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:17:04.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.128 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.128 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.128 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:04.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:04.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:04.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:17:04.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.130 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:17:04.130 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:17:04.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.130 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.130 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:17:04.130 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:17:04.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.131 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.131 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:17:04.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:17:04.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.131 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.131 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.131 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:17:04.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:17:04.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.131 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.131 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.132 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:17:04.132 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:17:04.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.132 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.132 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:17:04.132 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:17:04.132 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.132 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.132 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.132 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.132 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:04.132 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.100963263000001, 2024-09-14T06:17:04.133 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:17:04.133 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.133 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.133 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.133 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.133 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.133 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.133 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:17:04.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.133 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.133 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.133 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:04.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:04.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:04.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:17:04.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:17:04.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:17:04.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:17:04.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:17:04.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:17:04.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:17:04.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.136 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:17:04.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:17:04.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.136 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:17:04.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:17:04.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.137 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.137 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:17:04.137 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:17:04.137 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.137 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.137 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.137 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13590 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 491520~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.137 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:17:04.137 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.954988381, 2024-09-14T06:17:04.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042553466999999998, 2024-09-14T06:17:04.138 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.138 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.138 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.138 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.138 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.138 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.138 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13590 2024-09-14T06:17:04.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.138 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.138 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.138 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:17:04.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:17:04.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:17:04.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.34e-07 2024-09-14T06:17:04.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137642+0000", 2024-09-14T06:17:04.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0150000000000002e-06 2024-09-14T06:17:04.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137649+0000", 2024-09-14T06:17:04.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4010000000000002e-06 2024-09-14T06:17:04.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137653+0000", 2024-09-14T06:17:04.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:17:04.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.141 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.141 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.179985+0000", 2024-09-14T06:17:04.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042331580000000001 2024-09-14T06:17:04.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.141 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.141 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180076+0000", 2024-09-14T06:17:04.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0796000000000003e-05 2024-09-14T06:17:04.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.141 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180193+0000", 2024-09-14T06:17:04.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000117471 2024-09-14T06:17:04.142 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.142 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.142 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.142 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13592 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 606208~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.142 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:17:04.142 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.954963812999999, 2024-09-14T06:17:04.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042709440000000001, 2024-09-14T06:17:04.143 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.143 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.143 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.143 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.143 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.143 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.143 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13592 2024-09-14T06:17:04.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.143 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.143 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:17:04.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.144 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:17:04.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.144 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137665+0000", 2024-09-14T06:17:04.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4299999999999998e-07 2024-09-14T06:17:04.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.144 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137667+0000", 2024-09-14T06:17:04.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5780000000000001e-06 2024-09-14T06:17:04.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137675+0000", 2024-09-14T06:17:04.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0870000000000001e-06 2024-09-14T06:17:04.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137680+0000", 2024-09-14T06:17:04.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4999999999999999e-06 2024-09-14T06:17:04.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180223+0000", 2024-09-14T06:17:04.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042542808000000001 2024-09-14T06:17:04.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180294+0000", 2024-09-14T06:17:04.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1497999999999999e-05 2024-09-14T06:17:04.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180374+0000", 2024-09-14T06:17:04.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9525999999999997e-05 2024-09-14T06:17:04.147 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.147 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.148 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.148 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13594 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 729088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.148 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:17:04.148 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.954940249, 2024-09-14T06:17:04.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042830604000000001, 2024-09-14T06:17:04.149 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.149 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.149 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.149 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.149 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.149 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.150 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13594 2024-09-14T06:17:04.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.150 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.150 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:17:04.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.151 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:17:04.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:17:04.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0699999999999998e-07 2024-09-14T06:17:04.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137690+0000", 2024-09-14T06:17:04.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.012e-06 2024-09-14T06:17:04.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137698+0000", 2024-09-14T06:17:04.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4379999999999998e-06 2024-09-14T06:17:04.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137703+0000", 2024-09-14T06:17:04.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1250000000000001e-06 2024-09-14T06:17:04.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180404+0000", 2024-09-14T06:17:04.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042701465000000001 2024-09-14T06:17:04.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180456+0000", 2024-09-14T06:17:04.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1227999999999998e-05 2024-09-14T06:17:04.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180519+0000", 2024-09-14T06:17:04.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-09-14T06:17:04.156 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.156 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.156 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.156 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13595 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 851968~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.156 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:17:04.156 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.954917743999999, 2024-09-14T06:17:04.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043012433000000003, 2024-09-14T06:17:04.156 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.156 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.156 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.156 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.156 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.157 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.157 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13595 2024-09-14T06:17:04.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.157 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:17:04.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:17:04.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137711+0000", 2024-09-14T06:17:04.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:17:04.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137713+0000", 2024-09-14T06:17:04.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5170000000000002e-06 2024-09-14T06:17:04.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137720+0000", 2024-09-14T06:17:04.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-09-14T06:17:04.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137726+0000", 2024-09-14T06:17:04.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9580000000000004e-06 2024-09-14T06:17:04.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180551+0000", 2024-09-14T06:17:04.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042825611999999999 2024-09-14T06:17:04.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180623+0000", 2024-09-14T06:17:04.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1673000000000003e-05 2024-09-14T06:17:04.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180723+0000", 2024-09-14T06:17:04.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.9928999999999998e-05 2024-09-14T06:17:04.160 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.160 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.160 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.160 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13597 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 974848~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.160 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:17:04.160 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.954894858999999, 2024-09-14T06:17:04.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043180442999999999, 2024-09-14T06:17:04.161 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.161 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.161 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.161 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.161 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.161 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.161 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13597 2024-09-14T06:17:04.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.162 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:17:04.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:17:04.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137734+0000", 2024-09-14T06:17:04.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4499999999999997e-07 2024-09-14T06:17:04.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137736+0000", 2024-09-14T06:17:04.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0470000000000001e-06 2024-09-14T06:17:04.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137742+0000", 2024-09-14T06:17:04.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2269999999999998e-06 2024-09-14T06:17:04.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137747+0000", 2024-09-14T06:17:04.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6670000000000004e-06 2024-09-14T06:17:04.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180753+0000", 2024-09-14T06:17:04.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043006132000000002 2024-09-14T06:17:04.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180820+0000", 2024-09-14T06:17:04.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7516999999999997e-05 2024-09-14T06:17:04.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180914+0000", 2024-09-14T06:17:04.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3407999999999997e-05 2024-09-14T06:17:04.165 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.165 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.165 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.166 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13684 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 999424~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.166 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:17:04.166 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.952662316000001, 2024-09-14T06:17:04.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042643702999999998, 2024-09-14T06:17:04.166 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.166 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.166 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.166 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.166 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.166 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.166 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13684 2024-09-14T06:17:04.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.166 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:17:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:17:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139967+0000", 2024-09-14T06:17:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1200000000000003e-07 2024-09-14T06:17:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141080+0000", 2024-09-14T06:17:04.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011133029999999999 2024-09-14T06:17:04.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141084+0000", 2024-09-14T06:17:04.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.544e-06 2024-09-14T06:17:04.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141088+0000", 2024-09-14T06:17:04.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0280000000000002e-06 2024-09-14T06:17:04.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.169 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.169 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182457+0000", 2024-09-14T06:17:04.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041368690999999999 2024-09-14T06:17:04.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.169 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.169 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182528+0000", 2024-09-14T06:17:04.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0881000000000001e-05 2024-09-14T06:17:04.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.169 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.169 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182610+0000", 2024-09-14T06:17:04.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1644000000000006e-05 2024-09-14T06:17:04.170 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.170 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.170 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.170 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13721 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 532480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.170 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:17:04.170 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.951361582000001, 2024-09-14T06:17:04.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042577526999999997, 2024-09-14T06:17:04.170 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.170 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.170 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.170 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.170 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.170 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.171 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13721 2024-09-14T06:17:04.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.171 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:17:04.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:17:04.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:17:04.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2999999999999999e-07 2024-09-14T06:17:04.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141268+0000", 2024-09-14T06:17:04.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0130000000000001e-06 2024-09-14T06:17:04.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141271+0000", 2024-09-14T06:17:04.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0970000000000002e-06 2024-09-14T06:17:04.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.173 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.173 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141274+0000", 2024-09-14T06:17:04.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5809999999999999e-06 2024-09-14T06:17:04.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.173 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.173 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183522+0000", 2024-09-14T06:17:04.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042248591000000002 2024-09-14T06:17:04.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.174 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183691+0000", 2024-09-14T06:17:04.174 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016885399999999999 2024-09-14T06:17:04.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.174 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183844+0000", 2024-09-14T06:17:04.174 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015316099999999999 2024-09-14T06:17:04.174 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.174 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.174 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.175 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13722 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 655360~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.175 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:17:04.175 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.951351280000001, 2024-09-14T06:17:04.175 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042741610999999999, 2024-09-14T06:17:04.175 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.175 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.175 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.175 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.175 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.175 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.175 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13722 2024-09-14T06:17:04.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.175 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.175 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:17:04.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:17:04.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:17:04.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:17:04.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.177 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141278+0000", 2024-09-14T06:17:04.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.066e-06 2024-09-14T06:17:04.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.177 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.177 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.177 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141282+0000", 2024-09-14T06:17:04.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7100000000000001e-06 2024-09-14T06:17:04.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.177 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.177 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.177 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141285+0000", 2024-09-14T06:17:04.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.554e-06 2024-09-14T06:17:04.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.178 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.178 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183824+0000", 2024-09-14T06:17:04.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042539043999999998 2024-09-14T06:17:04.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.178 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.178 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183906+0000", 2024-09-14T06:17:04.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2298000000000004e-05 2024-09-14T06:17:04.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.178 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.178 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184019+0000", 2024-09-14T06:17:04.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000112724 2024-09-14T06:17:04.178 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.178 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.179 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.179 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13723 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 778240~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.179 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:17:04.179 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.951340310999999, 2024-09-14T06:17:04.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042995901000000003, 2024-09-14T06:17:04.179 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.179 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.179 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.179 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.179 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.179 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.179 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13723 2024-09-14T06:17:04.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.180 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:17:04.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:17:04.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.181 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.181 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:17:04.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9799999999999999e-07 2024-09-14T06:17:04.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.181 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.181 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.181 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141289+0000", 2024-09-14T06:17:04.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0589999999999999e-06 2024-09-14T06:17:04.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.181 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.181 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.181 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141292+0000", 2024-09-14T06:17:04.182 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.208e-06 2024-09-14T06:17:04.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.182 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.182 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.182 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141295+0000", 2024-09-14T06:17:04.182 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5160000000000001e-06 2024-09-14T06:17:04.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.182 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.182 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.182 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184069+0000", 2024-09-14T06:17:04.182 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042774176999999997 2024-09-14T06:17:04.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.183 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.183 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.183 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184169+0000", 2024-09-14T06:17:04.183 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000100073 2024-09-14T06:17:04.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.183 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.183 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.183 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184284+0000", 2024-09-14T06:17:04.183 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011457 2024-09-14T06:17:04.183 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.183 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.183 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.184 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13724 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 901120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.184 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:17:04.184 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.951329916999999, 2024-09-14T06:17:04.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043197179000000002, 2024-09-14T06:17:04.184 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.184 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.184 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.184 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.185 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.185 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.185 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13724 2024-09-14T06:17:04.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.185 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.185 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:17:04.185 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.185 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:17:04.185 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.186 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141299+0000", 2024-09-14T06:17:04.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.03e-07 2024-09-14T06:17:04.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.186 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141300+0000", 2024-09-14T06:17:04.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0389999999999999e-06 2024-09-14T06:17:04.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.186 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141303+0000", 2024-09-14T06:17:04.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0479999999999999e-06 2024-09-14T06:17:04.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141305+0000", 2024-09-14T06:17:04.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5739999999999998e-06 2024-09-14T06:17:04.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184317+0000", 2024-09-14T06:17:04.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043011701999999999 2024-09-14T06:17:04.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184394+0000", 2024-09-14T06:17:04.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7324999999999996e-05 2024-09-14T06:17:04.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.188 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184496+0000", 2024-09-14T06:17:04.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010118799999999999 2024-09-14T06:17:04.188 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.188 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.188 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.188 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13726 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 1024000~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:04.188 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:17:04.188 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.951319644000002, 2024-09-14T06:17:04.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043397185999999997, 2024-09-14T06:17:04.189 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:04.189 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:04.189 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:04.189 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:04.189 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:04.189 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.189 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13726 2024-09-14T06:17:04.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.189 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:04.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.189 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:04.190 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:17:04.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.190 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:04.190 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:17:04.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:04.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.190 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:04.190 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:17:04.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:17:04.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.190 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:04.190 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141310+0000", 2024-09-14T06:17:04.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:17:04.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:04.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141313+0000", 2024-09-14T06:17:04.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0309999999999999e-06 2024-09-14T06:17:04.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141315+0000", 2024-09-14T06:17:04.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.52e-06 2024-09-14T06:17:04.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:04.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184527+0000", 2024-09-14T06:17:04.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043211855 2024-09-14T06:17:04.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:04.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184606+0000", 2024-09-14T06:17:04.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8287000000000004e-05 2024-09-14T06:17:04.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:04.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184706+0000", 2024-09-14T06:17:04.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010021700000000001 2024-09-14T06:17:04.192 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.192 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.192 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.192 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.192 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.192 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:04.193 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:17:04.193 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:17:04.193 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T06:17:04.194 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T06:17:04.195 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:17:04.195 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:04.190+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:04.213 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:17:04.295 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:04.294+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:04.315 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone17 from 0xedff298 (overlap 0xf6b474) to 0xacb184 2024-09-14T06:17:04.356 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:04.356 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:04.356 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.356 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32297 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.356 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.330182+0000", 2024-09-14T06:17:04.356 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0077149660000000002, 2024-09-14T06:17:04.356 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0077311580000000001, 2024-09-14T06:17:04.357 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.357 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.357 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:04.357 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.357 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.357 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.357 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32297 2024-09-14T06:17:04.357 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.357 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.357 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.357 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.358 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.330182+0000", 2024-09-14T06:17:04.358 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.358 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.358 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.358 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.358 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.330182+0000", 2024-09-14T06:17:04.359 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.359 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.359 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.359 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.359 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.330182+0000", 2024-09-14T06:17:04.359 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1800000000000002e-07 2024-09-14T06:17:04.360 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.360 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.360 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.360 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.330183+0000", 2024-09-14T06:17:04.360 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.248e-06 2024-09-14T06:17:04.360 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.360 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.361 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.361 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.330191+0000", 2024-09-14T06:17:04.361 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.515e-06 2024-09-14T06:17:04.361 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.361 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.361 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.361 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.330194+0000", 2024-09-14T06:17:04.362 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3060000000000002e-06 2024-09-14T06:17:04.362 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.362 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.362 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:04.362 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337891+0000", 2024-09-14T06:17:04.362 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0076964590000000001 2024-09-14T06:17:04.362 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.362 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.363 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:04.363 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337913+0000", 2024-09-14T06:17:04.363 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2311999999999999e-05 2024-09-14T06:17:04.363 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.363 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.363 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.363 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.364 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.364 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32200 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.364 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.331691+0000", 2024-09-14T06:17:04.364 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0062054279999999998, 2024-09-14T06:17:04.364 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.006376713, 2024-09-14T06:17:04.364 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32200 2024-09-14T06:17:04.366 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.366 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.366 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.366 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.366 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.331691+0000", 2024-09-14T06:17:04.366 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.366 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.366 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.366 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.366 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.331691+0000", 2024-09-14T06:17:04.367 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.367 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.367 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.367 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.367 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.331692+0000", 2024-09-14T06:17:04.367 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7099999999999997e-07 2024-09-14T06:17:04.367 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.367 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.367 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.367 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.331693+0000", 2024-09-14T06:17:04.367 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.542e-06 2024-09-14T06:17:04.367 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.367 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.367 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.367 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.331703+0000", 2024-09-14T06:17:04.368 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0166000000000001e-05 2024-09-14T06:17:04.368 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.368 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.368 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.368 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.331707+0000", 2024-09-14T06:17:04.368 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0230000000000003e-06 2024-09-14T06:17:04.368 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.368 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.368 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.368 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.369 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.369 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32232 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.369 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.332529+0000", 2024-09-14T06:17:04.369 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0053677689999999997, 2024-09-14T06:17:04.369 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056168499999999996, 2024-09-14T06:17:04.369 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.369 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.369 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.369 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.369 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.369 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.369 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32232 2024-09-14T06:17:04.370 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.370 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.370 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.370 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.370 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332529+0000", 2024-09-14T06:17:04.370 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.370 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.370 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.370 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.370 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332529+0000", 2024-09-14T06:17:04.370 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.370 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.370 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.371 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.371 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332529+0000", 2024-09-14T06:17:04.371 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2599999999999998e-07 2024-09-14T06:17:04.371 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.371 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.371 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.371 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332531+0000", 2024-09-14T06:17:04.371 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0880000000000002e-06 2024-09-14T06:17:04.371 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.371 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.371 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.371 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332537+0000", 2024-09-14T06:17:04.371 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2020000000000003e-06 2024-09-14T06:17:04.371 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.371 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.372 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.372 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332540+0000", 2024-09-14T06:17:04.372 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.241e-06 2024-09-14T06:17:04.372 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.372 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.372 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.372 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.372 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.372 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32264 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.372 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.333448+0000", 2024-09-14T06:17:04.372 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0044489789999999996, 2024-09-14T06:17:04.372 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047716909999999998, 2024-09-14T06:17:04.372 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.372 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.372 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.372 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.373 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.373 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.373 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32264 2024-09-14T06:17:04.373 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.373 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.373 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.373 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.373 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333448+0000", 2024-09-14T06:17:04.373 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.373 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.373 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.373 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.373 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333448+0000", 2024-09-14T06:17:04.373 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.373 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.374 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.374 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.374 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333448+0000", 2024-09-14T06:17:04.374 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0199999999999998e-07 2024-09-14T06:17:04.374 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.374 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.374 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.374 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333453+0000", 2024-09-14T06:17:04.374 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3179999999999997e-06 2024-09-14T06:17:04.374 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.374 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.374 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.374 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333458+0000", 2024-09-14T06:17:04.375 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6099999999999999e-06 2024-09-14T06:17:04.375 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.375 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.375 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.375 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333462+0000", 2024-09-14T06:17:04.375 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5880000000000002e-06 2024-09-14T06:17:04.375 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.375 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.375 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.375 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.375 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.375 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32296 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.375 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.334406+0000", 2024-09-14T06:17:04.375 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0034904979999999999, 2024-09-14T06:17:04.375 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00388733, 2024-09-14T06:17:04.376 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.376 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.376 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.376 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.376 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.376 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.376 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32296 2024-09-14T06:17:04.376 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.376 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.376 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.376 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.376 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334406+0000", 2024-09-14T06:17:04.376 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.377 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.377 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.377 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.377 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334406+0000", 2024-09-14T06:17:04.377 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.377 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.377 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.377 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.377 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334407+0000", 2024-09-14T06:17:04.377 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2300000000000002e-07 2024-09-14T06:17:04.377 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.377 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.377 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.377 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334408+0000", 2024-09-14T06:17:04.378 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.094e-06 2024-09-14T06:17:04.378 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.378 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.378 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.378 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334411+0000", 2024-09-14T06:17:04.378 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8089999999999999e-06 2024-09-14T06:17:04.378 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.378 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.378 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.378 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334418+0000", 2024-09-14T06:17:04.378 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2979999999999997e-06 2024-09-14T06:17:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32199 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.335925+0000", 2024-09-14T06:17:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0019719630000000002, 2024-09-14T06:17:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.002445408, 2024-09-14T06:17:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32199 2024-09-14T06:17:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.335925+0000", 2024-09-14T06:17:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.335925+0000", 2024-09-14T06:17:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.335925+0000", 2024-09-14T06:17:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5099999999999999e-07 2024-09-14T06:17:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.335937+0000", 2024-09-14T06:17:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1182999999999999e-05 2024-09-14T06:17:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.335949+0000", 2024-09-14T06:17:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2456000000000001e-05 2024-09-14T06:17:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.335956+0000", 2024-09-14T06:17:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6919999999999996e-06 2024-09-14T06:17:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32231 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336820+0000", 2024-09-14T06:17:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0010766809999999999, 2024-09-14T06:17:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0016220589999999999, 2024-09-14T06:17:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32231 2024-09-14T06:17:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336820+0000", 2024-09-14T06:17:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336820+0000", 2024-09-14T06:17:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336820+0000", 2024-09-14T06:17:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:17:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336822+0000", 2024-09-14T06:17:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0640000000000001e-06 2024-09-14T06:17:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336829+0000", 2024-09-14T06:17:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.421e-06 2024-09-14T06:17:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336836+0000", 2024-09-14T06:17:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.939e-06 2024-09-14T06:17:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32263 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337748+0000", 2024-09-14T06:17:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00014822500000000001, 2024-09-14T06:17:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00076589699999999998, 2024-09-14T06:17:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32263 2024-09-14T06:17:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337748+0000", 2024-09-14T06:17:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337748+0000", 2024-09-14T06:17:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337749+0000", 2024-09-14T06:17:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1100000000000002e-07 2024-09-14T06:17:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337750+0000", 2024-09-14T06:17:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.443e-06 2024-09-14T06:17:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337755+0000", 2024-09-14T06:17:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2650000000000004e-06 2024-09-14T06:17:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337770+0000", 2024-09-14T06:17:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4731e-05 2024-09-14T06:17:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32201 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.331712+0000", 2024-09-14T06:17:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0061843489999999996, 2024-09-14T06:17:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0068929129999999996, 2024-09-14T06:17:04.389 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.389 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.389 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.389 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.389 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.389 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.389 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32201 2024-09-14T06:17:04.389 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.389 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.389 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.389 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.389 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.331712+0000", 2024-09-14T06:17:04.389 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.389 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.389 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.390 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.331712+0000", 2024-09-14T06:17:04.390 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.390 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.390 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.390 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.331713+0000", 2024-09-14T06:17:04.390 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.53e-07 2024-09-14T06:17:04.390 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.390 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.390 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.331715+0000", 2024-09-14T06:17:04.390 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8649999999999999e-06 2024-09-14T06:17:04.390 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.390 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.391 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.331723+0000", 2024-09-14T06:17:04.391 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0539999999999995e-06 2024-09-14T06:17:04.391 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.391 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.391 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.391 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.331726+0000", 2024-09-14T06:17:04.391 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2349999999999999e-06 2024-09-14T06:17:04.391 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.391 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.391 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.391 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.391 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.391 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32233 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.391 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.332548+0000", 2024-09-14T06:17:04.392 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0053482779999999997, 2024-09-14T06:17:04.392 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0062170380000000003, 2024-09-14T06:17:04.392 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.392 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.392 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.392 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.392 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.392 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.392 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32233 2024-09-14T06:17:04.392 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.392 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.392 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.392 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.392 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332548+0000", 2024-09-14T06:17:04.393 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.393 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.393 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332548+0000", 2024-09-14T06:17:04.393 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.393 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.393 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332549+0000", 2024-09-14T06:17:04.393 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8700000000000002e-07 2024-09-14T06:17:04.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.394 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.394 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332550+0000", 2024-09-14T06:17:04.394 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2780000000000001e-06 2024-09-14T06:17:04.394 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.394 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.394 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.394 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332555+0000", 2024-09-14T06:17:04.394 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7310000000000001e-06 2024-09-14T06:17:04.394 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.394 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.394 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.394 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332558+0000", 2024-09-14T06:17:04.394 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0589999999999998e-06 2024-09-14T06:17:04.394 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.395 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.395 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.395 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.395 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.395 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32265 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.395 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.333488+0000", 2024-09-14T06:17:04.395 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0044089289999999998, 2024-09-14T06:17:04.395 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0054286020000000003, 2024-09-14T06:17:04.395 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.395 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.395 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.395 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.395 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.395 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.395 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32265 2024-09-14T06:17:04.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.396 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.396 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.396 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.396 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333488+0000", 2024-09-14T06:17:04.396 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.396 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.396 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.396 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333488+0000", 2024-09-14T06:17:04.396 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.396 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.396 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.397 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333489+0000", 2024-09-14T06:17:04.397 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5700000000000002e-07 2024-09-14T06:17:04.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.397 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.397 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333491+0000", 2024-09-14T06:17:04.397 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0329999999999998e-06 2024-09-14T06:17:04.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.397 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.397 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333499+0000", 2024-09-14T06:17:04.397 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.833e-06 2024-09-14T06:17:04.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.398 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.398 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333510+0000", 2024-09-14T06:17:04.398 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0363000000000001e-05 2024-09-14T06:17:04.398 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.398 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.398 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.398 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.398 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.398 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32297 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.398 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.334439+0000", 2024-09-14T06:17:04.398 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.003457987, 2024-09-14T06:17:04.398 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0046286799999999996, 2024-09-14T06:17:04.398 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.398 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.399 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.399 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.399 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.399 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.399 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32297 2024-09-14T06:17:04.399 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.399 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.399 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.399 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.399 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334439+0000", 2024-09-14T06:17:04.399 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.399 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.399 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.399 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.400 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334439+0000", 2024-09-14T06:17:04.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.400 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.400 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.400 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334439+0000", 2024-09-14T06:17:04.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1099999999999995e-07 2024-09-14T06:17:04.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.400 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.400 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.400 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334447+0000", 2024-09-14T06:17:04.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5020000000000003e-06 2024-09-14T06:17:04.401 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.401 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.401 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.401 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334453+0000", 2024-09-14T06:17:04.401 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.5520000000000001e-06 2024-09-14T06:17:04.401 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.401 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.401 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.401 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334458+0000", 2024-09-14T06:17:04.401 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:17:04.401 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.402 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.402 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.402 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.402 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.402 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32200 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.402 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.335965+0000", 2024-09-14T06:17:04.402 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.001931629, 2024-09-14T06:17:04.402 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003253415, 2024-09-14T06:17:04.402 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.402 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.402 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.402 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.402 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.402 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.402 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32200 2024-09-14T06:17:04.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.403 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.403 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.403 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.403 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.335965+0000", 2024-09-14T06:17:04.403 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.403 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.403 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.403 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.335965+0000", 2024-09-14T06:17:04.403 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.403 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.403 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.403 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.335965+0000", 2024-09-14T06:17:04.403 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:17:04.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.404 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.404 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.335968+0000", 2024-09-14T06:17:04.404 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3590000000000002e-06 2024-09-14T06:17:04.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.404 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.404 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.335974+0000", 2024-09-14T06:17:04.404 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.37e-06 2024-09-14T06:17:04.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.404 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.404 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.335986+0000", 2024-09-14T06:17:04.405 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2041e-05 2024-09-14T06:17:04.405 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.405 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.405 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.405 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.405 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.405 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32232 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.405 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336854+0000", 2024-09-14T06:17:04.405 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.001042521, 2024-09-14T06:17:04.405 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025156269999999999, 2024-09-14T06:17:04.405 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.405 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.405 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.405 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.405 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.405 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.406 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32232 2024-09-14T06:17:04.406 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.406 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.406 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.406 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.406 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336854+0000", 2024-09-14T06:17:04.406 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.406 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.406 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.406 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.406 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336854+0000", 2024-09-14T06:17:04.406 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.406 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.406 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.406 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.406 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336854+0000", 2024-09-14T06:17:04.407 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:17:04.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.407 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.407 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.407 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336856+0000", 2024-09-14T06:17:04.407 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9180000000000001e-06 2024-09-14T06:17:04.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.407 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.407 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.407 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336863+0000", 2024-09-14T06:17:04.407 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3219999999999998e-06 2024-09-14T06:17:04.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.407 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.407 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.408 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336866+0000", 2024-09-14T06:17:04.408 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1949999999999999e-06 2024-09-14T06:17:04.408 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.408 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.408 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.408 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.408 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.408 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32264 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.408 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337777+0000", 2024-09-14T06:17:04.408 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.000119938, 2024-09-14T06:17:04.408 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0017418539999999999, 2024-09-14T06:17:04.408 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.408 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.408 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.409 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.409 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.409 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.409 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32264 2024-09-14T06:17:04.409 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.409 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.409 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.409 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.409 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337777+0000", 2024-09-14T06:17:04.409 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.409 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.409 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.409 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.409 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337777+0000", 2024-09-14T06:17:04.409 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.410 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.410 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.410 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.410 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337777+0000", 2024-09-14T06:17:04.410 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6300000000000001e-07 2024-09-14T06:17:04.410 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.410 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.410 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.410 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337779+0000", 2024-09-14T06:17:04.410 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3969999999999999e-06 2024-09-14T06:17:04.410 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.410 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.410 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.410 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337784+0000", 2024-09-14T06:17:04.411 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0030000000000002e-06 2024-09-14T06:17:04.411 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.411 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.411 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.411 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337786+0000", 2024-09-14T06:17:04.411 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8770000000000002e-06 2024-09-14T06:17:04.411 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.411 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.411 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.411 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.411 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.411 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32234 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.411 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.332563+0000", 2024-09-14T06:17:04.411 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0053332689999999999, 2024-09-14T06:17:04.411 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0071082510000000003, 2024-09-14T06:17:04.412 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.412 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.412 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.412 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.412 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.412 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.412 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32234 2024-09-14T06:17:04.412 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.412 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.412 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.412 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.412 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332563+0000", 2024-09-14T06:17:04.412 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.412 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.413 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.413 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.413 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332563+0000", 2024-09-14T06:17:04.413 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.413 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.413 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.413 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.413 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332564+0000", 2024-09-14T06:17:04.413 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5499999999999999e-07 2024-09-14T06:17:04.413 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.413 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.413 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.413 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332565+0000", 2024-09-14T06:17:04.413 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.113e-06 2024-09-14T06:17:04.414 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.414 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.414 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.414 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332568+0000", 2024-09-14T06:17:04.414 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6689999999999999e-06 2024-09-14T06:17:04.414 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.414 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.414 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.414 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332574+0000", 2024-09-14T06:17:04.414 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2560000000000002e-06 2024-09-14T06:17:04.414 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.414 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.414 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.414 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.415 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.415 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32266 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.415 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.333527+0000", 2024-09-14T06:17:04.415 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0043699309999999996, 2024-09-14T06:17:04.415 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0062948229999999997, 2024-09-14T06:17:04.415 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.415 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.415 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.415 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.416 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.416 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.416 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32266 2024-09-14T06:17:04.416 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.416 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.416 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.416 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.416 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333527+0000", 2024-09-14T06:17:04.416 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.416 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.416 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.416 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.416 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333527+0000", 2024-09-14T06:17:04.417 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.417 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.417 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.417 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333527+0000", 2024-09-14T06:17:04.417 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5199999999999998e-07 2024-09-14T06:17:04.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.417 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.417 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.417 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333529+0000", 2024-09-14T06:17:04.418 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.403e-06 2024-09-14T06:17:04.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.418 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.418 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.418 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333534+0000", 2024-09-14T06:17:04.418 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1080000000000001e-06 2024-09-14T06:17:04.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.418 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.418 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.418 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333541+0000", 2024-09-14T06:17:04.418 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7360000000000001e-06 2024-09-14T06:17:04.418 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.418 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.419 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.419 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.419 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32298 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.419 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.334491+0000", 2024-09-14T06:17:04.419 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.003405609, 2024-09-14T06:17:04.419 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0054878310000000003, 2024-09-14T06:17:04.419 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.419 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.419 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.419 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.419 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.419 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.419 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32298 2024-09-14T06:17:04.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.420 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.420 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.420 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.420 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334491+0000", 2024-09-14T06:17:04.420 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.420 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.420 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.420 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.420 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334491+0000", 2024-09-14T06:17:04.420 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.420 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.420 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.420 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.421 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334491+0000", 2024-09-14T06:17:04.421 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.39e-07 2024-09-14T06:17:04.421 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.421 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.421 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.421 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334493+0000", 2024-09-14T06:17:04.421 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4529999999999999e-06 2024-09-14T06:17:04.421 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.421 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.421 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.421 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334503+0000", 2024-09-14T06:17:04.421 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0518e-05 2024-09-14T06:17:04.421 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.421 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.422 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.422 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334507+0000", 2024-09-14T06:17:04.422 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6660000000000001e-06 2024-09-14T06:17:04.422 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.422 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.422 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.422 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.422 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.422 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32201 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.422 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336012+0000", 2024-09-14T06:17:04.422 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0018844910000000001, 2024-09-14T06:17:04.422 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041187419999999999, 2024-09-14T06:17:04.423 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.423 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.423 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.423 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.423 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.423 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.423 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32201 2024-09-14T06:17:04.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.423 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.423 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.423 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.423 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336012+0000", 2024-09-14T06:17:04.423 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.424 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.424 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.424 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336012+0000", 2024-09-14T06:17:04.424 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.424 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.424 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.424 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.424 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336013+0000", 2024-09-14T06:17:04.424 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3999999999999997e-07 2024-09-14T06:17:04.424 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.424 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.424 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.424 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336020+0000", 2024-09-14T06:17:04.424 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1740000000000003e-06 2024-09-14T06:17:04.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.425 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.425 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.425 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336025+0000", 2024-09-14T06:17:04.425 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8829999999999997e-06 2024-09-14T06:17:04.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.425 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.425 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.425 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336028+0000", 2024-09-14T06:17:04.425 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.163e-06 2024-09-14T06:17:04.425 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.425 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.425 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.426 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.426 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.426 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32233 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.426 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336871+0000", 2024-09-14T06:17:04.426 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0010257840000000001, 2024-09-14T06:17:04.426 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0034313239999999999, 2024-09-14T06:17:04.426 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.426 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.426 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.426 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.426 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.426 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.426 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32233 2024-09-14T06:17:04.426 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.426 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.427 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.427 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.427 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336871+0000", 2024-09-14T06:17:04.427 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.427 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.427 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.427 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.427 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336871+0000", 2024-09-14T06:17:04.427 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.427 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.427 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.427 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.427 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336871+0000", 2024-09-14T06:17:04.427 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3500000000000002e-07 2024-09-14T06:17:04.427 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.428 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.428 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.428 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336872+0000", 2024-09-14T06:17:04.428 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.057e-06 2024-09-14T06:17:04.428 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.428 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.428 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.428 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336877+0000", 2024-09-14T06:17:04.428 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.079e-06 2024-09-14T06:17:04.428 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.428 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.428 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.429 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336880+0000", 2024-09-14T06:17:04.429 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:17:04.429 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.429 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.429 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.429 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.429 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.429 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32265 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.429 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337790+0000", 2024-09-14T06:17:04.429 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.000106433, 2024-09-14T06:17:04.430 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0026692629999999998, 2024-09-14T06:17:04.430 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.430 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.430 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.430 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.430 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.430 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.430 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32265 2024-09-14T06:17:04.430 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.430 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.430 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.430 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.430 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337790+0000", 2024-09-14T06:17:04.430 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.431 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.431 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.431 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.431 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337790+0000", 2024-09-14T06:17:04.431 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.431 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.431 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.431 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.431 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337791+0000", 2024-09-14T06:17:04.431 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.91e-07 2024-09-14T06:17:04.431 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.431 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.431 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.431 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337792+0000", 2024-09-14T06:17:04.431 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.122e-06 2024-09-14T06:17:04.432 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.432 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.432 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.432 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337797+0000", 2024-09-14T06:17:04.432 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3159999999999996e-06 2024-09-14T06:17:04.432 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.432 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.432 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.432 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337803+0000", 2024-09-14T06:17:04.432 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7069999999999997e-06 2024-09-14T06:17:04.432 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.432 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.432 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.433 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.433 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.433 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32235 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.433 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.332596+0000", 2024-09-14T06:17:04.433 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0053005769999999999, 2024-09-14T06:17:04.433 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0080172000000000004, 2024-09-14T06:17:04.433 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.433 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.433 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.433 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.433 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.433 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.433 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32235 2024-09-14T06:17:04.434 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.434 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.434 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.434 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.434 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332596+0000", 2024-09-14T06:17:04.434 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.434 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.434 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.434 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.434 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332596+0000", 2024-09-14T06:17:04.434 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.434 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.434 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.434 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.435 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332597+0000", 2024-09-14T06:17:04.435 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3499999999999995e-07 2024-09-14T06:17:04.435 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.435 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.435 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.435 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332610+0000", 2024-09-14T06:17:04.435 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3463e-05 2024-09-14T06:17:04.435 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.435 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.435 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.435 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332620+0000", 2024-09-14T06:17:04.435 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3060000000000003e-06 2024-09-14T06:17:04.435 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.435 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.435 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.436 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.332625+0000", 2024-09-14T06:17:04.436 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.5910000000000003e-06 2024-09-14T06:17:04.436 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.436 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.436 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.436 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.436 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.436 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32267 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.436 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.333546+0000", 2024-09-14T06:17:04.436 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0043505180000000003, 2024-09-14T06:17:04.436 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0072207139999999996, 2024-09-14T06:17:04.436 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.436 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.437 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.437 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.437 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.437 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.437 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32267 2024-09-14T06:17:04.437 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.437 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.437 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.437 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.437 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333546+0000", 2024-09-14T06:17:04.437 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.437 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.437 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.438 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.438 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333546+0000", 2024-09-14T06:17:04.438 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.438 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.438 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.438 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.438 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333546+0000", 2024-09-14T06:17:04.438 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8599999999999999e-07 2024-09-14T06:17:04.438 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.438 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.438 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.438 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333548+0000", 2024-09-14T06:17:04.438 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1820000000000001e-06 2024-09-14T06:17:04.439 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.439 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.439 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.439 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333553+0000", 2024-09-14T06:17:04.439 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9089999999999999e-06 2024-09-14T06:17:04.439 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.439 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.439 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.439 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333555+0000", 2024-09-14T06:17:04.439 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:17:04.439 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.439 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.439 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.439 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.440 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.440 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32299 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.440 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.334512+0000", 2024-09-14T06:17:04.440 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0033849359999999998, 2024-09-14T06:17:04.440 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0064039149999999996, 2024-09-14T06:17:04.440 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.440 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.440 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.440 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.440 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.440 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.440 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32299 2024-09-14T06:17:04.440 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.440 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.441 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.441 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.441 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334512+0000", 2024-09-14T06:17:04.441 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.441 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.441 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.441 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.441 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334512+0000", 2024-09-14T06:17:04.441 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.441 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.441 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.441 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.441 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334512+0000", 2024-09-14T06:17:04.441 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0600000000000001e-07 2024-09-14T06:17:04.441 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.442 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.442 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.442 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334513+0000", 2024-09-14T06:17:04.442 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4330000000000001e-06 2024-09-14T06:17:04.442 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.442 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.442 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.442 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334521+0000", 2024-09-14T06:17:04.442 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4189999999999996e-06 2024-09-14T06:17:04.442 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.442 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.442 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.442 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334524+0000", 2024-09-14T06:17:04.442 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.163e-06 2024-09-14T06:17:04.443 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.443 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.443 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.443 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.443 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.443 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32202 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.443 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336033+0000", 2024-09-14T06:17:04.443 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.001864054, 2024-09-14T06:17:04.443 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0050329889999999999, 2024-09-14T06:17:04.443 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.443 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.443 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.443 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.443 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.443 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.444 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32202 2024-09-14T06:17:04.444 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.444 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.444 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.444 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.444 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336033+0000", 2024-09-14T06:17:04.444 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.444 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.444 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.444 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.444 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336033+0000", 2024-09-14T06:17:04.444 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.444 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.445 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.445 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336033+0000", 2024-09-14T06:17:04.445 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5400000000000002e-07 2024-09-14T06:17:04.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.445 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.445 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336035+0000", 2024-09-14T06:17:04.445 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.547e-06 2024-09-14T06:17:04.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.445 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.445 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336043+0000", 2024-09-14T06:17:04.445 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1280000000000008e-06 2024-09-14T06:17:04.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.446 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.446 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.446 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336046+0000", 2024-09-14T06:17:04.446 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1920000000000001e-06 2024-09-14T06:17:04.446 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.446 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.446 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.446 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.446 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.446 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32234 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.446 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336885+0000", 2024-09-14T06:17:04.446 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0010114290000000001, 2024-09-14T06:17:04.446 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043336199999999998, 2024-09-14T06:17:04.446 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.446 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.446 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.447 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.447 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.447 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.447 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32234 2024-09-14T06:17:04.447 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.447 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.447 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.447 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.447 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336885+0000", 2024-09-14T06:17:04.447 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.447 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.447 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.447 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.447 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336885+0000", 2024-09-14T06:17:04.447 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.448 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.448 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.448 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336886+0000", 2024-09-14T06:17:04.448 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9599999999999999e-07 2024-09-14T06:17:04.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.448 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.448 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.448 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336888+0000", 2024-09-14T06:17:04.448 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3250000000000002e-06 2024-09-14T06:17:04.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.448 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.448 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.448 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336900+0000", 2024-09-14T06:17:04.449 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1440999999999999e-05 2024-09-14T06:17:04.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.449 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.449 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336903+0000", 2024-09-14T06:17:04.449 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4970000000000001e-06 2024-09-14T06:17:04.449 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.449 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.449 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.450 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32266 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.450 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337809+0000", 2024-09-14T06:17:04.450 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 8.7472000000000005e-05, 2024-09-14T06:17:04.450 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0035196720000000002, 2024-09-14T06:17:04.450 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.450 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.450 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.450 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.450 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.450 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.450 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32266 2024-09-14T06:17:04.450 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.450 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.450 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.450 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.451 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337809+0000", 2024-09-14T06:17:04.451 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.451 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.451 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.451 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.451 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337809+0000", 2024-09-14T06:17:04.451 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.451 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.451 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.451 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.451 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337810+0000", 2024-09-14T06:17:04.451 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4600000000000004e-07 2024-09-14T06:17:04.451 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.451 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.452 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.452 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337812+0000", 2024-09-14T06:17:04.452 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.029e-06 2024-09-14T06:17:04.452 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.452 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.452 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.452 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337818+0000", 2024-09-14T06:17:04.452 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1249999999999998e-06 2024-09-14T06:17:04.452 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.452 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.452 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.452 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337825+0000", 2024-09-14T06:17:04.453 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0530000000000001e-06 2024-09-14T06:17:04.453 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.453 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.453 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.453 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.453 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.453 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32268 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.453 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.333559+0000", 2024-09-14T06:17:04.453 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0043372649999999999, 2024-09-14T06:17:04.453 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0078592309999999995, 2024-09-14T06:17:04.453 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.453 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.453 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.453 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.454 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.454 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.454 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32268 2024-09-14T06:17:04.454 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.454 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.454 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.454 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.454 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333559+0000", 2024-09-14T06:17:04.454 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.454 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.454 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.454 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.454 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333559+0000", 2024-09-14T06:17:04.454 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.454 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.454 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.455 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.455 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333560+0000", 2024-09-14T06:17:04.455 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5400000000000002e-07 2024-09-14T06:17:04.455 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.455 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.455 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.455 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333561+0000", 2024-09-14T06:17:04.455 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.114e-06 2024-09-14T06:17:04.455 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.455 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.455 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.455 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333570+0000", 2024-09-14T06:17:04.455 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.713e-06 2024-09-14T06:17:04.455 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.455 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.455 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.456 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333573+0000", 2024-09-14T06:17:04.456 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0029999999999999e-06 2024-09-14T06:17:04.456 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.456 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.456 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.456 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.456 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.456 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32300 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.456 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.334529+0000", 2024-09-14T06:17:04.456 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00336733, 2024-09-14T06:17:04.456 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0070395989999999997, 2024-09-14T06:17:04.456 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.457 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.457 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.457 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.457 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.457 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.457 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32300 2024-09-14T06:17:04.457 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.457 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.457 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.457 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.457 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334529+0000", 2024-09-14T06:17:04.457 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.457 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.457 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.457 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334529+0000", 2024-09-14T06:17:04.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.458 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.458 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334530+0000", 2024-09-14T06:17:04.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2299999999999999e-07 2024-09-14T06:17:04.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.458 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.458 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334532+0000", 2024-09-14T06:17:04.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9259999999999999e-06 2024-09-14T06:17:04.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.458 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.458 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.459 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334538+0000", 2024-09-14T06:17:04.459 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9900000000000002e-06 2024-09-14T06:17:04.459 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.459 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.459 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.459 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334543+0000", 2024-09-14T06:17:04.459 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.797e-06 2024-09-14T06:17:04.459 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.459 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.459 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.459 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.459 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.459 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32203 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.459 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336051+0000", 2024-09-14T06:17:04.459 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.001845911, 2024-09-14T06:17:04.460 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056672959999999996, 2024-09-14T06:17:04.460 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.460 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.460 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.460 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.460 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.460 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.460 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32203 2024-09-14T06:17:04.460 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.460 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.460 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.460 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.460 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336051+0000", 2024-09-14T06:17:04.460 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336051+0000", 2024-09-14T06:17:04.461 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336051+0000", 2024-09-14T06:17:04.461 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2900000000000003e-07 2024-09-14T06:17:04.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336054+0000", 2024-09-14T06:17:04.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0439999999999998e-06 2024-09-14T06:17:04.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.462 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.462 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336066+0000", 2024-09-14T06:17:04.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2072999999999999e-05 2024-09-14T06:17:04.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.462 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.462 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336071+0000", 2024-09-14T06:17:04.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8860000000000003e-06 2024-09-14T06:17:04.462 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.462 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.462 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.463 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.463 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.463 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32235 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.463 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336919+0000", 2024-09-14T06:17:04.463 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00097754299999999999, 2024-09-14T06:17:04.463 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0049528109999999997, 2024-09-14T06:17:04.463 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.463 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.463 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.463 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.463 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.463 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.463 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32235 2024-09-14T06:17:04.463 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.463 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.464 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.464 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.464 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336919+0000", 2024-09-14T06:17:04.464 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.464 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.464 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.464 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.464 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336919+0000", 2024-09-14T06:17:04.464 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.464 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.464 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.464 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.464 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336920+0000", 2024-09-14T06:17:04.465 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.37e-07 2024-09-14T06:17:04.465 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.465 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.465 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.465 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336926+0000", 2024-09-14T06:17:04.465 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8869999999999997e-06 2024-09-14T06:17:04.465 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.465 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.465 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.465 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336932+0000", 2024-09-14T06:17:04.465 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.528e-06 2024-09-14T06:17:04.465 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.466 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.466 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.466 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336937+0000", 2024-09-14T06:17:04.466 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6970000000000001e-06 2024-09-14T06:17:04.466 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.466 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.466 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.466 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.466 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.466 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32267 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.466 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337874+0000", 2024-09-14T06:17:04.466 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 2.2603999999999999e-05, 2024-09-14T06:17:04.466 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041513879999999998, 2024-09-14T06:17:04.466 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.467 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.467 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.467 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.467 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.467 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.467 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32267 2024-09-14T06:17:04.467 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.467 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.467 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.467 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.467 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337874+0000", 2024-09-14T06:17:04.467 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.467 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.467 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.468 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.468 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337874+0000", 2024-09-14T06:17:04.468 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.468 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.468 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.468 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.468 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337874+0000", 2024-09-14T06:17:04.468 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3000000000000002e-07 2024-09-14T06:17:04.468 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.468 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.468 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.468 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337880+0000", 2024-09-14T06:17:04.468 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.648e-06 2024-09-14T06:17:04.469 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.469 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.469 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.469 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337886+0000", 2024-09-14T06:17:04.469 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.5629999999999999e-06 2024-09-14T06:17:04.469 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.469 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.469 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.469 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337892+0000", 2024-09-14T06:17:04.470 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.268e-06 2024-09-14T06:17:04.470 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.470 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.470 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.470 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.470 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.470 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32269 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.470 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.333604+0000", 2024-09-14T06:17:04.470 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0042923350000000004, 2024-09-14T06:17:04.470 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0085751190000000008, 2024-09-14T06:17:04.470 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.470 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.470 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.471 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.471 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.471 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.471 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32269 2024-09-14T06:17:04.471 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.471 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.471 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.471 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.471 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333604+0000", 2024-09-14T06:17:04.471 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.471 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.471 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.472 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.472 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333604+0000", 2024-09-14T06:17:04.472 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.472 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.472 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.472 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333605+0000", 2024-09-14T06:17:04.472 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4399999999999999e-07 2024-09-14T06:17:04.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.472 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.472 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.472 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333607+0000", 2024-09-14T06:17:04.472 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1069999999999999e-06 2024-09-14T06:17:04.473 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.473 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.473 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.473 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333616+0000", 2024-09-14T06:17:04.473 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6670000000000002e-06 2024-09-14T06:17:04.473 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.473 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.473 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.473 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.333621+0000", 2024-09-14T06:17:04.473 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.5369999999999997e-06 2024-09-14T06:17:04.473 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.473 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.473 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.474 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.474 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32301 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.474 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.334552+0000", 2024-09-14T06:17:04.474 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0033446050000000001, 2024-09-14T06:17:04.474 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0077787150000000003, 2024-09-14T06:17:04.474 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.474 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.474 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.474 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.474 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.474 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.474 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32301 2024-09-14T06:17:04.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.475 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.475 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.475 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.475 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334552+0000", 2024-09-14T06:17:04.475 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.475 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.475 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.475 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334552+0000", 2024-09-14T06:17:04.475 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.475 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.475 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.475 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334552+0000", 2024-09-14T06:17:04.475 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2500000000000001e-07 2024-09-14T06:17:04.476 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.476 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.476 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.476 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334554+0000", 2024-09-14T06:17:04.476 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.945e-06 2024-09-14T06:17:04.476 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.476 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.476 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.476 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334561+0000", 2024-09-14T06:17:04.476 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9179999999999997e-06 2024-09-14T06:17:04.476 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.476 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.476 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.476 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334566+0000", 2024-09-14T06:17:04.476 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6600000000000003e-06 2024-09-14T06:17:04.477 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.477 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.477 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.477 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.477 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.477 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32204 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.477 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336078+0000", 2024-09-14T06:17:04.477 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0018189510000000001, 2024-09-14T06:17:04.477 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.006416728, 2024-09-14T06:17:04.477 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.477 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.477 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.477 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.477 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.477 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.477 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32204 2024-09-14T06:17:04.478 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.478 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.478 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.478 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.478 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336078+0000", 2024-09-14T06:17:04.478 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.478 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.478 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.478 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.478 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336078+0000", 2024-09-14T06:17:04.478 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.478 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.479 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.479 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.479 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336078+0000", 2024-09-14T06:17:04.479 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:17:04.479 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.479 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.479 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.479 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336080+0000", 2024-09-14T06:17:04.479 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.934e-06 2024-09-14T06:17:04.479 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.479 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.479 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.480 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336087+0000", 2024-09-14T06:17:04.480 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3180000000000001e-06 2024-09-14T06:17:04.480 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.480 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.480 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.480 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336092+0000", 2024-09-14T06:17:04.480 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.955e-06 2024-09-14T06:17:04.480 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.480 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.480 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.480 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.480 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.480 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32236 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.481 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336969+0000", 2024-09-14T06:17:04.481 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00092756000000000002, 2024-09-14T06:17:04.481 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056745199999999997, 2024-09-14T06:17:04.481 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.481 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.481 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.481 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.481 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.481 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.481 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32236 2024-09-14T06:17:04.481 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.481 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.481 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.482 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.482 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336969+0000", 2024-09-14T06:17:04.482 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.482 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.482 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.482 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336969+0000", 2024-09-14T06:17:04.482 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.482 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.482 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.482 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336969+0000", 2024-09-14T06:17:04.482 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7599999999999998e-07 2024-09-14T06:17:04.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.482 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.482 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.483 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336971+0000", 2024-09-14T06:17:04.483 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5179999999999999e-06 2024-09-14T06:17:04.483 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.483 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.483 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.483 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336977+0000", 2024-09-14T06:17:04.483 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4690000000000002e-06 2024-09-14T06:17:04.483 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.483 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.483 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.483 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336993+0000", 2024-09-14T06:17:04.483 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.523e-05 2024-09-14T06:17:04.483 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.484 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.484 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.484 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.484 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.484 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32268 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.484 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337897+0000", 2024-09-14T06:17:04.484 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967296, 2024-09-14T06:17:04.484 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0049047090000000002, 2024-09-14T06:17:04.484 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.484 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.484 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.484 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.484 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.484 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.485 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32268 2024-09-14T06:17:04.485 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.485 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.485 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.485 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.485 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337897+0000", 2024-09-14T06:17:04.485 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.485 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.485 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.485 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.485 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337897+0000", 2024-09-14T06:17:04.485 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.485 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.485 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337897+0000", 2024-09-14T06:17:04.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.22e-07 2024-09-14T06:17:04.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337898+0000", 2024-09-14T06:17:04.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2699999999999999e-06 2024-09-14T06:17:04.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337903+0000", 2024-09-14T06:17:04.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8339999999999998e-06 2024-09-14T06:17:04.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.487 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.487 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337906+0000", 2024-09-14T06:17:04.487 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8270000000000002e-06 2024-09-14T06:17:04.487 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.487 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.487 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.487 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.487 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.487 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32302 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.487 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.334582+0000", 2024-09-14T06:17:04.487 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.003314418, 2024-09-14T06:17:04.487 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0083729470000000004, 2024-09-14T06:17:04.487 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.487 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.487 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.487 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.488 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.488 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.488 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32302 2024-09-14T06:17:04.488 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.488 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.488 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.488 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.488 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334582+0000", 2024-09-14T06:17:04.488 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.488 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.488 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.488 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.488 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334582+0000", 2024-09-14T06:17:04.488 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.488 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.489 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.489 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.489 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334583+0000", 2024-09-14T06:17:04.489 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9200000000000002e-07 2024-09-14T06:17:04.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.489 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.489 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.489 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334584+0000", 2024-09-14T06:17:04.489 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.432e-06 2024-09-14T06:17:04.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.489 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.489 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.489 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334589+0000", 2024-09-14T06:17:04.489 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0989999999999998e-06 2024-09-14T06:17:04.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.490 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.490 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334599+0000", 2024-09-14T06:17:04.490 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.4579999999999999e-06 2024-09-14T06:17:04.490 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.490 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.490 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.490 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32205 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.490 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336117+0000", 2024-09-14T06:17:04.490 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0017798219999999999, 2024-09-14T06:17:04.490 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.006998013, 2024-09-14T06:17:04.490 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.490 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.491 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.491 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.491 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.491 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.491 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32205 2024-09-14T06:17:04.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.491 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.491 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.491 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.491 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336117+0000", 2024-09-14T06:17:04.491 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.491 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.491 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.492 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336117+0000", 2024-09-14T06:17:04.492 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.492 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.492 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.492 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.492 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336117+0000", 2024-09-14T06:17:04.492 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1800000000000002e-07 2024-09-14T06:17:04.492 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.492 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.492 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.492 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336119+0000", 2024-09-14T06:17:04.492 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4139999999999999e-06 2024-09-14T06:17:04.492 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.492 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.493 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.493 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336125+0000", 2024-09-14T06:17:04.493 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1380000000000004e-06 2024-09-14T06:17:04.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.493 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.493 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.493 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336128+0000", 2024-09-14T06:17:04.493 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1219999999999999e-06 2024-09-14T06:17:04.493 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.493 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.493 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.493 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.493 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32237 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.493 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336998+0000", 2024-09-14T06:17:04.494 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00089862099999999995, 2024-09-14T06:17:04.494 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0062704750000000002, 2024-09-14T06:17:04.494 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.494 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.494 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.494 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.494 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.494 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.494 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32237 2024-09-14T06:17:04.494 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.494 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.494 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.494 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.494 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336998+0000", 2024-09-14T06:17:04.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336998+0000", 2024-09-14T06:17:04.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336998+0000", 2024-09-14T06:17:04.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6199999999999999e-07 2024-09-14T06:17:04.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337000+0000", 2024-09-14T06:17:04.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.525e-06 2024-09-14T06:17:04.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.496 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.496 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337008+0000", 2024-09-14T06:17:04.496 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8520000000000004e-06 2024-09-14T06:17:04.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.496 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.496 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337011+0000", 2024-09-14T06:17:04.496 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2310000000000001e-06 2024-09-14T06:17:04.496 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.496 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.496 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.497 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32269 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.497 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337910+0000", 2024-09-14T06:17:04.497 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9999871, 2024-09-14T06:17:04.497 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0055122399999999998, 2024-09-14T06:17:04.497 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.497 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.497 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.497 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.497 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.497 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.497 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32269 2024-09-14T06:17:04.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.497 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.497 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337910+0000", 2024-09-14T06:17:04.498 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.498 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337910+0000", 2024-09-14T06:17:04.498 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.498 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337910+0000", 2024-09-14T06:17:04.498 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7799999999999997e-07 2024-09-14T06:17:04.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.498 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337911+0000", 2024-09-14T06:17:04.499 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1650000000000001e-06 2024-09-14T06:17:04.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.499 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.499 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337918+0000", 2024-09-14T06:17:04.499 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0269999999999998e-06 2024-09-14T06:17:04.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.499 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.499 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337923+0000", 2024-09-14T06:17:04.499 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8779999999999997e-06 2024-09-14T06:17:04.499 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.499 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.499 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.500 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32303 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.500 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.334604+0000", 2024-09-14T06:17:04.500 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0032927400000000002, 2024-09-14T06:17:04.500 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0089819770000000004, 2024-09-14T06:17:04.500 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.500 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.500 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.500 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.500 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.500 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.500 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32303 2024-09-14T06:17:04.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.501 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.501 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.501 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334604+0000", 2024-09-14T06:17:04.501 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.501 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.501 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334604+0000", 2024-09-14T06:17:04.501 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.501 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334604+0000", 2024-09-14T06:17:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0199999999999998e-07 2024-09-14T06:17:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334605+0000", 2024-09-14T06:17:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2270000000000001e-06 2024-09-14T06:17:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334611+0000", 2024-09-14T06:17:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8499999999999999e-06 2024-09-14T06:17:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334614+0000", 2024-09-14T06:17:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7439999999999999e-06 2024-09-14T06:17:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32206 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336132+0000", 2024-09-14T06:17:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.001764498, 2024-09-14T06:17:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0076031939999999998, 2024-09-14T06:17:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32206 2024-09-14T06:17:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336132+0000", 2024-09-14T06:17:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336132+0000", 2024-09-14T06:17:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336132+0000", 2024-09-14T06:17:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7000000000000001e-07 2024-09-14T06:17:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336134+0000", 2024-09-14T06:17:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.153e-06 2024-09-14T06:17:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336138+0000", 2024-09-14T06:17:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5589999999999999e-06 2024-09-14T06:17:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336141+0000", 2024-09-14T06:17:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:17:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32238 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337017+0000", 2024-09-14T06:17:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00087980999999999997, 2024-09-14T06:17:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0068758389999999999, 2024-09-14T06:17:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32238 2024-09-14T06:17:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337017+0000", 2024-09-14T06:17:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337017+0000", 2024-09-14T06:17:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337017+0000", 2024-09-14T06:17:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2699999999999999e-07 2024-09-14T06:17:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337020+0000", 2024-09-14T06:17:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0219999999999999e-06 2024-09-14T06:17:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337026+0000", 2024-09-14T06:17:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9360000000000002e-06 2024-09-14T06:17:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337032+0000", 2024-09-14T06:17:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0699999999999997e-06 2024-09-14T06:17:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32270 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337930+0000", 2024-09-14T06:17:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9999671, 2024-09-14T06:17:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0061149409999999996, 2024-09-14T06:17:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32270 2024-09-14T06:17:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337930+0000", 2024-09-14T06:17:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337930+0000", 2024-09-14T06:17:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337930+0000", 2024-09-14T06:17:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3500000000000002e-07 2024-09-14T06:17:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337932+0000", 2024-09-14T06:17:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.903e-06 2024-09-14T06:17:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337938+0000", 2024-09-14T06:17:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1449999999999996e-06 2024-09-14T06:17:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337944+0000", 2024-09-14T06:17:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.5459999999999999e-06 2024-09-14T06:17:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32304 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.334619+0000", 2024-09-14T06:17:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0032772769999999999, 2024-09-14T06:17:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0095971709999999998, 2024-09-14T06:17:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32304 2024-09-14T06:17:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334619+0000", 2024-09-14T06:17:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334619+0000", 2024-09-14T06:17:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334620+0000", 2024-09-14T06:17:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0899999999999997e-07 2024-09-14T06:17:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334622+0000", 2024-09-14T06:17:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6290000000000002e-06 2024-09-14T06:17:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334632+0000", 2024-09-14T06:17:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0105000000000001e-05 2024-09-14T06:17:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.334637+0000", 2024-09-14T06:17:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1749999999999998e-06 2024-09-14T06:17:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32207 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336146+0000", 2024-09-14T06:17:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0017505450000000001, 2024-09-14T06:17:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0082228990000000005, 2024-09-14T06:17:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32207 2024-09-14T06:17:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336146+0000", 2024-09-14T06:17:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336146+0000", 2024-09-14T06:17:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.518 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.518 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336147+0000", 2024-09-14T06:17:04.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8899999999999999e-07 2024-09-14T06:17:04.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.518 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.518 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336149+0000", 2024-09-14T06:17:04.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:17:04.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336159+0000", 2024-09-14T06:17:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.8439999999999993e-06 2024-09-14T06:17:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336163+0000", 2024-09-14T06:17:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0509999999999998e-06 2024-09-14T06:17:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32239 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337038+0000", 2024-09-14T06:17:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00085851199999999997, 2024-09-14T06:17:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0074885489999999997, 2024-09-14T06:17:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32239 2024-09-14T06:17:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337038+0000", 2024-09-14T06:17:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337038+0000", 2024-09-14T06:17:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337039+0000", 2024-09-14T06:17:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4400000000000001e-07 2024-09-14T06:17:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337040+0000", 2024-09-14T06:17:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.835e-06 2024-09-14T06:17:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337047+0000", 2024-09-14T06:17:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9920000000000001e-06 2024-09-14T06:17:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337052+0000", 2024-09-14T06:17:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8570000000000003e-06 2024-09-14T06:17:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32271 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337960+0000", 2024-09-14T06:17:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9999371, 2024-09-14T06:17:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0067212879999999997, 2024-09-14T06:17:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32271 2024-09-14T06:17:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337960+0000", 2024-09-14T06:17:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337960+0000", 2024-09-14T06:17:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337960+0000", 2024-09-14T06:17:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5900000000000002e-07 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337974+0000", 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4124999999999999e-05 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337991+0000", 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6135999999999999e-05 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337997+0000", 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7360000000000004e-06 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32208 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336178+0000", 2024-09-14T06:17:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0017184500000000001, 2024-09-14T06:17:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0086644489999999994, 2024-09-14T06:17:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32208 2024-09-14T06:17:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336178+0000", 2024-09-14T06:17:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336178+0000", 2024-09-14T06:17:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336179+0000", 2024-09-14T06:17:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6199999999999998e-07 2024-09-14T06:17:04.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.529 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.529 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336182+0000", 2024-09-14T06:17:04.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2380000000000002e-06 2024-09-14T06:17:04.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.529 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.529 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336192+0000", 2024-09-14T06:17:04.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7650000000000005e-06 2024-09-14T06:17:04.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.530 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.530 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.530 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336204+0000", 2024-09-14T06:17:04.530 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2123000000000001e-05 2024-09-14T06:17:04.530 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.530 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.530 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.530 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.530 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32240 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.530 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337059+0000", 2024-09-14T06:17:04.530 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00083761299999999996, 2024-09-14T06:17:04.530 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0079349650000000004, 2024-09-14T06:17:04.531 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.531 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.531 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.531 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.531 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.531 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.531 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32240 2024-09-14T06:17:04.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.531 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337059+0000", 2024-09-14T06:17:04.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337059+0000", 2024-09-14T06:17:04.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337059+0000", 2024-09-14T06:17:04.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0400000000000002e-07 2024-09-14T06:17:04.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337061+0000", 2024-09-14T06:17:04.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.328e-06 2024-09-14T06:17:04.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.533 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.533 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.533 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337066+0000", 2024-09-14T06:17:04.533 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1290000000000004e-06 2024-09-14T06:17:04.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.533 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.533 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.533 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337069+0000", 2024-09-14T06:17:04.533 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.96e-06 2024-09-14T06:17:04.533 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.533 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.533 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.534 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32272 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.534 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338010+0000", 2024-09-14T06:17:04.534 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9998865, 2024-09-14T06:17:04.534 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0071442069999999996, 2024-09-14T06:17:04.534 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.534 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.534 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.534 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.534 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.534 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.534 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32272 2024-09-14T06:17:04.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.534 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.535 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338010+0000", 2024-09-14T06:17:04.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.535 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338010+0000", 2024-09-14T06:17:04.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.535 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338011+0000", 2024-09-14T06:17:04.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8699999999999995e-07 2024-09-14T06:17:04.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338013+0000", 2024-09-14T06:17:04.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1940000000000001e-06 2024-09-14T06:17:04.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338019+0000", 2024-09-14T06:17:04.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:17:04.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338031+0000", 2024-09-14T06:17:04.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1384e-05 2024-09-14T06:17:04.537 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.537 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.537 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.537 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32209 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.537 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336230+0000", 2024-09-14T06:17:04.537 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0016671380000000001, 2024-09-14T06:17:04.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0090780119999999999, 2024-09-14T06:17:04.537 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.537 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.537 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.538 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.538 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.538 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.538 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32209 2024-09-14T06:17:04.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.538 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.538 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.538 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336230+0000", 2024-09-14T06:17:04.538 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.538 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.538 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336230+0000", 2024-09-14T06:17:04.539 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.539 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.539 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.539 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336230+0000", 2024-09-14T06:17:04.539 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4199999999999996e-07 2024-09-14T06:17:04.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.539 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.539 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.539 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336233+0000", 2024-09-14T06:17:04.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.441e-06 2024-09-14T06:17:04.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336247+0000", 2024-09-14T06:17:04.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4467999999999999e-05 2024-09-14T06:17:04.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336258+0000", 2024-09-14T06:17:04.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1100999999999999e-05 2024-09-14T06:17:04.540 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.541 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.541 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.541 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.541 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32241 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.541 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337073+0000", 2024-09-14T06:17:04.541 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00082382600000000005, 2024-09-14T06:17:04.541 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0083965900000000007, 2024-09-14T06:17:04.541 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.541 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.541 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.541 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.541 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.541 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.542 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32241 2024-09-14T06:17:04.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.542 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.542 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.542 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337073+0000", 2024-09-14T06:17:04.542 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.542 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.542 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337073+0000", 2024-09-14T06:17:04.542 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.542 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337073+0000", 2024-09-14T06:17:04.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.79e-07 2024-09-14T06:17:04.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337074+0000", 2024-09-14T06:17:04.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1459999999999999e-06 2024-09-14T06:17:04.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337079+0000", 2024-09-14T06:17:04.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5059999999999997e-06 2024-09-14T06:17:04.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337082+0000", 2024-09-14T06:17:04.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.004e-06 2024-09-14T06:17:04.544 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.544 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.544 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.544 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.544 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32273 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.544 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338039+0000", 2024-09-14T06:17:04.544 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9998579, 2024-09-14T06:17:04.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0075837530000000004, 2024-09-14T06:17:04.544 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.544 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.545 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.545 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.545 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.545 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.545 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32273 2024-09-14T06:17:04.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.545 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.545 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.545 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338039+0000", 2024-09-14T06:17:04.545 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.546 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.546 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338039+0000", 2024-09-14T06:17:04.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.546 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.546 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338039+0000", 2024-09-14T06:17:04.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0200000000000002e-07 2024-09-14T06:17:04.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.546 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.546 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338041+0000", 2024-09-14T06:17:04.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.04e-06 2024-09-14T06:17:04.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.546 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.547 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338048+0000", 2024-09-14T06:17:04.547 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.5910000000000001e-06 2024-09-14T06:17:04.547 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.547 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.547 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.547 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338052+0000", 2024-09-14T06:17:04.547 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.544e-06 2024-09-14T06:17:04.547 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.547 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.547 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.547 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.547 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.547 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32210 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.547 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336282+0000", 2024-09-14T06:17:04.547 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.001614716, 2024-09-14T06:17:04.547 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0094991350000000006, 2024-09-14T06:17:04.548 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.548 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.548 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.548 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.548 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.548 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.548 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32210 2024-09-14T06:17:04.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.548 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.548 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.548 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.548 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336282+0000", 2024-09-14T06:17:04.548 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336282+0000", 2024-09-14T06:17:04.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336282+0000", 2024-09-14T06:17:04.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:17:04.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336284+0000", 2024-09-14T06:17:04.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4780000000000001e-06 2024-09-14T06:17:04.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336291+0000", 2024-09-14T06:17:04.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7440000000000001e-06 2024-09-14T06:17:04.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336294+0000", 2024-09-14T06:17:04.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2559999999999998e-06 2024-09-14T06:17:04.550 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.550 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.551 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.551 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.551 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.551 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32242 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.551 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337086+0000", 2024-09-14T06:17:04.551 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00081104300000000001, 2024-09-14T06:17:04.551 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0088102270000000003, 2024-09-14T06:17:04.551 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.552 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.552 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.552 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.552 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.552 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.552 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32242 2024-09-14T06:17:04.552 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.552 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.552 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.552 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.553 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337086+0000", 2024-09-14T06:17:04.553 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.553 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.553 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.553 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.553 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337086+0000", 2024-09-14T06:17:04.553 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.553 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.553 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.553 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.553 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337086+0000", 2024-09-14T06:17:04.553 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.29e-07 2024-09-14T06:17:04.553 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.553 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337087+0000", 2024-09-14T06:17:04.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.08e-06 2024-09-14T06:17:04.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337096+0000", 2024-09-14T06:17:04.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5410000000000006e-06 2024-09-14T06:17:04.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337100+0000", 2024-09-14T06:17:04.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9689999999999996e-06 2024-09-14T06:17:04.554 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.554 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.555 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.555 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.555 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32274 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.555 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338059+0000", 2024-09-14T06:17:04.555 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9998374, 2024-09-14T06:17:04.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0079823280000000003, 2024-09-14T06:17:04.555 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.555 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.555 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.555 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.555 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.555 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.555 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32274 2024-09-14T06:17:04.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.556 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.556 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.556 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338059+0000", 2024-09-14T06:17:04.556 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.556 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.556 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338059+0000", 2024-09-14T06:17:04.556 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.556 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.556 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338060+0000", 2024-09-14T06:17:04.556 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4499999999999997e-07 2024-09-14T06:17:04.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.557 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.557 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.557 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338061+0000", 2024-09-14T06:17:04.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9460000000000001e-06 2024-09-14T06:17:04.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.557 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.557 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.557 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338068+0000", 2024-09-14T06:17:04.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7490000000000001e-06 2024-09-14T06:17:04.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.557 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.558 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.558 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338073+0000", 2024-09-14T06:17:04.558 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2170000000000002e-06 2024-09-14T06:17:04.558 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.558 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.558 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.558 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32211 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.558 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336298+0000", 2024-09-14T06:17:04.558 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.001598386, 2024-09-14T06:17:04.558 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0098948969999999997, 2024-09-14T06:17:04.558 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.558 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.558 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.558 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.559 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.559 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.559 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32211 2024-09-14T06:17:04.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.559 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.559 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.559 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.559 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336298+0000", 2024-09-14T06:17:04.559 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.559 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.559 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.559 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336298+0000", 2024-09-14T06:17:04.559 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.560 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.560 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336299+0000", 2024-09-14T06:17:04.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:17:04.560 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.560 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.560 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336309+0000", 2024-09-14T06:17:04.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0866999999999999e-05 2024-09-14T06:17:04.560 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.560 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.560 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336319+0000", 2024-09-14T06:17:04.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7540000000000001e-06 2024-09-14T06:17:04.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.561 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.561 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336323+0000", 2024-09-14T06:17:04.561 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9009999999999996e-06 2024-09-14T06:17:04.561 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.561 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.561 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.561 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32243 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.561 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337112+0000", 2024-09-14T06:17:04.561 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00078427499999999997, 2024-09-14T06:17:04.561 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0092321690000000001, 2024-09-14T06:17:04.562 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.562 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.562 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.562 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.562 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.562 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.562 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32243 2024-09-14T06:17:04.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.562 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.562 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.562 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337112+0000", 2024-09-14T06:17:04.562 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.563 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337112+0000", 2024-09-14T06:17:04.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.563 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.563 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337113+0000", 2024-09-14T06:17:04.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2200000000000003e-07 2024-09-14T06:17:04.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.563 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.563 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337115+0000", 2024-09-14T06:17:04.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1179999999999999e-06 2024-09-14T06:17:04.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.563 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.564 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.564 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337121+0000", 2024-09-14T06:17:04.564 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0669999999999997e-06 2024-09-14T06:17:04.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.564 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.564 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337129+0000", 2024-09-14T06:17:04.564 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0309999999999998e-06 2024-09-14T06:17:04.564 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.564 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.564 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.565 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32275 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.565 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338090+0000", 2024-09-14T06:17:04.565 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9998069, 2024-09-14T06:17:04.565 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0084136209999999996, 2024-09-14T06:17:04.565 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.565 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.565 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.565 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.565 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.565 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.565 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32275 2024-09-14T06:17:04.565 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.566 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.566 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.566 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338090+0000", 2024-09-14T06:17:04.566 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.566 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.566 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338090+0000", 2024-09-14T06:17:04.566 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338090+0000", 2024-09-14T06:17:04.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3599999999999999e-07 2024-09-14T06:17:04.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338092+0000", 2024-09-14T06:17:04.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0250000000000001e-06 2024-09-14T06:17:04.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338099+0000", 2024-09-14T06:17:04.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6590000000000001e-06 2024-09-14T06:17:04.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.568 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.568 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.568 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338104+0000", 2024-09-14T06:17:04.568 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.519e-06 2024-09-14T06:17:04.568 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.568 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.568 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.568 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.568 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32212 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.568 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336330+0000", 2024-09-14T06:17:04.568 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0015669359999999999, 2024-09-14T06:17:04.568 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010293504, 2024-09-14T06:17:04.568 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.569 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.569 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.569 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.569 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.569 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.569 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32212 2024-09-14T06:17:04.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.569 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.569 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.570 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336330+0000", 2024-09-14T06:17:04.570 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.570 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.570 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336330+0000", 2024-09-14T06:17:04.570 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.570 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.570 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336330+0000", 2024-09-14T06:17:04.570 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5499999999999997e-07 2024-09-14T06:17:04.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.571 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.571 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.571 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336338+0000", 2024-09-14T06:17:04.571 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8800000000000008e-06 2024-09-14T06:17:04.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.571 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.571 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.571 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336345+0000", 2024-09-14T06:17:04.571 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8260000000000004e-06 2024-09-14T06:17:04.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.571 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.571 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.572 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336350+0000", 2024-09-14T06:17:04.572 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8280000000000002e-06 2024-09-14T06:17:04.572 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.572 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.572 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.572 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.572 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32244 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.572 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337160+0000", 2024-09-14T06:17:04.572 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00073636399999999999, 2024-09-14T06:17:04.572 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0095794199999999999, 2024-09-14T06:17:04.572 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.572 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.572 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.573 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.573 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.573 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.573 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32244 2024-09-14T06:17:04.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.573 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.573 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.573 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.573 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337160+0000", 2024-09-14T06:17:04.573 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.573 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.573 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337160+0000", 2024-09-14T06:17:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337161+0000", 2024-09-14T06:17:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9999999999999997e-07 2024-09-14T06:17:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337163+0000", 2024-09-14T06:17:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1119999999999999e-06 2024-09-14T06:17:04.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337170+0000", 2024-09-14T06:17:04.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4860000000000002e-06 2024-09-14T06:17:04.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337174+0000", 2024-09-14T06:17:04.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9749999999999998e-06 2024-09-14T06:17:04.575 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.575 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.575 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.575 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32276 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.576 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338120+0000", 2024-09-14T06:17:04.576 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9997768, 2024-09-14T06:17:04.576 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0087691460000000002, 2024-09-14T06:17:04.576 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.576 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.576 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.576 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.576 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.576 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.576 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32276 2024-09-14T06:17:04.576 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.576 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.576 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.576 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.577 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338120+0000", 2024-09-14T06:17:04.577 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.577 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.577 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.577 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338120+0000", 2024-09-14T06:17:04.577 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.577 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.577 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.577 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338120+0000", 2024-09-14T06:17:04.577 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8500000000000002e-07 2024-09-14T06:17:04.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.578 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.578 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.578 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338122+0000", 2024-09-14T06:17:04.578 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.03e-06 2024-09-14T06:17:04.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.578 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.578 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.578 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338134+0000", 2024-09-14T06:17:04.578 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2028000000000001e-05 2024-09-14T06:17:04.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.578 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.578 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.578 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338139+0000", 2024-09-14T06:17:04.578 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.904e-06 2024-09-14T06:17:04.578 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.578 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.579 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.579 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.579 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.579 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32213 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.579 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336365+0000", 2024-09-14T06:17:04.579 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0015315649999999999, 2024-09-14T06:17:04.579 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010702056, 2024-09-14T06:17:04.579 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.579 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.579 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.579 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.580 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.580 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.580 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32213 2024-09-14T06:17:04.580 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.580 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.580 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.580 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.580 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336365+0000", 2024-09-14T06:17:04.580 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.580 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.580 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.580 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.580 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336365+0000", 2024-09-14T06:17:04.580 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.581 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.581 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336366+0000", 2024-09-14T06:17:04.581 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.75e-07 2024-09-14T06:17:04.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.581 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.581 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336368+0000", 2024-09-14T06:17:04.581 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4399999999999999e-06 2024-09-14T06:17:04.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.581 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.581 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336377+0000", 2024-09-14T06:17:04.582 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4849999999999992e-06 2024-09-14T06:17:04.582 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.582 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.582 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.582 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336388+0000", 2024-09-14T06:17:04.582 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0786e-05 2024-09-14T06:17:04.582 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.582 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.582 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.582 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.582 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.582 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32245 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.582 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337180+0000", 2024-09-14T06:17:04.582 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00071688399999999999, 2024-09-14T06:17:04.583 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010035028, 2024-09-14T06:17:04.583 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.583 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.583 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.583 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.583 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.583 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.583 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32245 2024-09-14T06:17:04.583 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.583 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.583 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.583 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.583 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337180+0000", 2024-09-14T06:17:04.583 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.584 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.584 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337180+0000", 2024-09-14T06:17:04.584 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.584 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.584 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337181+0000", 2024-09-14T06:17:04.584 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1800000000000004e-07 2024-09-14T06:17:04.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.584 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.584 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337188+0000", 2024-09-14T06:17:04.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5730000000000002e-06 2024-09-14T06:17:04.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.585 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337196+0000", 2024-09-14T06:17:04.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8210000000000002e-06 2024-09-14T06:17:04.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.585 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337206+0000", 2024-09-14T06:17:04.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.4690000000000003e-06 2024-09-14T06:17:04.585 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.585 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.585 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.586 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.586 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32277 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.586 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338151+0000", 2024-09-14T06:17:04.586 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9997458, 2024-09-14T06:17:04.586 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0091945949999999999, 2024-09-14T06:17:04.586 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.586 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.586 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.586 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.586 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.586 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.586 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32277 2024-09-14T06:17:04.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.586 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.587 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.587 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.587 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338151+0000", 2024-09-14T06:17:04.587 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.587 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.587 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.587 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.587 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338151+0000", 2024-09-14T06:17:04.587 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.587 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.587 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.587 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.587 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338151+0000", 2024-09-14T06:17:04.587 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7800000000000002e-07 2024-09-14T06:17:04.588 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.588 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.588 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.588 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338153+0000", 2024-09-14T06:17:04.588 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9690000000000001e-06 2024-09-14T06:17:04.588 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.588 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.588 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.588 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338162+0000", 2024-09-14T06:17:04.588 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6920000000000008e-06 2024-09-14T06:17:04.588 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.588 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.589 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.589 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338168+0000", 2024-09-14T06:17:04.589 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.178e-06 2024-09-14T06:17:04.589 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.589 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.589 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.589 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.589 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.589 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32214 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.589 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336411+0000", 2024-09-14T06:17:04.589 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0014853620000000001, 2024-09-14T06:17:04.589 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011075793, 2024-09-14T06:17:04.590 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.590 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.590 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.590 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.590 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.590 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.590 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32214 2024-09-14T06:17:04.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.590 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.590 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.590 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.590 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336411+0000", 2024-09-14T06:17:04.590 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.590 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.591 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.591 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336411+0000", 2024-09-14T06:17:04.591 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.591 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.591 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.591 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.591 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336412+0000", 2024-09-14T06:17:04.591 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.46e-07 2024-09-14T06:17:04.591 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.591 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.591 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.591 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336413+0000", 2024-09-14T06:17:04.591 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.536e-06 2024-09-14T06:17:04.591 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.592 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.592 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.592 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336420+0000", 2024-09-14T06:17:04.592 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4350000000000002e-06 2024-09-14T06:17:04.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.592 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.592 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.592 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336423+0000", 2024-09-14T06:17:04.592 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.038e-06 2024-09-14T06:17:04.592 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.592 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.592 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.592 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.593 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32246 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.593 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337215+0000", 2024-09-14T06:17:04.593 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00068172399999999998, 2024-09-14T06:17:04.593 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010626218999999999, 2024-09-14T06:17:04.593 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.593 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.593 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.594 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.594 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.594 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.594 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32246 2024-09-14T06:17:04.594 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.594 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.594 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.595 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.595 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337215+0000", 2024-09-14T06:17:04.595 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.595 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.595 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.595 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337215+0000", 2024-09-14T06:17:04.595 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.596 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.596 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.596 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.596 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337215+0000", 2024-09-14T06:17:04.596 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7599999999999997e-07 2024-09-14T06:17:04.596 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.597 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.597 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.597 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337218+0000", 2024-09-14T06:17:04.597 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.695e-06 2024-09-14T06:17:04.597 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.597 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.597 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.598 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337223+0000", 2024-09-14T06:17:04.598 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4800000000000003e-06 2024-09-14T06:17:04.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.598 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.598 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.598 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337226+0000", 2024-09-14T06:17:04.598 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.157e-06 2024-09-14T06:17:04.598 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.598 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.599 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.599 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.599 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.599 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32278 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.599 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338188+0000", 2024-09-14T06:17:04.599 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9997082, 2024-09-14T06:17:04.599 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0098066060000000007, 2024-09-14T06:17:04.599 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.599 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.599 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.599 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.599 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.600 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.600 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32278 2024-09-14T06:17:04.600 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.600 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.600 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.600 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.601 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338188+0000", 2024-09-14T06:17:04.601 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.601 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.601 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338188+0000", 2024-09-14T06:17:04.601 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.601 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.601 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338189+0000", 2024-09-14T06:17:04.601 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4499999999999997e-07 2024-09-14T06:17:04.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.602 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.602 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.602 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338196+0000", 2024-09-14T06:17:04.602 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5299999999999999e-06 2024-09-14T06:17:04.602 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.602 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.602 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.602 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338203+0000", 2024-09-14T06:17:04.602 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9450000000000004e-06 2024-09-14T06:17:04.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.603 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.603 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338208+0000", 2024-09-14T06:17:04.603 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0529999999999997e-06 2024-09-14T06:17:04.603 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.603 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.603 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.603 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32215 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.603 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336428+0000", 2024-09-14T06:17:04.603 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.001468429, 2024-09-14T06:17:04.603 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011675764999999999, 2024-09-14T06:17:04.604 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.604 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.604 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.604 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.604 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.604 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.604 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32215 2024-09-14T06:17:04.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.604 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.604 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.604 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.604 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336428+0000", 2024-09-14T06:17:04.604 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.605 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.605 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336428+0000", 2024-09-14T06:17:04.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.605 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.605 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336429+0000", 2024-09-14T06:17:04.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.41e-07 2024-09-14T06:17:04.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.605 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.605 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336430+0000", 2024-09-14T06:17:04.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1769999999999999e-06 2024-09-14T06:17:04.606 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.606 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.606 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.606 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336437+0000", 2024-09-14T06:17:04.606 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.323e-06 2024-09-14T06:17:04.606 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.606 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.606 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.606 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336442+0000", 2024-09-14T06:17:04.606 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6110000000000004e-06 2024-09-14T06:17:04.606 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.606 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.606 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.606 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.607 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.607 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32247 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.607 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337246+0000", 2024-09-14T06:17:04.607 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00065095300000000001, 2024-09-14T06:17:04.607 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011018276, 2024-09-14T06:17:04.607 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.607 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.607 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.607 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.607 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.607 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.607 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32247 2024-09-14T06:17:04.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.607 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337246+0000", 2024-09-14T06:17:04.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337246+0000", 2024-09-14T06:17:04.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337246+0000", 2024-09-14T06:17:04.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2800000000000003e-07 2024-09-14T06:17:04.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.609 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.609 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.609 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337251+0000", 2024-09-14T06:17:04.609 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6550000000000003e-06 2024-09-14T06:17:04.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.609 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.609 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.609 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337259+0000", 2024-09-14T06:17:04.609 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3550000000000005e-06 2024-09-14T06:17:04.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.609 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.609 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.610 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337262+0000", 2024-09-14T06:17:04.610 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3100000000000001e-06 2024-09-14T06:17:04.610 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.610 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.610 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.610 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.610 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.610 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32279 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.610 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338216+0000", 2024-09-14T06:17:04.610 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9996805, 2024-09-14T06:17:04.610 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010209047000000001, 2024-09-14T06:17:04.610 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.610 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.611 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.611 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.611 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.611 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.611 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32279 2024-09-14T06:17:04.611 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.611 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.611 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.611 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.611 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338216+0000", 2024-09-14T06:17:04.611 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.611 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.611 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.611 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.612 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338216+0000", 2024-09-14T06:17:04.612 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.612 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.612 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.612 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.612 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338217+0000", 2024-09-14T06:17:04.612 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0100000000000005e-07 2024-09-14T06:17:04.612 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.612 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.612 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.612 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338219+0000", 2024-09-14T06:17:04.612 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.978e-06 2024-09-14T06:17:04.612 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.612 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.613 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.613 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338225+0000", 2024-09-14T06:17:04.613 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1870000000000002e-06 2024-09-14T06:17:04.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.613 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.613 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.613 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338231+0000", 2024-09-14T06:17:04.613 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1870000000000002e-06 2024-09-14T06:17:04.613 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.613 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.613 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.613 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.613 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32216 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.614 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336446+0000", 2024-09-14T06:17:04.614 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0014509729999999999, 2024-09-14T06:17:04.614 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.012133494, 2024-09-14T06:17:04.614 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.614 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.614 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.614 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.614 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.614 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.614 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32216 2024-09-14T06:17:04.614 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.614 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.614 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.614 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.615 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336446+0000", 2024-09-14T06:17:04.615 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.615 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.615 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.615 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.615 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336446+0000", 2024-09-14T06:17:04.615 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.615 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.615 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.615 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.615 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336446+0000", 2024-09-14T06:17:04.615 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4999999999999999e-07 2024-09-14T06:17:04.615 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.615 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.616 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.616 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336447+0000", 2024-09-14T06:17:04.616 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1489999999999999e-06 2024-09-14T06:17:04.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.616 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.616 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.616 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336451+0000", 2024-09-14T06:17:04.616 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.121e-06 2024-09-14T06:17:04.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.616 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.616 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.616 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336457+0000", 2024-09-14T06:17:04.616 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.224e-06 2024-09-14T06:17:04.616 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.617 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.617 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.617 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.617 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.617 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32248 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.617 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337268+0000", 2024-09-14T06:17:04.617 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00062898399999999997, 2024-09-14T06:17:04.617 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011468987999999999, 2024-09-14T06:17:04.617 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.617 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.617 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.617 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.617 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.618 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.618 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32248 2024-09-14T06:17:04.618 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.618 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.618 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.618 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.618 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337268+0000", 2024-09-14T06:17:04.618 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.618 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.618 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.618 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.618 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337268+0000", 2024-09-14T06:17:04.619 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.619 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.619 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.619 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.619 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337268+0000", 2024-09-14T06:17:04.619 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.92e-07 2024-09-14T06:17:04.619 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.619 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.619 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.619 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337271+0000", 2024-09-14T06:17:04.619 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.266e-06 2024-09-14T06:17:04.619 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.619 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.619 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.620 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337277+0000", 2024-09-14T06:17:04.620 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0530000000000003e-06 2024-09-14T06:17:04.620 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.620 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.620 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.620 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337282+0000", 2024-09-14T06:17:04.620 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8040000000000001e-06 2024-09-14T06:17:04.620 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.620 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.620 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.620 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.620 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.620 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32280 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.620 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338249+0000", 2024-09-14T06:17:04.621 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9996476, 2024-09-14T06:17:04.621 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010569723, 2024-09-14T06:17:04.621 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.621 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.621 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.621 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.621 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.621 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.621 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32280 2024-09-14T06:17:04.621 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.621 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.621 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.621 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.622 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338249+0000", 2024-09-14T06:17:04.622 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.622 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.622 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.622 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.622 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338249+0000", 2024-09-14T06:17:04.622 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.622 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.622 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.622 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.622 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338250+0000", 2024-09-14T06:17:04.622 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6899999999999998e-07 2024-09-14T06:17:04.622 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.622 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.622 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.623 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338252+0000", 2024-09-14T06:17:04.623 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.137e-06 2024-09-14T06:17:04.623 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.623 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.623 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.623 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338263+0000", 2024-09-14T06:17:04.623 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.184e-05 2024-09-14T06:17:04.623 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.623 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.623 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.623 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338269+0000", 2024-09-14T06:17:04.624 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8590000000000001e-06 2024-09-14T06:17:04.624 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.624 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.624 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.624 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.624 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.624 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32217 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.624 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336479+0000", 2024-09-14T06:17:04.624 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.001417363, 2024-09-14T06:17:04.624 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.012415259999999999, 2024-09-14T06:17:04.624 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.624 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.624 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.624 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.625 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.625 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.625 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32217 2024-09-14T06:17:04.625 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.625 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.625 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.625 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.625 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336479+0000", 2024-09-14T06:17:04.625 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.625 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.625 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.625 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.625 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336479+0000", 2024-09-14T06:17:04.626 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.626 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.626 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.626 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.626 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336480+0000", 2024-09-14T06:17:04.626 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3499999999999995e-07 2024-09-14T06:17:04.626 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.626 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.626 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.626 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336482+0000", 2024-09-14T06:17:04.626 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2630000000000002e-06 2024-09-14T06:17:04.626 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.626 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.626 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.627 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336489+0000", 2024-09-14T06:17:04.627 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6089999999999997e-06 2024-09-14T06:17:04.627 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.627 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.627 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.627 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336494+0000", 2024-09-14T06:17:04.627 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7119999999999999e-06 2024-09-14T06:17:04.627 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.627 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.627 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.627 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.627 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.627 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32249 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.627 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337289+0000", 2024-09-14T06:17:04.627 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00060754500000000003, 2024-09-14T06:17:04.628 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011686645000000001, 2024-09-14T06:17:04.628 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.628 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.628 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.628 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.628 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.628 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.628 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32249 2024-09-14T06:17:04.628 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.628 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.628 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.628 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.628 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337289+0000", 2024-09-14T06:17:04.628 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.629 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.629 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.629 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.629 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337289+0000", 2024-09-14T06:17:04.629 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.629 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.629 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.629 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.629 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337290+0000", 2024-09-14T06:17:04.629 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5600000000000001e-07 2024-09-14T06:17:04.629 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.629 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.630 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.630 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337292+0000", 2024-09-14T06:17:04.630 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9130000000000001e-06 2024-09-14T06:17:04.630 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.630 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.630 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.630 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337297+0000", 2024-09-14T06:17:04.630 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9410000000000004e-06 2024-09-14T06:17:04.630 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.630 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.630 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.630 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337310+0000", 2024-09-14T06:17:04.631 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2625e-05 2024-09-14T06:17:04.631 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.631 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.631 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.631 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.631 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.631 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32281 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.631 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338283+0000", 2024-09-14T06:17:04.631 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9996142, 2024-09-14T06:17:04.631 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010766906, 2024-09-14T06:17:04.631 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.631 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.631 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.631 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.632 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.632 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.632 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32281 2024-09-14T06:17:04.632 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.632 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.632 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.632 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.632 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338283+0000", 2024-09-14T06:17:04.632 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.632 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.632 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.632 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.632 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338283+0000", 2024-09-14T06:17:04.633 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.633 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.633 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.633 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.633 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338283+0000", 2024-09-14T06:17:04.633 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6600000000000002e-07 2024-09-14T06:17:04.633 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.633 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.633 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.633 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338285+0000", 2024-09-14T06:17:04.633 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1299999999999999e-06 2024-09-14T06:17:04.633 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.633 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.633 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.634 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338297+0000", 2024-09-14T06:17:04.634 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2231e-05 2024-09-14T06:17:04.634 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.634 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.634 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.634 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338310+0000", 2024-09-14T06:17:04.634 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2319e-05 2024-09-14T06:17:04.634 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.634 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.635 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.635 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.635 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.635 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32218 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.635 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336530+0000", 2024-09-14T06:17:04.635 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0013666990000000001, 2024-09-14T06:17:04.635 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.012611145000000001, 2024-09-14T06:17:04.635 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.636 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.636 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.636 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.636 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.636 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.636 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32218 2024-09-14T06:17:04.636 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.636 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.636 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.636 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.636 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336530+0000", 2024-09-14T06:17:04.636 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.636 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.637 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.637 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.637 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336530+0000", 2024-09-14T06:17:04.637 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.637 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.637 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.637 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.637 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336530+0000", 2024-09-14T06:17:04.637 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.96e-07 2024-09-14T06:17:04.637 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.637 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.637 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.637 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336532+0000", 2024-09-14T06:17:04.637 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.423e-06 2024-09-14T06:17:04.638 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.638 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.638 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.638 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336538+0000", 2024-09-14T06:17:04.638 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4180000000000002e-06 2024-09-14T06:17:04.639 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.639 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.639 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.640 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336542+0000", 2024-09-14T06:17:04.640 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6459999999999999e-06 2024-09-14T06:17:04.640 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.640 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.640 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.640 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.640 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.640 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32250 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.640 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337333+0000", 2024-09-14T06:17:04.640 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00056387699999999995, 2024-09-14T06:17:04.640 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011930237, 2024-09-14T06:17:04.640 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.640 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.640 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.641 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.641 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.641 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.641 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32250 2024-09-14T06:17:04.641 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.641 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.641 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.641 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.641 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337333+0000", 2024-09-14T06:17:04.641 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.641 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.641 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.641 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.642 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337333+0000", 2024-09-14T06:17:04.642 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.642 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.642 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.642 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.642 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337333+0000", 2024-09-14T06:17:04.642 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8700000000000001e-07 2024-09-14T06:17:04.642 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.642 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.642 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.642 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337335+0000", 2024-09-14T06:17:04.642 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.516e-06 2024-09-14T06:17:04.642 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.643 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.643 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.643 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337345+0000", 2024-09-14T06:17:04.643 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0623e-05 2024-09-14T06:17:04.643 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.643 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.643 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.643 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337350+0000", 2024-09-14T06:17:04.643 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1799999999999998e-06 2024-09-14T06:17:04.643 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.643 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.644 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.644 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.644 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.644 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32282 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.644 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338318+0000", 2024-09-14T06:17:04.644 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.999579, 2024-09-14T06:17:04.644 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011102575, 2024-09-14T06:17:04.644 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.644 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.645 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.645 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.645 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.645 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.645 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32282 2024-09-14T06:17:04.645 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.645 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.645 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.645 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.645 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338318+0000", 2024-09-14T06:17:04.645 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.645 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.646 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.646 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.646 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338318+0000", 2024-09-14T06:17:04.646 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.646 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.646 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.646 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.646 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338318+0000", 2024-09-14T06:17:04.646 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:17:04.646 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.646 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.646 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.646 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338320+0000", 2024-09-14T06:17:04.646 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2019999999999998e-06 2024-09-14T06:17:04.647 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.647 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.647 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.647 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338327+0000", 2024-09-14T06:17:04.647 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.5309999999999998e-06 2024-09-14T06:17:04.647 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.647 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.647 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.647 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338332+0000", 2024-09-14T06:17:04.647 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4920000000000001e-06 2024-09-14T06:17:04.647 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.647 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.647 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.647 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.647 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.648 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32219 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.648 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336546+0000", 2024-09-14T06:17:04.648 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.001350424, 2024-09-14T06:17:04.648 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013024037, 2024-09-14T06:17:04.648 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.648 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.648 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.648 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.648 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.648 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.648 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32219 2024-09-14T06:17:04.648 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.648 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.649 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.649 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.649 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336546+0000", 2024-09-14T06:17:04.649 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.649 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.649 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.649 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.649 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336546+0000", 2024-09-14T06:17:04.649 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.649 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.649 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.649 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.649 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336547+0000", 2024-09-14T06:17:04.649 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3000000000000002e-07 2024-09-14T06:17:04.650 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.650 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.650 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.650 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336548+0000", 2024-09-14T06:17:04.650 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:17:04.650 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.650 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.650 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.650 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336552+0000", 2024-09-14T06:17:04.650 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6489999999999999e-06 2024-09-14T06:17:04.650 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336555+0000", 2024-09-14T06:17:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:17:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32251 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337355+0000", 2024-09-14T06:17:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00054216099999999999, 2024-09-14T06:17:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01238042, 2024-09-14T06:17:04.651 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32251 2024-09-14T06:17:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.652 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337355+0000", 2024-09-14T06:17:04.653 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.653 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.653 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.653 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.653 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337355+0000", 2024-09-14T06:17:04.653 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.653 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.653 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.653 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.653 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337355+0000", 2024-09-14T06:17:04.653 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1300000000000001e-07 2024-09-14T06:17:04.653 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.653 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.654 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.654 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337356+0000", 2024-09-14T06:17:04.654 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2330000000000001e-06 2024-09-14T06:17:04.654 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.654 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.654 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.654 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337364+0000", 2024-09-14T06:17:04.654 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.447e-06 2024-09-14T06:17:04.654 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.655 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.655 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.655 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337367+0000", 2024-09-14T06:17:04.655 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1480000000000002e-06 2024-09-14T06:17:04.655 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.655 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.656 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.656 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.656 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.656 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32283 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.656 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338346+0000", 2024-09-14T06:17:04.656 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9995508, 2024-09-14T06:17:04.656 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01151253, 2024-09-14T06:17:04.656 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.656 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.656 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.657 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.657 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.657 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.657 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32283 2024-09-14T06:17:04.657 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.657 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.657 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.657 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.657 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338346+0000", 2024-09-14T06:17:04.657 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.658 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.658 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.658 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.658 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338346+0000", 2024-09-14T06:17:04.658 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.658 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.658 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.658 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.658 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338347+0000", 2024-09-14T06:17:04.658 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1099999999999996e-07 2024-09-14T06:17:04.658 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.658 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.658 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.658 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338352+0000", 2024-09-14T06:17:04.659 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8499999999999999e-06 2024-09-14T06:17:04.659 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.659 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.659 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.659 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338359+0000", 2024-09-14T06:17:04.659 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8090000000000004e-06 2024-09-14T06:17:04.659 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.659 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.659 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.659 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338364+0000", 2024-09-14T06:17:04.659 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8709999999999996e-06 2024-09-14T06:17:04.659 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.659 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.660 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.660 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.660 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.660 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32220 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.660 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336559+0000", 2024-09-14T06:17:04.660 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0013373619999999999, 2024-09-14T06:17:04.660 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013383211000000001, 2024-09-14T06:17:04.660 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.660 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.660 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.660 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.660 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.660 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.660 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32220 2024-09-14T06:17:04.661 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.661 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.661 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.661 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.661 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336559+0000", 2024-09-14T06:17:04.661 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.661 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.661 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.661 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.661 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336559+0000", 2024-09-14T06:17:04.661 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.661 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.661 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.661 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.662 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336560+0000", 2024-09-14T06:17:04.662 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:17:04.662 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.662 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.662 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.662 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336562+0000", 2024-09-14T06:17:04.662 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:17:04.662 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.662 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.662 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.662 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336569+0000", 2024-09-14T06:17:04.662 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.289e-06 2024-09-14T06:17:04.662 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.662 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.662 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.662 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336572+0000", 2024-09-14T06:17:04.663 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0939999999999999e-06 2024-09-14T06:17:04.663 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.663 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.663 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.663 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.663 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.663 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32252 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.663 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337371+0000", 2024-09-14T06:17:04.663 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00052541199999999995, 2024-09-14T06:17:04.663 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.012644601, 2024-09-14T06:17:04.663 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.663 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.663 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.664 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.664 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.664 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.664 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32252 2024-09-14T06:17:04.664 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.664 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.664 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.664 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.664 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337371+0000", 2024-09-14T06:17:04.664 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.664 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.664 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.664 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.664 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337371+0000", 2024-09-14T06:17:04.665 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.665 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.665 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.665 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.665 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337372+0000", 2024-09-14T06:17:04.665 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2200000000000003e-07 2024-09-14T06:17:04.665 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.665 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.665 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.665 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337374+0000", 2024-09-14T06:17:04.665 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0829999999999998e-06 2024-09-14T06:17:04.665 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.665 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.665 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.666 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337386+0000", 2024-09-14T06:17:04.666 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2261e-05 2024-09-14T06:17:04.666 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.666 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.666 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.666 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337392+0000", 2024-09-14T06:17:04.666 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0170000000000002e-06 2024-09-14T06:17:04.666 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.666 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.666 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.666 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.666 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.666 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32284 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.666 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338378+0000", 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9995184, 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01171637, 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32284 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338378+0000", 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.667 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.668 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.668 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338378+0000", 2024-09-14T06:17:04.668 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.668 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.668 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.668 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.668 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338379+0000", 2024-09-14T06:17:04.668 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.82e-07 2024-09-14T06:17:04.668 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.668 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.668 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.668 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338381+0000", 2024-09-14T06:17:04.668 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0169999999999999e-06 2024-09-14T06:17:04.668 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.668 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.669 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.669 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338393+0000", 2024-09-14T06:17:04.669 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1968e-05 2024-09-14T06:17:04.669 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.669 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.669 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.669 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338398+0000", 2024-09-14T06:17:04.669 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3820000000000003e-06 2024-09-14T06:17:04.669 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.669 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.669 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.669 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.669 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.670 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32221 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.670 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336599+0000", 2024-09-14T06:17:04.670 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0012981729999999999, 2024-09-14T06:17:04.670 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013571642, 2024-09-14T06:17:04.670 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.670 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.670 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.670 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.670 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.670 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.670 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32221 2024-09-14T06:17:04.671 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.671 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.671 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.671 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.671 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336599+0000", 2024-09-14T06:17:04.671 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.671 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.671 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.671 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.671 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336599+0000", 2024-09-14T06:17:04.671 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.671 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.672 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.672 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.672 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336599+0000", 2024-09-14T06:17:04.672 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8800000000000002e-07 2024-09-14T06:17:04.672 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.672 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.672 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.672 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336601+0000", 2024-09-14T06:17:04.672 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0719999999999998e-06 2024-09-14T06:17:04.672 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.672 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.672 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.672 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336610+0000", 2024-09-14T06:17:04.672 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7290000000000006e-06 2024-09-14T06:17:04.672 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.673 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.673 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.673 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336615+0000", 2024-09-14T06:17:04.673 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4399999999999996e-06 2024-09-14T06:17:04.673 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.673 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.673 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.673 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.673 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.673 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32253 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.673 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337399+0000", 2024-09-14T06:17:04.673 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00049719200000000003, 2024-09-14T06:17:04.673 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.012883179, 2024-09-14T06:17:04.673 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.674 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.674 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.674 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.674 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.674 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.674 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32253 2024-09-14T06:17:04.674 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.674 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.674 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.674 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.674 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337399+0000", 2024-09-14T06:17:04.674 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.674 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.674 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.675 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.675 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337399+0000", 2024-09-14T06:17:04.675 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.675 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.675 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.675 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.675 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337400+0000", 2024-09-14T06:17:04.675 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7700000000000005e-07 2024-09-14T06:17:04.675 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.675 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.675 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.676 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337402+0000", 2024-09-14T06:17:04.676 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9240000000000001e-06 2024-09-14T06:17:04.676 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.676 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.676 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.676 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337409+0000", 2024-09-14T06:17:04.676 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1199999999999996e-06 2024-09-14T06:17:04.676 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.676 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.676 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.676 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337414+0000", 2024-09-14T06:17:04.676 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8380000000000001e-06 2024-09-14T06:17:04.676 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.676 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.677 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.677 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.677 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.677 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32285 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.677 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338407+0000", 2024-09-14T06:17:04.677 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9994898, 2024-09-14T06:17:04.677 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.012025310000000001, 2024-09-14T06:17:04.677 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.677 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.677 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.677 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.677 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.678 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.678 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32285 2024-09-14T06:17:04.678 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.678 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.678 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.678 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.678 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338407+0000", 2024-09-14T06:17:04.678 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.678 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.678 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.678 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.678 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338407+0000", 2024-09-14T06:17:04.678 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.678 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.679 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.679 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.679 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338407+0000", 2024-09-14T06:17:04.679 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6699999999999999e-07 2024-09-14T06:17:04.679 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.679 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.679 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.679 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338409+0000", 2024-09-14T06:17:04.679 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0719999999999998e-06 2024-09-14T06:17:04.679 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.679 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.680 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.680 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338416+0000", 2024-09-14T06:17:04.680 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4529999999999999e-06 2024-09-14T06:17:04.680 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.680 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.680 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.680 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338426+0000", 2024-09-14T06:17:04.680 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0206e-05 2024-09-14T06:17:04.680 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.680 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.680 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.680 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.680 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.680 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32222 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.681 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336632+0000", 2024-09-14T06:17:04.681 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0012642339999999999, 2024-09-14T06:17:04.681 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013951684000000001, 2024-09-14T06:17:04.681 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.681 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.681 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.681 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.681 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.681 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.681 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32222 2024-09-14T06:17:04.681 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.681 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.681 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.682 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.682 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336632+0000", 2024-09-14T06:17:04.682 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.682 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.682 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.682 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.682 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336632+0000", 2024-09-14T06:17:04.682 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.682 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.682 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.682 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.683 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336633+0000", 2024-09-14T06:17:04.683 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3500000000000002e-07 2024-09-14T06:17:04.683 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.683 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.683 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.683 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336634+0000", 2024-09-14T06:17:04.683 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3909999999999999e-06 2024-09-14T06:17:04.683 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.683 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.684 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.684 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336640+0000", 2024-09-14T06:17:04.684 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3380000000000004e-06 2024-09-14T06:17:04.684 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.684 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.684 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.684 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336643+0000", 2024-09-14T06:17:04.684 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.083e-06 2024-09-14T06:17:04.684 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.685 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.685 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.685 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.685 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.685 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32254 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.685 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337437+0000", 2024-09-14T06:17:04.685 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00045944300000000003, 2024-09-14T06:17:04.685 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013304465, 2024-09-14T06:17:04.686 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.686 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.686 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.686 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.686 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.686 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.686 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32254 2024-09-14T06:17:04.686 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.686 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.687 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.687 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.687 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337437+0000", 2024-09-14T06:17:04.687 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.687 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.687 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.687 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.687 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337437+0000", 2024-09-14T06:17:04.687 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.688 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.688 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.688 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.688 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337438+0000", 2024-09-14T06:17:04.688 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1600000000000002e-07 2024-09-14T06:17:04.688 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.688 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.688 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.689 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337439+0000", 2024-09-14T06:17:04.689 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.449e-06 2024-09-14T06:17:04.689 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.689 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.689 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.689 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337447+0000", 2024-09-14T06:17:04.689 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6109999999999996e-06 2024-09-14T06:17:04.689 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.689 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.690 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.690 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337454+0000", 2024-09-14T06:17:04.690 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3719999999999999e-06 2024-09-14T06:17:04.690 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.690 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.690 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.690 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.690 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.690 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32286 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.690 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338435+0000", 2024-09-14T06:17:04.690 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9994621, 2024-09-14T06:17:04.690 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.012444536000000001, 2024-09-14T06:17:04.691 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.691 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.691 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.691 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.691 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.691 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.691 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32286 2024-09-14T06:17:04.691 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.691 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.691 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.691 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.691 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338435+0000", 2024-09-14T06:17:04.691 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.692 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.692 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.692 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.692 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338435+0000", 2024-09-14T06:17:04.692 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.692 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.692 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.692 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.692 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338435+0000", 2024-09-14T06:17:04.692 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0299999999999999e-07 2024-09-14T06:17:04.692 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.692 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.692 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.693 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338437+0000", 2024-09-14T06:17:04.693 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0040000000000002e-06 2024-09-14T06:17:04.693 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.693 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.693 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.693 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338444+0000", 2024-09-14T06:17:04.693 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6440000000000003e-06 2024-09-14T06:17:04.693 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.693 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.693 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.693 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338449+0000", 2024-09-14T06:17:04.693 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8500000000000002e-06 2024-09-14T06:17:04.693 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.693 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.694 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.694 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.694 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.694 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32223 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.694 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336653+0000", 2024-09-14T06:17:04.694 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0012441069999999999, 2024-09-14T06:17:04.694 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014312412, 2024-09-14T06:17:04.694 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.694 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.694 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.694 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.694 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.694 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.694 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32223 2024-09-14T06:17:04.695 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.695 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.695 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.695 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.695 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336653+0000", 2024-09-14T06:17:04.695 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.695 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.695 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.695 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.695 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336653+0000", 2024-09-14T06:17:04.695 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.695 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.695 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.695 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.696 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336653+0000", 2024-09-14T06:17:04.696 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9799999999999999e-07 2024-09-14T06:17:04.696 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.696 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.696 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.696 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336659+0000", 2024-09-14T06:17:04.696 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0589999999999999e-06 2024-09-14T06:17:04.696 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.696 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.696 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.696 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336663+0000", 2024-09-14T06:17:04.696 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3680000000000004e-06 2024-09-14T06:17:04.696 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.696 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.697 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.697 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336666+0000", 2024-09-14T06:17:04.697 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9100000000000001e-06 2024-09-14T06:17:04.697 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.697 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.697 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.697 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.697 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.697 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32255 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.697 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337459+0000", 2024-09-14T06:17:04.697 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00043816699999999999, 2024-09-14T06:17:04.697 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013579849, 2024-09-14T06:17:04.697 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.697 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.698 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.698 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.698 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.698 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.698 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32255 2024-09-14T06:17:04.698 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.698 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.698 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.698 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.698 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337459+0000", 2024-09-14T06:17:04.698 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.698 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.698 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.698 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.699 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337459+0000", 2024-09-14T06:17:04.699 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.699 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.699 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.699 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.699 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337459+0000", 2024-09-14T06:17:04.699 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3599999999999999e-07 2024-09-14T06:17:04.699 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.699 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.699 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.699 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337462+0000", 2024-09-14T06:17:04.699 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3009999999999998e-06 2024-09-14T06:17:04.699 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.699 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.700 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.700 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337467+0000", 2024-09-14T06:17:04.700 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9609999999999996e-06 2024-09-14T06:17:04.700 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.700 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.700 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.700 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337470+0000", 2024-09-14T06:17:04.700 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8260000000000001e-06 2024-09-14T06:17:04.700 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.700 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.700 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.700 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.700 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.701 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32287 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.701 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338468+0000", 2024-09-14T06:17:04.701 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9994283, 2024-09-14T06:17:04.701 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.012648856, 2024-09-14T06:17:04.701 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.701 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.701 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.701 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.701 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.701 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.701 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32287 2024-09-14T06:17:04.701 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.701 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.701 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.702 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.702 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338468+0000", 2024-09-14T06:17:04.702 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.702 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.702 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.702 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.702 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338468+0000", 2024-09-14T06:17:04.702 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.702 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.702 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.702 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.702 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338469+0000", 2024-09-14T06:17:04.702 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6499999999999999e-07 2024-09-14T06:17:04.702 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.702 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.703 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.703 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338471+0000", 2024-09-14T06:17:04.703 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1610000000000001e-06 2024-09-14T06:17:04.703 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.703 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.703 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.703 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338483+0000", 2024-09-14T06:17:04.703 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2213e-05 2024-09-14T06:17:04.703 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.703 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.703 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.703 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338488+0000", 2024-09-14T06:17:04.703 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8520000000000003e-06 2024-09-14T06:17:04.703 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.704 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.704 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.704 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.704 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.704 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32224 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.704 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336672+0000", 2024-09-14T06:17:04.704 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0012251790000000001, 2024-09-14T06:17:04.704 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014547127, 2024-09-14T06:17:04.704 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.704 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.704 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.704 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.704 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.704 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.705 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32224 2024-09-14T06:17:04.705 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.705 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.705 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.705 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.705 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336672+0000", 2024-09-14T06:17:04.705 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.705 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.705 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.705 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.705 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336672+0000", 2024-09-14T06:17:04.705 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.705 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.706 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.706 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.706 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336672+0000", 2024-09-14T06:17:04.706 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0800000000000001e-07 2024-09-14T06:17:04.706 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.706 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.706 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.706 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336673+0000", 2024-09-14T06:17:04.706 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.068e-06 2024-09-14T06:17:04.706 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.706 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.706 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.706 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336678+0000", 2024-09-14T06:17:04.706 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3940000000000004e-06 2024-09-14T06:17:04.707 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.707 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.707 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.707 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336689+0000", 2024-09-14T06:17:04.707 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0426e-05 2024-09-14T06:17:04.707 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.707 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.707 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.707 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.707 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.707 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32256 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.707 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337474+0000", 2024-09-14T06:17:04.707 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00042291900000000001, 2024-09-14T06:17:04.707 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013900048999999999, 2024-09-14T06:17:04.707 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.708 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.708 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.708 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.708 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.708 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.708 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32256 2024-09-14T06:17:04.708 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.708 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.708 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.708 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.708 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337474+0000", 2024-09-14T06:17:04.708 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.708 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.708 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.709 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.709 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337474+0000", 2024-09-14T06:17:04.709 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.709 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.709 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.709 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.709 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337474+0000", 2024-09-14T06:17:04.709 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1e-07 2024-09-14T06:17:04.709 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.709 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.709 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.709 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337479+0000", 2024-09-14T06:17:04.709 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2429999999999996e-06 2024-09-14T06:17:04.710 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.710 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.710 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.710 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337491+0000", 2024-09-14T06:17:04.710 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1754e-05 2024-09-14T06:17:04.710 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.710 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.710 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.710 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337497+0000", 2024-09-14T06:17:04.710 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3820000000000001e-06 2024-09-14T06:17:04.710 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.710 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.710 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.711 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.711 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.711 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32288 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.711 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338496+0000", 2024-09-14T06:17:04.711 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9994006, 2024-09-14T06:17:04.711 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013029183, 2024-09-14T06:17:04.711 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.711 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.711 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.711 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.711 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.711 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.711 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32288 2024-09-14T06:17:04.712 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.712 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.712 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.712 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.712 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338496+0000", 2024-09-14T06:17:04.712 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.712 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.712 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.712 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.712 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338496+0000", 2024-09-14T06:17:04.712 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.712 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.712 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.712 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.713 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338497+0000", 2024-09-14T06:17:04.713 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7999999999999996e-07 2024-09-14T06:17:04.713 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.713 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.713 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.713 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338498+0000", 2024-09-14T06:17:04.713 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9640000000000002e-06 2024-09-14T06:17:04.713 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.713 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.713 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.713 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338505+0000", 2024-09-14T06:17:04.713 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6259999999999997e-06 2024-09-14T06:17:04.713 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.713 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.714 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.714 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338511+0000", 2024-09-14T06:17:04.714 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4739999999999996e-06 2024-09-14T06:17:04.714 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.714 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.714 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.714 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.714 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.714 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32225 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.714 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336696+0000", 2024-09-14T06:17:04.714 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.001200935, 2024-09-14T06:17:04.714 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014992164000000001, 2024-09-14T06:17:04.714 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.714 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.715 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.715 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.715 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.715 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.715 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32225 2024-09-14T06:17:04.715 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.715 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.715 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.715 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.715 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336696+0000", 2024-09-14T06:17:04.715 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.715 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.715 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.715 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.715 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336696+0000", 2024-09-14T06:17:04.716 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.716 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.716 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.716 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.716 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336696+0000", 2024-09-14T06:17:04.716 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0100000000000004e-07 2024-09-14T06:17:04.716 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.716 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.716 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.716 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336698+0000", 2024-09-14T06:17:04.716 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9250000000000002e-06 2024-09-14T06:17:04.716 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.716 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.716 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.717 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336705+0000", 2024-09-14T06:17:04.717 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9700000000000002e-06 2024-09-14T06:17:04.717 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.717 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.717 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.717 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336712+0000", 2024-09-14T06:17:04.717 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6150000000000002e-06 2024-09-14T06:17:04.717 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.717 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.717 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.717 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.717 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.717 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32257 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.717 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337551+0000", 2024-09-14T06:17:04.718 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00034533600000000002, 2024-09-14T06:17:04.718 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014282835000000001, 2024-09-14T06:17:04.718 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.718 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.718 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.718 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.718 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.718 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.718 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32257 2024-09-14T06:17:04.718 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.718 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.718 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.718 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.718 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337551+0000", 2024-09-14T06:17:04.719 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.719 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.719 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.719 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.719 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337551+0000", 2024-09-14T06:17:04.719 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.719 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.719 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.719 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.719 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337552+0000", 2024-09-14T06:17:04.719 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:17:04.719 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.719 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.719 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.720 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337553+0000", 2024-09-14T06:17:04.720 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.522e-06 2024-09-14T06:17:04.720 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.720 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.720 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.720 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337559+0000", 2024-09-14T06:17:04.720 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.5269999999999998e-06 2024-09-14T06:17:04.720 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.720 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.720 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.720 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337563+0000", 2024-09-14T06:17:04.720 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8530000000000002e-06 2024-09-14T06:17:04.720 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.721 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.721 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.721 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.721 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.721 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32289 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.721 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338529+0000", 2024-09-14T06:17:04.721 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9993672, 2024-09-14T06:17:04.721 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013387929999999999, 2024-09-14T06:17:04.721 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.721 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.721 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.721 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.721 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.721 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.722 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32289 2024-09-14T06:17:04.722 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.722 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.722 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.722 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.722 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338529+0000", 2024-09-14T06:17:04.722 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.722 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.722 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.722 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.722 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338529+0000", 2024-09-14T06:17:04.722 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.722 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.722 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.723 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.723 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338530+0000", 2024-09-14T06:17:04.723 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5499999999999998e-07 2024-09-14T06:17:04.723 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.723 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.723 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.723 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338538+0000", 2024-09-14T06:17:04.723 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0290000000000005e-06 2024-09-14T06:17:04.723 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.723 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.723 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.723 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338545+0000", 2024-09-14T06:17:04.723 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0439999999999998e-06 2024-09-14T06:17:04.723 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.723 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.724 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.724 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338551+0000", 2024-09-14T06:17:04.724 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8379999999999998e-06 2024-09-14T06:17:04.724 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.724 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.724 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.724 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.724 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.724 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32226 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.724 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336718+0000", 2024-09-14T06:17:04.724 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.001178375, 2024-09-14T06:17:04.724 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015275103, 2024-09-14T06:17:04.724 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.724 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.725 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.725 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.725 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.725 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.725 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32226 2024-09-14T06:17:04.725 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.725 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.725 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.725 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.725 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336718+0000", 2024-09-14T06:17:04.725 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.725 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.725 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.726 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.726 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336718+0000", 2024-09-14T06:17:04.726 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.726 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.726 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.726 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.726 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336719+0000", 2024-09-14T06:17:04.726 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4499999999999997e-07 2024-09-14T06:17:04.726 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.726 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.726 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.726 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336721+0000", 2024-09-14T06:17:04.726 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7689999999999999e-06 2024-09-14T06:17:04.726 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.726 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.727 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.727 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336726+0000", 2024-09-14T06:17:04.727 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6829999999999996e-06 2024-09-14T06:17:04.727 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.727 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.727 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.727 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336733+0000", 2024-09-14T06:17:04.727 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3230000000000003e-06 2024-09-14T06:17:04.727 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.727 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.727 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.727 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.727 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.727 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32258 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.728 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337568+0000", 2024-09-14T06:17:04.728 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00032852700000000001, 2024-09-14T06:17:04.728 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014501239000000001, 2024-09-14T06:17:04.728 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.728 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.728 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.728 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.728 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.728 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.728 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32258 2024-09-14T06:17:04.728 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.728 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.728 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.728 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.729 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337568+0000", 2024-09-14T06:17:04.729 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.729 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.729 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.729 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.729 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337568+0000", 2024-09-14T06:17:04.729 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.729 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.729 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.729 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.729 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337569+0000", 2024-09-14T06:17:04.729 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0699999999999996e-07 2024-09-14T06:17:04.729 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.729 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.730 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.730 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337571+0000", 2024-09-14T06:17:04.730 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.148e-06 2024-09-14T06:17:04.730 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.730 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.730 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.730 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337582+0000", 2024-09-14T06:17:04.730 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1321000000000001e-05 2024-09-14T06:17:04.730 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.730 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.730 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.730 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337585+0000", 2024-09-14T06:17:04.730 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1549999999999999e-06 2024-09-14T06:17:04.730 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.731 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.731 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.731 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.731 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.731 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32290 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.731 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338560+0000", 2024-09-14T06:17:04.731 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9993367, 2024-09-14T06:17:04.731 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013588816, 2024-09-14T06:17:04.731 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.731 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.731 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.731 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.731 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.731 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.731 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32290 2024-09-14T06:17:04.732 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.732 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.732 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.732 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.732 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338560+0000", 2024-09-14T06:17:04.732 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.732 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.732 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.732 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.732 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338560+0000", 2024-09-14T06:17:04.732 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.732 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.732 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.732 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.733 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338560+0000", 2024-09-14T06:17:04.733 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8800000000000003e-07 2024-09-14T06:17:04.733 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.733 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.733 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.733 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338562+0000", 2024-09-14T06:17:04.733 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0080000000000001e-06 2024-09-14T06:17:04.733 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.733 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.733 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.733 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338569+0000", 2024-09-14T06:17:04.733 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2670000000000003e-06 2024-09-14T06:17:04.733 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.734 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.734 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.734 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338579+0000", 2024-09-14T06:17:04.734 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9690000000000006e-06 2024-09-14T06:17:04.734 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.734 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.734 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.734 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.734 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.734 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32227 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.735 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336739+0000", 2024-09-14T06:17:04.735 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.001157651, 2024-09-14T06:17:04.735 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015579548, 2024-09-14T06:17:04.735 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.735 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.735 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.735 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.735 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.735 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.735 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32227 2024-09-14T06:17:04.735 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.735 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.735 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.736 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.736 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336739+0000", 2024-09-14T06:17:04.736 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.736 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.736 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.736 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.736 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336739+0000", 2024-09-14T06:17:04.736 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.736 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.736 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.736 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.736 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336740+0000", 2024-09-14T06:17:04.736 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4499999999999997e-07 2024-09-14T06:17:04.736 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.737 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.737 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.737 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336741+0000", 2024-09-14T06:17:04.737 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4339999999999999e-06 2024-09-14T06:17:04.737 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.737 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.737 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.737 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336746+0000", 2024-09-14T06:17:04.737 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0819999999999998e-06 2024-09-14T06:17:04.737 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.737 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.737 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.737 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336750+0000", 2024-09-14T06:17:04.737 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6289999999999999e-06 2024-09-14T06:17:04.738 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.738 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.738 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.738 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.738 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.738 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32259 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.738 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337602+0000", 2024-09-14T06:17:04.738 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00029468700000000002, 2024-09-14T06:17:04.738 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014865201, 2024-09-14T06:17:04.738 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.738 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.738 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.738 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.738 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.739 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.739 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32259 2024-09-14T06:17:04.739 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.739 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.739 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.739 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.739 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337602+0000", 2024-09-14T06:17:04.739 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.739 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.739 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.739 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.739 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337602+0000", 2024-09-14T06:17:04.739 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.739 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.740 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.740 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.740 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337603+0000", 2024-09-14T06:17:04.740 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1900000000000003e-07 2024-09-14T06:17:04.740 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.740 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.740 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.740 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337608+0000", 2024-09-14T06:17:04.740 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.93e-06 2024-09-14T06:17:04.740 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.740 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.740 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.740 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337615+0000", 2024-09-14T06:17:04.740 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6120000000000004e-06 2024-09-14T06:17:04.741 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.741 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.741 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.741 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337620+0000", 2024-09-14T06:17:04.741 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9679999999999997e-06 2024-09-14T06:17:04.741 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.741 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.741 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.741 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.741 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.741 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32291 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.741 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338594+0000", 2024-09-14T06:17:04.741 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9993029, 2024-09-14T06:17:04.741 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0140372, 2024-09-14T06:17:04.742 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.742 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.742 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.742 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.742 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.742 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.742 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32291 2024-09-14T06:17:04.742 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.742 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.742 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.742 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.742 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338594+0000", 2024-09-14T06:17:04.742 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.742 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.743 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.743 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.743 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338594+0000", 2024-09-14T06:17:04.743 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.743 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.743 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.743 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.743 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338595+0000", 2024-09-14T06:17:04.743 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8599999999999997e-07 2024-09-14T06:17:04.743 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.743 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.743 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.743 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338597+0000", 2024-09-14T06:17:04.743 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2180000000000002e-06 2024-09-14T06:17:04.744 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.744 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.744 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.744 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338609+0000", 2024-09-14T06:17:04.744 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2105e-05 2024-09-14T06:17:04.744 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.744 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.744 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.744 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338614+0000", 2024-09-14T06:17:04.744 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1970000000000004e-06 2024-09-14T06:17:04.744 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.744 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.744 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.744 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.745 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.745 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32228 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.745 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336761+0000", 2024-09-14T06:17:04.745 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.001136153, 2024-09-14T06:17:04.745 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016023445000000001, 2024-09-14T06:17:04.745 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.745 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.745 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.745 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.745 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.745 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.745 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32228 2024-09-14T06:17:04.745 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.745 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.745 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.746 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.746 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336761+0000", 2024-09-14T06:17:04.746 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.746 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.746 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.746 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.746 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336761+0000", 2024-09-14T06:17:04.746 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.746 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.746 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.746 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.746 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336761+0000", 2024-09-14T06:17:04.746 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:17:04.747 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.747 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.747 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.747 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336762+0000", 2024-09-14T06:17:04.747 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.274e-06 2024-09-14T06:17:04.747 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.747 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.747 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.747 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336768+0000", 2024-09-14T06:17:04.747 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9830000000000001e-06 2024-09-14T06:17:04.747 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.747 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.747 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.747 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336771+0000", 2024-09-14T06:17:04.747 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0089999999999999e-06 2024-09-14T06:17:04.748 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.748 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.748 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.748 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.748 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.748 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32260 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.748 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337629+0000", 2024-09-14T06:17:04.748 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.00026734399999999998, 2024-09-14T06:17:04.748 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015238069, 2024-09-14T06:17:04.748 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.748 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.748 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.748 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.748 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.749 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.749 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32260 2024-09-14T06:17:04.749 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.749 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.749 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.749 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.749 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337629+0000", 2024-09-14T06:17:04.749 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.749 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.749 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.749 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.749 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337629+0000", 2024-09-14T06:17:04.749 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.749 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.750 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.750 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.750 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337630+0000", 2024-09-14T06:17:04.750 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5600000000000001e-07 2024-09-14T06:17:04.750 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.750 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.750 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.750 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337632+0000", 2024-09-14T06:17:04.750 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0329999999999998e-06 2024-09-14T06:17:04.750 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.750 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.750 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.750 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337637+0000", 2024-09-14T06:17:04.750 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.04e-06 2024-09-14T06:17:04.750 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.751 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.751 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.751 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337641+0000", 2024-09-14T06:17:04.751 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1819999999999999e-06 2024-09-14T06:17:04.751 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.751 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.751 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.751 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.751 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.751 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32292 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.751 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338622+0000", 2024-09-14T06:17:04.751 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9992743, 2024-09-14T06:17:04.751 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014326675000000001, 2024-09-14T06:17:04.752 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.752 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.752 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.752 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.752 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.752 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.752 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32292 2024-09-14T06:17:04.752 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.752 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.752 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.752 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.752 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338622+0000", 2024-09-14T06:17:04.752 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.752 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.753 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.753 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.753 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338622+0000", 2024-09-14T06:17:04.753 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.753 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.753 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.753 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.753 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338623+0000", 2024-09-14T06:17:04.753 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9599999999999999e-07 2024-09-14T06:17:04.753 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.753 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.753 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.753 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338625+0000", 2024-09-14T06:17:04.753 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2560000000000001e-06 2024-09-14T06:17:04.753 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.754 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.754 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.754 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338630+0000", 2024-09-14T06:17:04.754 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.305e-06 2024-09-14T06:17:04.754 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.754 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.754 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.754 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338633+0000", 2024-09-14T06:17:04.754 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9790000000000002e-06 2024-09-14T06:17:04.754 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.754 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.754 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.754 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.754 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.755 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32229 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.755 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336776+0000", 2024-09-14T06:17:04.755 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.001120825, 2024-09-14T06:17:04.755 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016248643, 2024-09-14T06:17:04.755 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.755 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.755 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.755 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.755 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.755 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.755 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32229 2024-09-14T06:17:04.755 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.755 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.755 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.755 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.756 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336776+0000", 2024-09-14T06:17:04.756 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.756 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.756 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.756 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.756 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336776+0000", 2024-09-14T06:17:04.756 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.756 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.756 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.756 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.756 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336776+0000", 2024-09-14T06:17:04.756 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4200000000000002e-07 2024-09-14T06:17:04.756 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.756 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.757 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.757 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336777+0000", 2024-09-14T06:17:04.757 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.048e-06 2024-09-14T06:17:04.757 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.757 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.757 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.757 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336782+0000", 2024-09-14T06:17:04.757 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4089999999999996e-06 2024-09-14T06:17:04.757 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.757 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.757 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.757 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336784+0000", 2024-09-14T06:17:04.757 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7700000000000002e-06 2024-09-14T06:17:04.758 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.758 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.758 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.758 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.758 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.758 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32261 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.758 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337652+0000", 2024-09-14T06:17:04.758 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.000244949, 2024-09-14T06:17:04.758 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015452169999999999, 2024-09-14T06:17:04.758 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.758 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.758 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.758 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.759 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.759 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.759 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32261 2024-09-14T06:17:04.759 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.759 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.759 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.759 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.759 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337652+0000", 2024-09-14T06:17:04.759 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.759 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.759 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.759 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.759 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337652+0000", 2024-09-14T06:17:04.759 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.759 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.759 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.760 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.760 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337652+0000", 2024-09-14T06:17:04.760 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0699999999999998e-07 2024-09-14T06:17:04.760 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.760 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.760 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.760 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337653+0000", 2024-09-14T06:17:04.760 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2810000000000001e-06 2024-09-14T06:17:04.760 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.760 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.760 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.760 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337659+0000", 2024-09-14T06:17:04.760 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4199999999999998e-06 2024-09-14T06:17:04.760 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.761 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.761 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.761 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337663+0000", 2024-09-14T06:17:04.761 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1980000000000003e-06 2024-09-14T06:17:04.761 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.761 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.761 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.761 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.761 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.761 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32293 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.761 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338656+0000", 2024-09-14T06:17:04.761 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.9992409, 2024-09-14T06:17:04.761 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014589598, 2024-09-14T06:17:04.762 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.762 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.762 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.762 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.762 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.762 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.762 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32293 2024-09-14T06:17:04.762 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.762 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.762 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.762 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.762 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338656+0000", 2024-09-14T06:17:04.762 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.763 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.763 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.763 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.763 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338656+0000", 2024-09-14T06:17:04.763 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.763 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.763 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.763 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.763 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338656+0000", 2024-09-14T06:17:04.763 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3999999999999997e-07 2024-09-14T06:17:04.763 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.763 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.763 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.764 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338657+0000", 2024-09-14T06:17:04.764 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4109999999999999e-06 2024-09-14T06:17:04.764 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.764 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.764 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.764 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338662+0000", 2024-09-14T06:17:04.764 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9429999999999999e-06 2024-09-14T06:17:04.764 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.764 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.764 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.764 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338665+0000", 2024-09-14T06:17:04.764 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0860000000000002e-06 2024-09-14T06:17:04.764 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.764 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.765 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.765 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.765 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.765 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32230 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.765 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.336789+0000", 2024-09-14T06:17:04.765 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0011078609999999999, 2024-09-14T06:17:04.765 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016616618, 2024-09-14T06:17:04.765 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.765 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.765 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.765 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.765 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.765 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.765 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32230 2024-09-14T06:17:04.766 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.766 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.766 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.766 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.766 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336789+0000", 2024-09-14T06:17:04.766 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.766 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.766 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.766 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.766 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336789+0000", 2024-09-14T06:17:04.766 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.766 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.766 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.767 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.767 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336790+0000", 2024-09-14T06:17:04.767 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1500000000000003e-07 2024-09-14T06:17:04.767 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.767 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.767 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.767 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336791+0000", 2024-09-14T06:17:04.767 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.658e-06 2024-09-14T06:17:04.767 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.767 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.767 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.767 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336797+0000", 2024-09-14T06:17:04.767 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0430000000000004e-06 2024-09-14T06:17:04.767 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.768 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.768 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.768 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.336803+0000", 2024-09-14T06:17:04.768 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1649999999999997e-06 2024-09-14T06:17:04.768 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.768 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.768 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.768 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.768 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.768 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32262 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.768 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.337667+0000", 2024-09-14T06:17:04.768 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.000229681, 2024-09-14T06:17:04.768 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015887992, 2024-09-14T06:17:04.768 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.769 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.769 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.769 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.769 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.769 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.769 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32262 2024-09-14T06:17:04.769 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.769 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.769 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.769 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.769 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337667+0000", 2024-09-14T06:17:04.769 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.769 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.769 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.770 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.770 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337667+0000", 2024-09-14T06:17:04.770 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.770 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.770 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.770 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.770 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337667+0000", 2024-09-14T06:17:04.770 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3999999999999998e-07 2024-09-14T06:17:04.770 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.770 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.770 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.770 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337668+0000", 2024-09-14T06:17:04.770 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.218e-06 2024-09-14T06:17:04.771 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.771 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.771 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.771 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337672+0000", 2024-09-14T06:17:04.771 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5449999999999999e-06 2024-09-14T06:17:04.771 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.771 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.771 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.771 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.337675+0000", 2024-09-14T06:17:04.771 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.864e-06 2024-09-14T06:17:04.771 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.771 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.771 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.771 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.772 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.772 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32294 3.1b 3.c3e12f9b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T06:17:04.772 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:04.338670+0000", 2024-09-14T06:17:04.772 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 4294967295.999227, 2024-09-14T06:17:04.772 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015043539, 2024-09-14T06:17:04.772 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:04.772 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:04.772 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "queued for pg", 2024-09-14T06:17:04.772 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:04.772 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:04.772 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:04.772 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32294 2024-09-14T06:17:04.772 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.772 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:04.773 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.773 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:04.773 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338670+0000", 2024-09-14T06:17:04.773 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.773 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.773 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.773 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:04.773 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338670+0000", 2024-09-14T06:17:04.773 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:04.773 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.773 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.773 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:04.773 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338670+0000", 2024-09-14T06:17:04.773 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:17:04.774 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.774 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.774 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:04.774 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338671+0000", 2024-09-14T06:17:04.774 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1149999999999999e-06 2024-09-14T06:17:04.774 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.774 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.774 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:04.774 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338675+0000", 2024-09-14T06:17:04.774 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5760000000000002e-06 2024-09-14T06:17:04.774 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:04.774 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:04.774 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:04.774 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:04.338677+0000", 2024-09-14T06:17:04.774 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6699999999999998e-06 2024-09-14T06:17:04.775 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.775 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:04.775 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.775 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:04.775 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-09-14T06:17:04.775 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 118 2024-09-14T06:17:04.775 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:04.776 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-14T06:17:04.778 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:17:04.778 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:04.774+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:04.778 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:04.778 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:04.778 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 59, 2024-09-14T06:17:04.778 INFO:teuthology.orchestra.run.smithi039.stdout: "fsid": "038563b1-268f-461f-b6a6-4b4b41ad0236", 2024-09-14T06:17:04.779 INFO:teuthology.orchestra.run.smithi039.stdout: "created": "2024-09-14T06:15:49.493035+0000", 2024-09-14T06:17:04.779 INFO:teuthology.orchestra.run.smithi039.stdout: "modified": "2024-09-14T06:17:04.200986+0000", 2024-09-14T06:17:04.779 INFO:teuthology.orchestra.run.smithi039.stdout: "last_up_change": "2024-09-14T06:16:02.014989+0000", 2024-09-14T06:17:04.779 INFO:teuthology.orchestra.run.smithi039.stdout: "last_in_change": "2024-09-14T06:16:26.221200+0000", 2024-09-14T06:17:04.779 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T06:17:04.779 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_num": 5804032, 2024-09-14T06:17:04.779 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_set": [ 2024-09-14T06:17:04.779 INFO:teuthology.orchestra.run.smithi039.stdout: "nodeep-scrub", 2024-09-14T06:17:04.779 INFO:teuthology.orchestra.run.smithi039.stdout: "pglog_hardlimit", 2024-09-14T06:17:04.779 INFO:teuthology.orchestra.run.smithi039.stdout: "purged_snapdirs", 2024-09-14T06:17:04.779 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_deletes", 2024-09-14T06:17:04.779 INFO:teuthology.orchestra.run.smithi039.stdout: "sortbitwise" 2024-09-14T06:17:04.779 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:17:04.780 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_version": 5, 2024-09-14T06:17:04.780 INFO:teuthology.orchestra.run.smithi039.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T06:17:04.780 INFO:teuthology.orchestra.run.smithi039.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T06:17:04.780 INFO:teuthology.orchestra.run.smithi039.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T06:17:04.780 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_snapshot": "", 2024-09-14T06:17:04.780 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_max": 3, 2024-09-14T06:17:04.780 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd": 8, 2024-09-14T06:17:04.780 INFO:teuthology.orchestra.run.smithi039.stdout: "require_min_compat_client": "luminous", 2024-09-14T06:17:04.780 INFO:teuthology.orchestra.run.smithi039.stdout: "min_compat_client": "luminous", 2024-09-14T06:17:04.780 INFO:teuthology.orchestra.run.smithi039.stdout: "require_osd_release": "squid", 2024-09-14T06:17:04.780 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_crimson": false, 2024-09-14T06:17:04.780 INFO:teuthology.orchestra.run.smithi039.stdout: "pools": [ 2024-09-14T06:17:04.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.781 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 1, 2024-09-14T06:17:04.781 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": ".mgr", 2024-09-14T06:17:04.781 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-09-14T06:16:02.429225+0000", 2024-09-14T06:17:04.781 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-14T06:17:04.781 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool", 2024-09-14T06:17:04.781 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-09-14T06:17:04.781 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-09-14T06:17:04.781 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-09-14T06:17:04.781 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-09-14T06:17:04.781 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:17:04.781 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:17:04.781 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:17:04.781 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:17:04.781 INFO:teuthology.orchestra.run.smithi039.stdout: "is_stretch_pool": false, 2024-09-14T06:17:04.782 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-09-14T06:17:04.782 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:17:04.782 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 1, 2024-09-14T06:17:04.782 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 1, 2024-09-14T06:17:04.782 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 1, 2024-09-14T06:17:04.782 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 1, 2024-09-14T06:17:04.782 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 1, 2024-09-14T06:17:04.782 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-09-14T06:17:04.782 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-09-14T06:17:04.782 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-09-14T06:17:04.782 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-09-14T06:17:04.782 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-09-14T06:17:04.782 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-09-14T06:17:04.782 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-09-14T06:17:04.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.783 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "18", 2024-09-14T06:17:04.783 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-09-14T06:17:04.783 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:17:04.783 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:17:04.783 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-09-14T06:17:04.783 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:17:04.783 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 0, 2024-09-14T06:17:04.783 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 0, 2024-09-14T06:17:04.783 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-09-14T06:17:04.783 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-09-14T06:17:04.783 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-09-14T06:17:04.783 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-09-14T06:17:04.783 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-09-14T06:17:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-09-14T06:17:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-09-14T06:17:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-09-14T06:17:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-09-14T06:17:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-09-14T06:17:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-09-14T06:17:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:17:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:17:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:17:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-09-14T06:17:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-09-14T06:17:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-09-14T06:17:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-09-14T06:17:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-09-14T06:17:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-09-14T06:17:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-09-14T06:17:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-09-14T06:17:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:17:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:17:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:17:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:17:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-09-14T06:17:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-09-14T06:17:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-09-14T06:17:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-09-14T06:17:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "options": { 2024-09-14T06:17:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_max": 32, 2024-09-14T06:17:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_min": 1 2024-09-14T06:17:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-09-14T06:17:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": {} 2024-09-14T06:17:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-09-14T06:17:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "score_type": "Fair distribution", 2024-09-14T06:17:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 6.9000000953674316, 2024-09-14T06:17:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 6.9000000953674316, 2024-09-14T06:17:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T06:17:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 2, 2024-09-14T06:17:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 2, 2024-09-14T06:17:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:17:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 1, 2024-09-14T06:17:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:17:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-14T06:17:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": "rbd", 2024-09-14T06:17:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-09-14T06:16:08.129454+0000", 2024-09-14T06:17:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 8193, 2024-09-14T06:17:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:17:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-09-14T06:17:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-09-14T06:17:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-09-14T06:17:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-09-14T06:17:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:17:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:17:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:17:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:17:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "is_stretch_pool": false, 2024-09-14T06:17:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-09-14T06:17:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:17:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 8, 2024-09-14T06:17:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 8, 2024-09-14T06:17:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 8, 2024-09-14T06:17:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 8, 2024-09-14T06:17:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 8, 2024-09-14T06:17:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-09-14T06:17:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-09-14T06:17:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-09-14T06:17:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-09-14T06:17:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-09-14T06:17:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-09-14T06:17:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-09-14T06:17:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "22", 2024-09-14T06:17:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-09-14T06:17:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:17:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:17:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-09-14T06:17:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:17:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 2, 2024-09-14T06:17:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 22, 2024-09-14T06:17:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-09-14T06:17:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-09-14T06:17:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-09-14T06:17:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-09-14T06:17:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-09-14T06:17:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-09-14T06:17:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-09-14T06:17:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-09-14T06:17:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-09-14T06:17:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-09-14T06:17:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-09-14T06:17:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:17:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:17:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:17:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-09-14T06:17:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-09-14T06:17:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-09-14T06:17:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-09-14T06:17:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-09-14T06:17:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-09-14T06:17:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-09-14T06:17:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-09-14T06:17:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:17:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:17:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:17:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:17:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-09-14T06:17:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-09-14T06:17:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-09-14T06:17:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-09-14T06:17:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "options": {}, 2024-09-14T06:17:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-09-14T06:17:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd": {} 2024-09-14T06:17:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-09-14T06:17:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "score_type": "Fair distribution", 2024-09-14T06:17:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 4.380000114440918, 2024-09-14T06:17:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 4.380000114440918, 2024-09-14T06:17:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 1, 2024-09-14T06:17:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 4.380000114440918, 2024-09-14T06:17:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 4.380000114440918, 2024-09-14T06:17:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:17:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 1, 2024-09-14T06:17:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:17:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 3, 2024-09-14T06:17:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": "pool_client.0", 2024-09-14T06:17:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-09-14T06:16:18.928556+0000", 2024-09-14T06:17:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 8193, 2024-09-14T06:17:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:17:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-09-14T06:17:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-09-14T06:17:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-09-14T06:17:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-09-14T06:17:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:17:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:17:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:17:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:17:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "is_stretch_pool": false, 2024-09-14T06:17:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-09-14T06:17:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:17:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 32, 2024-09-14T06:17:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 32, 2024-09-14T06:17:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 32, 2024-09-14T06:17:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 32, 2024-09-14T06:17:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 32, 2024-09-14T06:17:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-09-14T06:17:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-09-14T06:17:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-09-14T06:17:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-09-14T06:17:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-09-14T06:17:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-09-14T06:17:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-09-14T06:17:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "59", 2024-09-14T06:17:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-09-14T06:17:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:17:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:17:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-09-14T06:17:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:17:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 20, 2024-09-14T06:17:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 59, 2024-09-14T06:17:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-09-14T06:17:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-09-14T06:17:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-09-14T06:17:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-09-14T06:17:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-09-14T06:17:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-09-14T06:17:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-09-14T06:17:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-09-14T06:17:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-09-14T06:17:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-09-14T06:17:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-09-14T06:17:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:17:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:17:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:17:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-09-14T06:17:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-09-14T06:17:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-09-14T06:17:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-09-14T06:17:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-09-14T06:17:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-09-14T06:17:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-09-14T06:17:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-09-14T06:17:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:17:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:17:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:17:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:17:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-09-14T06:17:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-09-14T06:17:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-09-14T06:17:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-09-14T06:17:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "options": {}, 2024-09-14T06:17:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-09-14T06:17:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd": {} 2024-09-14T06:17:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-09-14T06:17:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "score_type": "Fair distribution", 2024-09-14T06:17:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 1.5399999618530273, 2024-09-14T06:17:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 1.5399999618530273, 2024-09-14T06:17:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 1.1399999856948853, 2024-09-14T06:17:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 1.75, 2024-09-14T06:17:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 1.75, 2024-09-14T06:17:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:17:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 1, 2024-09-14T06:17:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:17:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:17:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "osds": [ 2024-09-14T06:17:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 0, 2024-09-14T06:17:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "c669ea76-ed19-4c12-9e56-d79d62846f95", 2024-09-14T06:17:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:17:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:17:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:17:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:17:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:17:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:17:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:17:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 34, 2024-09-14T06:17:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:17:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:17:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:17:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6808", 2024-09-14T06:17:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:17:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6809", 2024-09-14T06:17:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:17:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:17:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6810", 2024-09-14T06:17:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:17:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6811", 2024-09-14T06:17:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:17:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:17:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6814", 2024-09-14T06:17:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:17:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6815", 2024-09-14T06:17:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:17:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:17:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6812", 2024-09-14T06:17:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:17:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6813", 2024-09-14T06:17:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:17:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6809/1157559201", 2024-09-14T06:17:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6811/1157559201", 2024-09-14T06:17:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6815/1157559201", 2024-09-14T06:17:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6813/1157559201", 2024-09-14T06:17:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:17:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:17:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:17:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 1, 2024-09-14T06:17:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b", 2024-09-14T06:17:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:17:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:17:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:17:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:17:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:17:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:17:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:17:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 37, 2024-09-14T06:17:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:17:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:17:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:17:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6816", 2024-09-14T06:17:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:17:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6817", 2024-09-14T06:17:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:17:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:17:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6818", 2024-09-14T06:17:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:17:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6819", 2024-09-14T06:17:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:17:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:17:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6822", 2024-09-14T06:17:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:17:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6823", 2024-09-14T06:17:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:17:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:17:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6820", 2024-09-14T06:17:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:17:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6821", 2024-09-14T06:17:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:17:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6817/4062089212", 2024-09-14T06:17:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6819/4062089212", 2024-09-14T06:17:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6823/4062089212", 2024-09-14T06:17:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6821/4062089212", 2024-09-14T06:17:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:17:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:17:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:17:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 2, 2024-09-14T06:17:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "9bda988f-e615-4545-849b-318c408c4c42", 2024-09-14T06:17:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:17:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:17:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:17:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 0, 2024-09-14T06:17:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:17:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:17:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:17:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 29, 2024-09-14T06:17:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:17:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:17:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:17:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6824", 2024-09-14T06:17:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:17:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6825", 2024-09-14T06:17:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:17:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:17:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6826", 2024-09-14T06:17:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:17:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6827", 2024-09-14T06:17:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:17:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:17:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6830", 2024-09-14T06:17:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:17:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6831", 2024-09-14T06:17:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:17:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:17:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6828", 2024-09-14T06:17:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:17:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6829", 2024-09-14T06:17:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:17:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6825/1724773882", 2024-09-14T06:17:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6827/1724773882", 2024-09-14T06:17:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6831/1724773882", 2024-09-14T06:17:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6829/1724773882", 2024-09-14T06:17:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:17:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:17:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:17:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 3, 2024-09-14T06:17:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "8ea6cd69-2ef1-42ea-b339-7394bf8f4a87", 2024-09-14T06:17:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:17:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:17:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:17:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:17:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:17:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:17:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:17:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 34, 2024-09-14T06:17:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:17:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:17:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:17:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6800", 2024-09-14T06:17:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:17:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6801", 2024-09-14T06:17:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:17:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:17:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6802", 2024-09-14T06:17:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:17:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6803", 2024-09-14T06:17:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:17:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:17:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6806", 2024-09-14T06:17:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:17:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6807", 2024-09-14T06:17:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:17:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:17:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6804", 2024-09-14T06:17:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:17:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6805", 2024-09-14T06:17:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:17:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6801/3775031293", 2024-09-14T06:17:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6803/3775031293", 2024-09-14T06:17:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6807/3775031293", 2024-09-14T06:17:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6805/3775031293", 2024-09-14T06:17:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:17:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:17:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:17:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 4, 2024-09-14T06:17:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "fea45e8a-e8ea-4963-b93f-0651b84c9e74", 2024-09-14T06:17:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:17:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:17:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:17:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:17:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:17:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:17:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:17:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 29, 2024-09-14T06:17:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:17:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:17:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:17:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6824", 2024-09-14T06:17:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:17:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6825", 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6826", 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6827", 2024-09-14T06:17:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:17:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:17:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6830", 2024-09-14T06:17:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:17:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6831", 2024-09-14T06:17:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:17:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:17:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6828", 2024-09-14T06:17:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:17:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6829", 2024-09-14T06:17:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:17:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6825/2632317380", 2024-09-14T06:17:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6827/2632317380", 2024-09-14T06:17:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6831/2632317380", 2024-09-14T06:17:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6829/2632317380", 2024-09-14T06:17:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:17:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:17:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:17:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 5, 2024-09-14T06:17:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "b601e852-a180-477f-86fc-6a213fa866f0", 2024-09-14T06:17:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:17:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:17:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:17:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:17:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:17:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:17:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:17:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 56, 2024-09-14T06:17:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:17:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:17:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:17:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6816", 2024-09-14T06:17:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:17:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6817", 2024-09-14T06:17:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:17:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:17:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6818", 2024-09-14T06:17:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:17:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6819", 2024-09-14T06:17:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:17:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:17:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6822", 2024-09-14T06:17:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:17:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6823", 2024-09-14T06:17:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:17:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:17:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6820", 2024-09-14T06:17:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:17:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6821", 2024-09-14T06:17:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:17:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6817/3501664149", 2024-09-14T06:17:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6819/3501664149", 2024-09-14T06:17:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6823/3501664149", 2024-09-14T06:17:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6821/3501664149", 2024-09-14T06:17:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:17:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:17:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:17:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 6, 2024-09-14T06:17:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "1113afe0-f524-41b1-8e51-eb5164602d38", 2024-09-14T06:17:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:17:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:17:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:17:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:17:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:17:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:17:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:17:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 34, 2024-09-14T06:17:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:17:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:17:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:17:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6800", 2024-09-14T06:17:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:17:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6801", 2024-09-14T06:17:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:17:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:17:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6802", 2024-09-14T06:17:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:17:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6803", 2024-09-14T06:17:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:17:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:17:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6806", 2024-09-14T06:17:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:17:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6807", 2024-09-14T06:17:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:17:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:17:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6804", 2024-09-14T06:17:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:17:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6805", 2024-09-14T06:17:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:17:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6801/1395936437", 2024-09-14T06:17:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6803/1395936437", 2024-09-14T06:17:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6807/1395936437", 2024-09-14T06:17:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6805/1395936437", 2024-09-14T06:17:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:17:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:17:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:17:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 7, 2024-09-14T06:17:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "c9162545-318b-4591-be64-cc8ace9097fe", 2024-09-14T06:17:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:17:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:17:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:17:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:17:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:17:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:17:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:17:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 23, 2024-09-14T06:17:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:17:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:17:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:17:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6808", 2024-09-14T06:17:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:17:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6809", 2024-09-14T06:17:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:17:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:17:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6810", 2024-09-14T06:17:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:17:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6811", 2024-09-14T06:17:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:17:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:17:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6814", 2024-09-14T06:17:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:17:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6815", 2024-09-14T06:17:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:17:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:17:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6812", 2024-09-14T06:17:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:17:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6813", 2024-09-14T06:17:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:17:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6809/711112216", 2024-09-14T06:17:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6811/711112216", 2024-09-14T06:17:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6815/711112216", 2024-09-14T06:17:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6813/711112216", 2024-09-14T06:17:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:17:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:17:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:17:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:17:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_xinfo": [ 2024-09-14T06:17:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 0, 2024-09-14T06:17:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:17:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:17:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:17:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:17:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:17:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.705575+0000", 2024-09-14T06:17:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:17:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 1, 2024-09-14T06:17:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:17:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:17:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:17:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:17:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:17:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.806946+0000", 2024-09-14T06:17:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:17:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 2, 2024-09-14T06:17:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:17:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:17:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:17:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:17:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:17:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.853831+0000", 2024-09-14T06:17:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:17:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 3, 2024-09-14T06:17:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:17:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:17:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:17:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:17:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:17:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.690315+0000", 2024-09-14T06:17:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:17:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 4, 2024-09-14T06:17:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:17:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:17:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:17:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:17:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:17:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.494526+0000", 2024-09-14T06:17:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:17:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 5, 2024-09-14T06:17:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:17:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:17:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:17:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:17:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:17:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.546122+0000", 2024-09-14T06:17:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:17:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 6, 2024-09-14T06:17:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:17:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:17:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:17:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:17:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:17:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.173691+0000", 2024-09-14T06:17:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:17:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 7, 2024-09-14T06:17:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:17:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:17:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:17:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:17:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:17:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.148509+0000", 2024-09-14T06:17:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:17:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:17:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap": [], 2024-09-14T06:17:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap_items": [ 2024-09-14T06:17:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "pgid": "3.14", 2024-09-14T06:17:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "mappings": [ 2024-09-14T06:17:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "from": 3, 2024-09-14T06:17:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "to": 1 2024-09-14T06:17:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "pgid": "3.1f", 2024-09-14T06:17:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "mappings": [ 2024-09-14T06:17:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "from": 1, 2024-09-14T06:17:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "to": 2 2024-09-14T06:17:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:17:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap_primaries": [], 2024-09-14T06:17:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_temp": [], 2024-09-14T06:17:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_temp": [], 2024-09-14T06:17:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "blocklist": {}, 2024-09-14T06:17:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "range_blocklist": {}, 2024-09-14T06:17:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profiles": { 2024-09-14T06:17:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "default": { 2024-09-14T06:17:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-failure-domain": "osd", 2024-09-14T06:17:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "k": "2", 2024-09-14T06:17:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "m": "1", 2024-09-14T06:17:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "plugin": "jerasure", 2024-09-14T06:17:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "technique": "reed_sol_van" 2024-09-14T06:17:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps_queue": [], 2024-09-14T06:17:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "new_removed_snaps": [], 2024-09-14T06:17:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "new_purged_snaps": [], 2024-09-14T06:17:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_node_flags": {}, 2024-09-14T06:17:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "device_class_flags": {}, 2024-09-14T06:17:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode": { 2024-09-14T06:17:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode_enabled": false, 2024-09-14T06:17:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_bucket_count": 0, 2024-09-14T06:17:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "degraded_stretch_mode": 0, 2024-09-14T06:17:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "recovering_stretch_mode": 0, 2024-09-14T06:17:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode_bucket": 0 2024-09-14T06:17:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:04.851 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:04.852 INFO:tasks.thrashosds.thrasher:j is {'epoch': 59, 'fsid': '038563b1-268f-461f-b6a6-4b4b41ad0236', 'created': '2024-09-14T06:15:49.493035+0000', 'modified': '2024-09-14T06:17:04.200986+0000', 'last_up_change': '2024-09-14T06:16:02.014989+0000', 'last_in_change': '2024-09-14T06:16:26.221200+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:16:02.429225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:16:08.129454+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.380000114440918, 'score_stable': 4.380000114440918, 'optimal_score': 1, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:16:18.928556+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 20, 'snap_epoch': 59, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'c669ea76-ed19-4c12-9e56-d79d62846f95', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6808', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6809', 'nonce': 1157559201}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6810', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6811', 'nonce': 1157559201}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6814', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6815', 'nonce': 1157559201}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6812', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6813', 'nonce': 1157559201}]}, 'public_addr': '172.21.15.39:6809/1157559201', 'cluster_addr': '172.21.15.39:6811/1157559201', 'heartbeat_back_addr': '172.21.15.39:6815/1157559201', 'heartbeat_front_addr': '172.21.15.39:6813/1157559201', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6816', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6817', 'nonce': 4062089212}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6818', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6819', 'nonce': 4062089212}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6822', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6823', 'nonce': 4062089212}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6820', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6821', 'nonce': 4062089212}]}, 'public_addr': '172.21.15.39:6817/4062089212', 'cluster_addr': '172.21.15.39:6819/4062089212', 'heartbeat_back_addr': '172.21.15.39:6823/4062089212', 'heartbeat_front_addr': '172.21.15.39:6821/4062089212', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9bda988f-e615-4545-849b-318c408c4c42', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 29, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6824', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6825', 'nonce': 1724773882}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6826', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6827', 'nonce': 1724773882}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6830', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6831', 'nonce': 1724773882}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6828', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6829', 'nonce': 1724773882}]}, 'public_addr': '172.21.15.39:6825/1724773882', 'cluster_addr': '172.21.15.39:6827/1724773882', 'heartbeat_back_addr': '172.21.15.39:6831/1724773882', 'heartbeat_front_addr': '172.21.15.39:6829/1724773882', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8ea6cd69-2ef1-42ea-b339-7394bf8f4a87', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6800', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6801', 'nonce': 3775031293}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6802', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6803', 'nonce': 3775031293}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6806', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6807', 'nonce': 3775031293}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6804', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6805', 'nonce': 3775031293}]}, 'public_addr': '172.21.15.39:6801/3775031293', 'cluster_addr': '172.21.15.39:6803/3775031293', 'heartbeat_back_addr': '172.21.15.39:6807/3775031293', 'heartbeat_front_addr': '172.21.15.39:6805/3775031293', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fea45e8a-e8ea-4963-b93f-0651b84c9e74', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 29, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6824', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6825', 'nonce': 2632317380}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6826', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6827', 'nonce': 2632317380}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6830', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6831', 'nonce': 2632317380}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6828', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6829', 'nonce': 2632317380}]}, 'public_addr': '172.21.15.87:6825/2632317380', 'cluster_addr': '172.21.15.87:6827/2632317380', 'heartbeat_back_addr': '172.21.15.87:6831/2632317380', 'heartbeat_front_addr': '172.21.15.87:6829/2632317380', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b601e852-a180-477f-86fc-6a213fa866f0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6816', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6817', 'nonce': 3501664149}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6818', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6819', 'nonce': 3501664149}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6822', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6823', 'nonce': 3501664149}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6820', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6821', 'nonce': 3501664149}]}, 'public_addr': '172.21.15.87:6817/3501664149', 'cluster_addr': '172.21.15.87:6819/3501664149', 'heartbeat_back_addr': '172.21.15.87:6823/3501664149', 'heartbeat_front_addr': '172.21.15.87:6821/3501664149', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '1113afe0-f524-41b1-8e51-eb5164602d38', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6800', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6801', 'nonce': 1395936437}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6802', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6803', 'nonce': 1395936437}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6806', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6807', 'nonce': 1395936437}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6804', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6805', 'nonce': 1395936437}]}, 'public_addr': '172.21.15.87:6801/1395936437', 'cluster_addr': '172.21.15.87:6803/1395936437', 'heartbeat_back_addr': '172.21.15.87:6807/1395936437', 'heartbeat_front_addr': '172.21.15.87:6805/1395936437', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c9162545-318b-4591-be64-cc8ace9097fe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6808', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6809', 'nonce': 711112216}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6810', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6811', 'nonce': 711112216}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6814', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6815', 'nonce': 711112216}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6812', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6813', 'nonce': 711112216}]}, 'public_addr': '172.21.15.87:6809/711112216', 'cluster_addr': '172.21.15.87:6811/711112216', 'heartbeat_back_addr': '172.21.15.87:6815/711112216', 'heartbeat_front_addr': '172.21.15.87:6813/711112216', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.705575+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.806946+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.853831+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.690315+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.494526+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.546122+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.173691+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.148509+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.14', 'mappings': [{'from': 3, 'to': 1}]}, {'pgid': '3.1f', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T06:17:04.852 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:17:04.899 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:04.898+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:04.978 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:04.974+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:04.986 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:04.986 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:04.986 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:04.986 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:04.986 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:05.001 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:17:05.025 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:05.025 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:05.025 INFO:teuthology.orchestra.run.smithi039.stdout:3362 read 0xaab2122 thru 0xaab8949 (0x6828 bytes) 2024-09-14T06:17:05.029 INFO:teuthology.orchestra.run.smithi039.stdout:3363 read 0x417f6ce thru 0x4188d03 (0x9636 bytes) 2024-09-14T06:17:05.030 INFO:teuthology.orchestra.run.smithi039.stdout:3364 write 0x1bd5c29 thru 0x1be1dd0 (0xc1a8 bytes) 2024-09-14T06:17:05.040 INFO:teuthology.orchestra.run.smithi039.stdout:3365 write 0x3782e6e thru 0x3785960 (0x2af3 bytes) 2024-09-14T06:17:05.055 INFO:teuthology.orchestra.run.smithi039.stdout:3366 trunc from 0xedff298 to 0x894e34a 2024-09-14T06:17:05.073 INFO:teuthology.orchestra.run.smithi039.stdout:3367 writesame 0x788e736 thru 0x7894479 data_size 0x2f(0x5d44 bytes) 2024-09-14T06:17:05.081 INFO:teuthology.orchestra.run.smithi039.stdout:3368 write 0x82681ca thru 0x8272a31 (0xa868 bytes) 2024-09-14T06:17:05.089 INFO:teuthology.orchestra.run.smithi039.stdout:3370 writesame 0x32b3fbd thru 0x32b84ba data_size 0x1(0x44fe bytes) 2024-09-14T06:17:05.091 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:05.090+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:05.099 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:05.099 INFO:teuthology.orchestra.run.smithi039.stdout:3372 write 0xfdb0c6 thru 0xfdc5b5 (0x14f0 bytes) 2024-09-14T06:17:05.133 INFO:teuthology.orchestra.run.smithi039.stdout:3373 writesame 0xcd1443b thru 0xcd1d34e data_size 0x2(0x8f14 bytes) 2024-09-14T06:17:05.133 INFO:teuthology.orchestra.run.smithi039.stdout:3374 write 0xc6a3923 thru 0xc6a8522 (0x4c00 bytes) 2024-09-14T06:17:05.139 INFO:teuthology.orchestra.run.smithi039.stdout:3376 write 0x7718cc1 thru 0x7723f22 (0xb262 bytes) 2024-09-14T06:17:05.147 INFO:teuthology.orchestra.run.smithi039.stdout:3377 write 0x1829afb thru 0x183918a (0xf690 bytes) 2024-09-14T06:17:05.154 INFO:teuthology.orchestra.run.smithi039.stdout:3378 write 0xc43a36f thru 0xc43a5f5 (0x287 bytes) 2024-09-14T06:17:05.159 INFO:teuthology.orchestra.run.smithi039.stdout:3379 read 0x915e29c thru 0x9166cd6 (0x8a3b bytes) 2024-09-14T06:17:05.159 INFO:teuthology.orchestra.run.smithi039.stdout:3381 write 0x1cef0ff thru 0x1cf85a3 (0x94a5 bytes) 2024-09-14T06:17:05.169 INFO:teuthology.orchestra.run.smithi039.stdout:3382 writesame 0xc0cea33 thru 0xc0dde00 data_size 0xb(0xf3ce bytes) 2024-09-14T06:17:05.179 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:05.178+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:05.182 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:05.182 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:05.182 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:05.182 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:05.182 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.182 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13229 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.182 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:05.183 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.082851739999999, 2024-09-14T06:17:05.183 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081798868999999996, 2024-09-14T06:17:05.183 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.183 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.183 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.183 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.183 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.183 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.183 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13229 2024-09-14T06:17:05.183 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.183 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.183 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.183 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.183 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:05.184 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.184 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.184 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.184 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.184 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:05.184 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.184 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.184 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.184 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.184 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098710+0000", 2024-09-14T06:17:05.184 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8199999999999999e-07 2024-09-14T06:17:05.184 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.184 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.184 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.185 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098714+0000", 2024-09-14T06:17:05.185 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:17:05.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.185 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.185 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.185 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098727+0000", 2024-09-14T06:17:05.185 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2520999999999999e-05 2024-09-14T06:17:05.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.185 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.185 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.185 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098749+0000", 2024-09-14T06:17:05.185 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2773999999999999e-05 2024-09-14T06:17:05.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160234+0000", 2024-09-14T06:17:05.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.061484430999999999 2024-09-14T06:17:05.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160281+0000", 2024-09-14T06:17:05.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7437e-05 2024-09-14T06:17:05.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160569+0000", 2024-09-14T06:17:05.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028788700000000002 2024-09-14T06:17:05.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163121+0000", 2024-09-14T06:17:05.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025514090000000001 2024-09-14T06:17:05.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180410+0000", 2024-09-14T06:17:05.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.017289098999999999 2024-09-14T06:17:05.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.188 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180472+0000", 2024-09-14T06:17:05.188 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2547999999999998e-05 2024-09-14T06:17:05.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.188 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.188 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.188 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180508+0000", 2024-09-14T06:17:05.188 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5534000000000002e-05 2024-09-14T06:17:05.188 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.188 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.188 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.188 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.188 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13230 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.188 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:05.189 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.082770537999998, 2024-09-14T06:17:05.189 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081851558000000005, 2024-09-14T06:17:05.189 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.189 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.189 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.189 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.189 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.189 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.189 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13230 2024-09-14T06:17:05.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.189 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.189 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.189 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.189 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:05.189 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.190 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.190 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:05.190 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.190 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.190 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098791+0000", 2024-09-14T06:17:05.190 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8800000000000001e-07 2024-09-14T06:17:05.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.190 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.190 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098795+0000", 2024-09-14T06:17:05.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.391e-06 2024-09-14T06:17:05.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.191 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098804+0000", 2024-09-14T06:17:05.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3440000000000001e-06 2024-09-14T06:17:05.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.191 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098810+0000", 2024-09-14T06:17:05.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2339999999999999e-06 2024-09-14T06:17:05.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.191 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160930+0000", 2024-09-14T06:17:05.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062120018999999999 2024-09-14T06:17:05.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161017+0000", 2024-09-14T06:17:05.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6868999999999998e-05 2024-09-14T06:17:05.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161300+0000", 2024-09-14T06:17:05.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028349499999999998 2024-09-14T06:17:05.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.193 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164612+0000", 2024-09-14T06:17:05.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033119680000000002 2024-09-14T06:17:05.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.193 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.193 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180559+0000", 2024-09-14T06:17:05.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015946612999999998 2024-09-14T06:17:05.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.193 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.193 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180605+0000", 2024-09-14T06:17:05.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5890000000000003e-05 2024-09-14T06:17:05.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.194 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.194 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180642+0000", 2024-09-14T06:17:05.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:17:05.194 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.194 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.194 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.194 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13231 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.194 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:05.194 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.082508124, 2024-09-14T06:17:05.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081750685000000003, 2024-09-14T06:17:05.194 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.194 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.195 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.195 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.195 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.195 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.195 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13231 2024-09-14T06:17:05.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.195 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:05.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:05.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.196 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099053+0000", 2024-09-14T06:17:05.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:17:05.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.196 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099059+0000", 2024-09-14T06:17:05.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1789999999999999e-06 2024-09-14T06:17:05.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099072+0000", 2024-09-14T06:17:05.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3179999999999999e-05 2024-09-14T06:17:05.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099080+0000", 2024-09-14T06:17:05.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9030000000000004e-06 2024-09-14T06:17:05.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161681+0000", 2024-09-14T06:17:05.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062601641999999999 2024-09-14T06:17:05.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.198 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.198 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161766+0000", 2024-09-14T06:17:05.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4909999999999998e-05 2024-09-14T06:17:05.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.198 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.198 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162109+0000", 2024-09-14T06:17:05.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00034264899999999999 2024-09-14T06:17:05.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.198 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.198 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164615+0000", 2024-09-14T06:17:05.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025061979999999998 2024-09-14T06:17:05.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.199 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180718+0000", 2024-09-14T06:17:05.199 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016103240000000001 2024-09-14T06:17:05.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.199 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180768+0000", 2024-09-14T06:17:05.199 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9781999999999999e-05 2024-09-14T06:17:05.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.199 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180803+0000", 2024-09-14T06:17:05.199 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5021999999999998e-05 2024-09-14T06:17:05.199 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.200 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.200 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.200 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.200 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13232 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.200 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:05.200 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.082466350000001, 2024-09-14T06:17:05.200 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081861981, 2024-09-14T06:17:05.200 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.200 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.200 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.200 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.200 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.201 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.201 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13232 2024-09-14T06:17:05.201 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.201 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.201 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.201 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.201 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:05.201 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.201 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.201 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.201 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.201 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:05.201 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.201 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.202 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.202 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.202 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099095+0000", 2024-09-14T06:17:05.202 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:17:05.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.202 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.202 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.202 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099098+0000", 2024-09-14T06:17:05.202 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5420000000000001e-06 2024-09-14T06:17:05.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.202 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.202 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.202 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099106+0000", 2024-09-14T06:17:05.202 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.824e-06 2024-09-14T06:17:05.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.203 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.203 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.203 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099112+0000", 2024-09-14T06:17:05.203 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9789999999999998e-06 2024-09-14T06:17:05.203 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.203 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.203 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.203 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162576+0000", 2024-09-14T06:17:05.203 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.063463723999999999 2024-09-14T06:17:05.203 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.203 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.203 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.203 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162627+0000", 2024-09-14T06:17:05.203 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1530000000000003e-05 2024-09-14T06:17:05.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.204 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162854+0000", 2024-09-14T06:17:05.204 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000226159 2024-09-14T06:17:05.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.204 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166090+0000", 2024-09-14T06:17:05.204 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032362319999999999 2024-09-14T06:17:05.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.204 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180877+0000", 2024-09-14T06:17:05.205 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014786726 2024-09-14T06:17:05.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.205 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.205 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.205 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180920+0000", 2024-09-14T06:17:05.205 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3427000000000003e-05 2024-09-14T06:17:05.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.205 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.205 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.205 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180956+0000", 2024-09-14T06:17:05.205 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6195999999999997e-05 2024-09-14T06:17:05.205 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.205 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.205 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.206 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.206 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13233 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.206 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:05.206 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.082441086999999, 2024-09-14T06:17:05.206 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081924860000000002, 2024-09-14T06:17:05.206 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.206 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.206 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.206 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.206 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.206 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.206 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13233 2024-09-14T06:17:05.206 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.206 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.207 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.207 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:05.207 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.207 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.207 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.207 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.207 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:05.207 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.207 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.207 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.207 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.207 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099120+0000", 2024-09-14T06:17:05.207 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:17:05.207 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.207 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.208 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.208 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099122+0000", 2024-09-14T06:17:05.208 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:17:05.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.208 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.208 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.208 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099126+0000", 2024-09-14T06:17:05.208 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2540000000000003e-06 2024-09-14T06:17:05.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.208 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.208 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.208 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099130+0000", 2024-09-14T06:17:05.208 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4060000000000001e-06 2024-09-14T06:17:05.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.209 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.209 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.209 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163169+0000", 2024-09-14T06:17:05.209 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064039415000000002 2024-09-14T06:17:05.209 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.209 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.209 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.209 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163260+0000", 2024-09-14T06:17:05.209 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1038000000000005e-05 2024-09-14T06:17:05.209 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.209 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.209 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.209 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163640+0000", 2024-09-14T06:17:05.209 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037954699999999999 2024-09-14T06:17:05.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.210 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.210 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.210 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166093+0000", 2024-09-14T06:17:05.210 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024533229999999999 2024-09-14T06:17:05.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.210 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.210 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.210 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180993+0000", 2024-09-14T06:17:05.210 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01489997 2024-09-14T06:17:05.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.210 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.210 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.210 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181020+0000", 2024-09-14T06:17:05.211 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6806999999999999e-05 2024-09-14T06:17:05.211 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.211 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.211 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.211 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181044+0000", 2024-09-14T06:17:05.211 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4405000000000001e-05 2024-09-14T06:17:05.211 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.211 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.211 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.211 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.211 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.211 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13234 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.211 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:05.211 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.082398882, 2024-09-14T06:17:05.212 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082015597999999995, 2024-09-14T06:17:05.212 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.212 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.212 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.212 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.212 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.212 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.212 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13234 2024-09-14T06:17:05.212 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.212 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.212 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.212 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.212 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:05.213 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.213 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.213 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.213 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.213 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:05.213 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.213 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.213 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.213 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.213 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:05.213 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0999999999999998e-07 2024-09-14T06:17:05.213 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.213 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.213 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.214 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099166+0000", 2024-09-14T06:17:05.214 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:17:05.214 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.214 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.214 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.214 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099171+0000", 2024-09-14T06:17:05.214 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6949999999999997e-06 2024-09-14T06:17:05.214 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.214 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.214 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.214 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099175+0000", 2024-09-14T06:17:05.214 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9029999999999997e-06 2024-09-14T06:17:05.214 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.215 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.215 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.215 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164063+0000", 2024-09-14T06:17:05.215 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064887608999999999 2024-09-14T06:17:05.215 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.215 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.215 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.215 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164111+0000", 2024-09-14T06:17:05.215 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7648999999999997e-05 2024-09-14T06:17:05.215 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.215 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.215 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.215 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164351+0000", 2024-09-14T06:17:05.215 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024072899999999999 2024-09-14T06:17:05.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.216 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.216 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.216 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167561+0000", 2024-09-14T06:17:05.216 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032094179999999999 2024-09-14T06:17:05.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.216 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.216 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.216 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181093+0000", 2024-09-14T06:17:05.216 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013532245 2024-09-14T06:17:05.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.216 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.216 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.217 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181139+0000", 2024-09-14T06:17:05.217 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6483999999999997e-05 2024-09-14T06:17:05.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.217 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.217 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.217 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181177+0000", 2024-09-14T06:17:05.217 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7861000000000002e-05 2024-09-14T06:17:05.217 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.217 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.217 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.217 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.217 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13235 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.217 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:05.218 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.082291607999998, 2024-09-14T06:17:05.218 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081988088000000001, 2024-09-14T06:17:05.218 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.218 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.218 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.218 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.218 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.218 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.218 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13235 2024-09-14T06:17:05.218 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.218 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.218 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.218 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.219 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:05.219 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.219 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.219 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.219 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.219 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:05.219 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.219 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.219 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.219 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.219 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099270+0000", 2024-09-14T06:17:05.219 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7400000000000003e-07 2024-09-14T06:17:05.219 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.220 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.220 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.220 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099276+0000", 2024-09-14T06:17:05.220 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:17:05.220 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.220 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.220 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.220 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099284+0000", 2024-09-14T06:17:05.220 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7979999999999995e-06 2024-09-14T06:17:05.220 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.220 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.220 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.220 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099289+0000", 2024-09-14T06:17:05.221 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:17:05.221 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.221 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.221 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.221 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164664+0000", 2024-09-14T06:17:05.221 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065374901999999999 2024-09-14T06:17:05.221 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.221 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.221 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.221 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164755+0000", 2024-09-14T06:17:05.221 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0790000000000003e-05 2024-09-14T06:17:05.221 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.221 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.222 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.222 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165143+0000", 2024-09-14T06:17:05.222 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000387904 2024-09-14T06:17:05.222 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.222 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.222 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.222 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167564+0000", 2024-09-14T06:17:05.222 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024208989999999998 2024-09-14T06:17:05.222 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.222 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.222 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.222 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181212+0000", 2024-09-14T06:17:05.222 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013648664 2024-09-14T06:17:05.222 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.223 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.223 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.223 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181237+0000", 2024-09-14T06:17:05.223 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5177999999999998e-05 2024-09-14T06:17:05.223 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.223 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.223 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.223 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181257+0000", 2024-09-14T06:17:05.223 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9660000000000002e-05 2024-09-14T06:17:05.223 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.223 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.223 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.223 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.224 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.224 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13236 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.224 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:05.224 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.082184788999999, 2024-09-14T06:17:05.224 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082010939000000005, 2024-09-14T06:17:05.224 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.224 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.224 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.224 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.224 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.224 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.224 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13236 2024-09-14T06:17:05.224 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.225 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.225 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.225 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.225 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:05.225 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.225 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.225 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.225 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.225 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:05.225 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.225 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.225 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.225 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.225 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:05.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.61e-07 2024-09-14T06:17:05.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.226 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.226 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099382+0000", 2024-09-14T06:17:05.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3589999999999999e-06 2024-09-14T06:17:05.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.226 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.226 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099388+0000", 2024-09-14T06:17:05.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9619999999999998e-06 2024-09-14T06:17:05.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.227 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.227 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099393+0000", 2024-09-14T06:17:05.227 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3410000000000002e-06 2024-09-14T06:17:05.227 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.227 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.227 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.227 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165553+0000", 2024-09-14T06:17:05.227 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066159604999999996 2024-09-14T06:17:05.228 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.228 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.228 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.228 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165603+0000", 2024-09-14T06:17:05.228 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0427999999999999e-05 2024-09-14T06:17:05.228 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.228 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.229 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.229 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165827+0000", 2024-09-14T06:17:05.229 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022358299999999999 2024-09-14T06:17:05.229 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.229 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.229 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.229 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177005+0000", 2024-09-14T06:17:05.229 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011178044 2024-09-14T06:17:05.229 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.229 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.229 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.229 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181302+0000", 2024-09-14T06:17:05.229 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042970669999999999 2024-09-14T06:17:05.230 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.230 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.230 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.230 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181349+0000", 2024-09-14T06:17:05.230 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7287999999999999e-05 2024-09-14T06:17:05.230 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.230 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.230 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.230 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181387+0000", 2024-09-14T06:17:05.230 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7601000000000001e-05 2024-09-14T06:17:05.230 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.230 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.230 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.230 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.231 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.231 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13237 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.231 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:05.231 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.082026641999999, 2024-09-14T06:17:05.231 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081927299999999995, 2024-09-14T06:17:05.231 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.231 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.231 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.232 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.232 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.232 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.232 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13237 2024-09-14T06:17:05.232 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.232 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.232 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.232 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.232 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:05.233 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.233 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.233 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.233 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.233 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:05.233 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.233 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.233 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.233 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.234 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099535+0000", 2024-09-14T06:17:05.234 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:17:05.234 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.234 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.234 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.234 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099538+0000", 2024-09-14T06:17:05.234 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:17:05.234 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.234 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.234 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.234 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099545+0000", 2024-09-14T06:17:05.234 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:17:05.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.235 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.235 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.235 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099551+0000", 2024-09-14T06:17:05.235 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1770000000000003e-06 2024-09-14T06:17:05.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.235 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.235 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.235 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166131+0000", 2024-09-14T06:17:05.235 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066580499000000001 2024-09-14T06:17:05.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.235 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.235 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.235 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166221+0000", 2024-09-14T06:17:05.235 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0000000000000006e-05 2024-09-14T06:17:05.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.236 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.236 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.236 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166622+0000", 2024-09-14T06:17:05.236 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040029799999999999 2024-09-14T06:17:05.236 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.236 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.236 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.236 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177008+0000", 2024-09-14T06:17:05.236 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010386772000000001 2024-09-14T06:17:05.236 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.236 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.236 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.236 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181420+0000", 2024-09-14T06:17:05.237 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044119190000000003 2024-09-14T06:17:05.237 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.237 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.237 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.237 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181443+0000", 2024-09-14T06:17:05.237 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2603999999999999e-05 2024-09-14T06:17:05.237 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.237 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.237 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.237 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181461+0000", 2024-09-14T06:17:05.237 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8405000000000001e-05 2024-09-14T06:17:05.237 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.237 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.237 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.237 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.238 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.238 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13238 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.238 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:05.238 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.081990334, 2024-09-14T06:17:05.238 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082014927000000001, 2024-09-14T06:17:05.238 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.238 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.238 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.238 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.238 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.238 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.238 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13238 2024-09-14T06:17:05.238 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.238 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.239 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.239 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.239 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:05.239 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.239 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.239 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.239 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.239 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:05.239 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.239 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.239 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.239 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.239 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099571+0000", 2024-09-14T06:17:05.239 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9699999999999996e-07 2024-09-14T06:17:05.240 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.240 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.240 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.240 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099577+0000", 2024-09-14T06:17:05.240 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2890000000000003e-06 2024-09-14T06:17:05.240 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.240 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.241 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.241 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099587+0000", 2024-09-14T06:17:05.241 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0002e-05 2024-09-14T06:17:05.241 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.241 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.241 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.241 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099593+0000", 2024-09-14T06:17:05.241 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4539999999999998e-06 2024-09-14T06:17:05.242 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.242 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.242 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.242 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167024+0000", 2024-09-14T06:17:05.242 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067431480000000002 2024-09-14T06:17:05.242 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.242 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.242 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.242 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167074+0000", 2024-09-14T06:17:05.242 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9996e-05 2024-09-14T06:17:05.242 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.242 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.242 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.242 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167288+0000", 2024-09-14T06:17:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021410900000000001 2024-09-14T06:17:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177997+0000", 2024-09-14T06:17:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010708756 2024-09-14T06:17:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181508+0000", 2024-09-14T06:17:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003511363 2024-09-14T06:17:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.243 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181552+0000", 2024-09-14T06:17:05.244 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3609999999999998e-05 2024-09-14T06:17:05.244 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.244 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.244 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.244 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181585+0000", 2024-09-14T06:17:05.244 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3271e-05 2024-09-14T06:17:05.244 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.244 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.244 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.244 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.244 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.244 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13239 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.244 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:05.244 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.081898515999999, 2024-09-14T06:17:05.245 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081998772999999997, 2024-09-14T06:17:05.245 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.245 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.245 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.245 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.245 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.245 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.245 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13239 2024-09-14T06:17:05.245 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.245 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.245 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.245 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.245 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:05.246 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.246 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.246 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.246 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.246 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:05.246 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.246 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.246 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.246 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.246 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099663+0000", 2024-09-14T06:17:05.246 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6100000000000001e-07 2024-09-14T06:17:05.246 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.246 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.246 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.247 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099666+0000", 2024-09-14T06:17:05.247 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.056e-06 2024-09-14T06:17:05.247 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.247 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.247 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.247 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099672+0000", 2024-09-14T06:17:05.247 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8120000000000002e-06 2024-09-14T06:17:05.247 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.247 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.247 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.247 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099676+0000", 2024-09-14T06:17:05.247 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:17:05.247 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.247 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.248 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.248 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167602+0000", 2024-09-14T06:17:05.248 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067925867000000001 2024-09-14T06:17:05.248 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.248 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.248 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.248 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167693+0000", 2024-09-14T06:17:05.248 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0736000000000007e-05 2024-09-14T06:17:05.248 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.248 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.248 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.248 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168065+0000", 2024-09-14T06:17:05.248 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037245399999999998 2024-09-14T06:17:05.248 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.249 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.249 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.249 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178003+0000", 2024-09-14T06:17:05.249 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0099372360000000003 2024-09-14T06:17:05.249 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.249 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.249 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.249 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181621+0000", 2024-09-14T06:17:05.249 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036178489999999998 2024-09-14T06:17:05.249 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.249 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.249 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.249 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181643+0000", 2024-09-14T06:17:05.249 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2155e-05 2024-09-14T06:17:05.249 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.250 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.250 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.250 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181661+0000", 2024-09-14T06:17:05.250 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8147000000000001e-05 2024-09-14T06:17:05.250 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.250 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.250 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.250 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.250 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.250 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13240 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.250 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:05.250 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.081779423, 2024-09-14T06:17:05.250 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081942607000000001, 2024-09-14T06:17:05.250 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.250 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.250 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.251 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.251 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.251 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.251 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13240 2024-09-14T06:17:05.251 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.251 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.251 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.251 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.251 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:05.251 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.251 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.251 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.251 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.251 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:05.251 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.252 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.252 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.252 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.252 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099782+0000", 2024-09-14T06:17:05.252 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3900000000000004e-07 2024-09-14T06:17:05.252 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.252 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.252 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.252 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099787+0000", 2024-09-14T06:17:05.252 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6189999999999999e-06 2024-09-14T06:17:05.252 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.252 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.252 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.252 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099793+0000", 2024-09-14T06:17:05.252 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.998e-06 2024-09-14T06:17:05.253 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.253 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.253 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.253 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099797+0000", 2024-09-14T06:17:05.253 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8449999999999996e-06 2024-09-14T06:17:05.253 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.253 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.253 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.253 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168494+0000", 2024-09-14T06:17:05.253 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.068696500999999993 2024-09-14T06:17:05.253 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.253 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.253 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.253 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168590+0000", 2024-09-14T06:17:05.253 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6733999999999999e-05 2024-09-14T06:17:05.253 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.254 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.254 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.254 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168973+0000", 2024-09-14T06:17:05.254 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038229399999999998 2024-09-14T06:17:05.254 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.254 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.254 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.254 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178013+0000", 2024-09-14T06:17:05.254 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0090397199999999994 2024-09-14T06:17:05.254 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.254 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.254 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.254 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181683+0000", 2024-09-14T06:17:05.254 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036705819999999999 2024-09-14T06:17:05.254 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.255 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.255 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.255 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181705+0000", 2024-09-14T06:17:05.255 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1551000000000001e-05 2024-09-14T06:17:05.255 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.255 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.255 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.255 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181724+0000", 2024-09-14T06:17:05.255 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9123999999999999e-05 2024-09-14T06:17:05.255 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.255 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.255 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.255 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.255 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.255 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13243 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.256 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:05.256 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.081245693, 2024-09-14T06:17:05.256 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081767815999999993, 2024-09-14T06:17:05.256 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.256 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.256 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.256 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.256 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.256 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.256 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13243 2024-09-14T06:17:05.256 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.256 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.256 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.256 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.257 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:05.257 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.257 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.257 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.257 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.257 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:05.257 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.257 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.257 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.257 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.257 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:05.257 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1e-07 2024-09-14T06:17:05.257 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.257 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.257 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.257 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100317+0000", 2024-09-14T06:17:05.258 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.092e-06 2024-09-14T06:17:05.258 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.258 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.258 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.258 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100321+0000", 2024-09-14T06:17:05.258 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7809999999999999e-06 2024-09-14T06:17:05.258 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.258 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.258 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.258 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100324+0000", 2024-09-14T06:17:05.258 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.117e-06 2024-09-14T06:17:05.258 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.258 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.258 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.258 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171113+0000", 2024-09-14T06:17:05.259 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.070788816000000004 2024-09-14T06:17:05.259 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.259 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.259 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.259 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171202+0000", 2024-09-14T06:17:05.259 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8843000000000003e-05 2024-09-14T06:17:05.259 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.259 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.259 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.259 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171584+0000", 2024-09-14T06:17:05.259 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038231099999999998 2024-09-14T06:17:05.259 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.259 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.259 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.259 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178101+0000", 2024-09-14T06:17:05.260 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0065168329999999997 2024-09-14T06:17:05.260 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.260 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.260 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.260 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182041+0000", 2024-09-14T06:17:05.260 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039401810000000001 2024-09-14T06:17:05.260 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.260 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.260 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.260 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182065+0000", 2024-09-14T06:17:05.260 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3623000000000001e-05 2024-09-14T06:17:05.260 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.260 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.260 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.261 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182083+0000", 2024-09-14T06:17:05.261 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7909000000000001e-05 2024-09-14T06:17:05.261 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.261 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.261 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.261 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.261 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.261 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13244 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.261 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:05.261 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.081230872999999, 2024-09-14T06:17:05.261 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081837800000000002, 2024-09-14T06:17:05.261 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.261 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.261 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.261 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.262 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.262 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.262 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13244 2024-09-14T06:17:05.262 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.262 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.262 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.262 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.262 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:05.262 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.262 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.262 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.262 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.262 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:05.262 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.262 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:17:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100337+0000", 2024-09-14T06:17:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.939e-06 2024-09-14T06:17:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100341+0000", 2024-09-14T06:17:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2320000000000003e-06 2024-09-14T06:17:05.264 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.264 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.264 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.264 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100344+0000", 2024-09-14T06:17:05.264 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0759999999999999e-06 2024-09-14T06:17:05.264 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.264 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.264 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.264 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171996+0000", 2024-09-14T06:17:05.264 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.071651919999999994 2024-09-14T06:17:05.264 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.264 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.264 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.264 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172079+0000", 2024-09-14T06:17:05.265 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2579999999999999e-05 2024-09-14T06:17:05.265 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.265 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.265 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.265 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172457+0000", 2024-09-14T06:17:05.265 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037785100000000001 2024-09-14T06:17:05.265 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.265 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.265 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.265 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178104+0000", 2024-09-14T06:17:05.265 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056471730000000001 2024-09-14T06:17:05.265 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.265 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.265 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.266 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182123+0000", 2024-09-14T06:17:05.266 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040190019999999998 2024-09-14T06:17:05.266 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.266 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.266 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.266 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182147+0000", 2024-09-14T06:17:05.266 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4241000000000001e-05 2024-09-14T06:17:05.266 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.266 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.266 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.266 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182168+0000", 2024-09-14T06:17:05.266 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0492999999999998e-05 2024-09-14T06:17:05.266 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.266 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.267 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.267 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.267 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.267 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13245 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.267 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:05.267 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.081211366000002, 2024-09-14T06:17:05.267 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081893899000000006, 2024-09-14T06:17:05.267 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.267 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.267 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.267 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.267 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.267 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.267 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13245 2024-09-14T06:17:05.267 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.268 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.268 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.268 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.268 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:05.268 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.268 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.268 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.268 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.268 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:05.268 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.268 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.268 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.268 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.268 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100350+0000", 2024-09-14T06:17:05.269 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5499999999999999e-07 2024-09-14T06:17:05.269 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.269 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.269 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.269 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100353+0000", 2024-09-14T06:17:05.269 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:17:05.269 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.269 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.269 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.269 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100357+0000", 2024-09-14T06:17:05.269 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8500000000000004e-06 2024-09-14T06:17:05.269 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.269 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.270 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.270 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100360+0000", 2024-09-14T06:17:05.270 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8700000000000001e-06 2024-09-14T06:17:05.270 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.270 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.270 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.270 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172862+0000", 2024-09-14T06:17:05.270 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.072501992000000001 2024-09-14T06:17:05.270 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.270 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.270 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.270 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172950+0000", 2024-09-14T06:17:05.270 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8988999999999998e-05 2024-09-14T06:17:05.270 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.270 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.271 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.271 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173324+0000", 2024-09-14T06:17:05.271 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037366199999999997 2024-09-14T06:17:05.271 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.271 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.271 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.271 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178107+0000", 2024-09-14T06:17:05.271 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047830520000000003 2024-09-14T06:17:05.271 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.271 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.271 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.271 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182202+0000", 2024-09-14T06:17:05.271 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040949280000000003 2024-09-14T06:17:05.271 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.271 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.271 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.272 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182226+0000", 2024-09-14T06:17:05.272 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3535000000000001e-05 2024-09-14T06:17:05.272 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.272 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.272 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.272 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182243+0000", 2024-09-14T06:17:05.272 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7445e-05 2024-09-14T06:17:05.272 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.272 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.272 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.272 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.272 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.272 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13246 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.272 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:05.273 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.081197044, 2024-09-14T06:17:05.273 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081962276000000001, 2024-09-14T06:17:05.273 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.273 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.273 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.273 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.273 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.273 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.273 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13246 2024-09-14T06:17:05.273 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.273 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.273 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.273 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.273 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:05.274 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.274 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.274 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.274 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.274 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:05.274 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.274 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.274 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.274 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.274 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:05.274 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0800000000000001e-07 2024-09-14T06:17:05.274 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.274 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.274 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.275 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100367+0000", 2024-09-14T06:17:05.275 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8439999999999998e-06 2024-09-14T06:17:05.275 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.275 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.275 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.275 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100372+0000", 2024-09-14T06:17:05.275 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4009999999999996e-06 2024-09-14T06:17:05.275 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.275 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.275 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.275 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100378+0000", 2024-09-14T06:17:05.275 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7180000000000001e-06 2024-09-14T06:17:05.275 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.275 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.275 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.276 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173780+0000", 2024-09-14T06:17:05.276 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.073401991 2024-09-14T06:17:05.276 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.276 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.276 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.276 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173879+0000", 2024-09-14T06:17:05.276 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9449000000000003e-05 2024-09-14T06:17:05.276 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.276 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.276 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.276 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174240+0000", 2024-09-14T06:17:05.276 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036118100000000002 2024-09-14T06:17:05.276 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.276 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.276 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.277 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178109+0000", 2024-09-14T06:17:05.277 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038690619999999999 2024-09-14T06:17:05.277 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.277 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.277 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.277 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182281+0000", 2024-09-14T06:17:05.277 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041715900000000002 2024-09-14T06:17:05.277 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.277 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.277 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.277 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182306+0000", 2024-09-14T06:17:05.277 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4477e-05 2024-09-14T06:17:05.277 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.277 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.277 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.278 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182326+0000", 2024-09-14T06:17:05.278 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0254999999999998e-05 2024-09-14T06:17:05.278 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.278 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.278 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.278 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.278 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.278 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13247 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.278 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:05.278 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.081099866999999, 2024-09-14T06:17:05.278 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966593000000004, 2024-09-14T06:17:05.278 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.278 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.278 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.278 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.279 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.279 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.279 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13247 2024-09-14T06:17:05.279 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.279 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.279 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.279 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.279 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:05.279 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.279 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.279 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.279 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.279 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:05.279 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.279 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.280 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.280 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.280 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100462+0000", 2024-09-14T06:17:05.280 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9800000000000002e-07 2024-09-14T06:17:05.280 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.280 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.280 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.280 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100466+0000", 2024-09-14T06:17:05.280 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4039999999999997e-06 2024-09-14T06:17:05.280 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.280 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.280 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.280 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100475+0000", 2024-09-14T06:17:05.281 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3680000000000008e-06 2024-09-14T06:17:05.281 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.281 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.281 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.281 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100482+0000", 2024-09-14T06:17:05.281 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8630000000000002e-06 2024-09-14T06:17:05.281 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.281 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.281 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.281 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174669+0000", 2024-09-14T06:17:05.281 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074186849999999999 2024-09-14T06:17:05.281 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.281 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.281 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.282 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174758+0000", 2024-09-14T06:17:05.282 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9021e-05 2024-09-14T06:17:05.282 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.282 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.282 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.282 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175137+0000", 2024-09-14T06:17:05.282 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378511 2024-09-14T06:17:05.282 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.282 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.282 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.282 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178158+0000", 2024-09-14T06:17:05.282 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003020956 2024-09-14T06:17:05.282 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.283 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.283 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.283 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182386+0000", 2024-09-14T06:17:05.283 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042278840000000003 2024-09-14T06:17:05.283 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.283 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.283 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.283 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182410+0000", 2024-09-14T06:17:05.283 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4026e-05 2024-09-14T06:17:05.283 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.283 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.283 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.283 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182427+0000", 2024-09-14T06:17:05.283 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7711999999999999e-05 2024-09-14T06:17:05.283 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.283 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.284 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.284 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.284 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.284 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13248 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.284 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:05.284 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.081017786, 2024-09-14T06:17:05.284 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966527999999997, 2024-09-14T06:17:05.284 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.284 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.284 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.284 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.284 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.284 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.284 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13248 2024-09-14T06:17:05.284 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.285 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.285 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.285 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.285 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:05.285 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.285 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.285 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.285 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.285 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:05.285 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.285 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.285 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.285 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.285 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:05.285 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:17:05.286 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.286 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.286 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.286 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100548+0000", 2024-09-14T06:17:05.286 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:17:05.286 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.286 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.286 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.286 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100557+0000", 2024-09-14T06:17:05.286 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:17:05.286 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.286 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.286 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.286 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100563+0000", 2024-09-14T06:17:05.287 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2519999999999996e-06 2024-09-14T06:17:05.287 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.287 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.287 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.287 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175572+0000", 2024-09-14T06:17:05.287 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075008459 2024-09-14T06:17:05.287 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.287 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.287 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.287 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175656+0000", 2024-09-14T06:17:05.287 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3906999999999995e-05 2024-09-14T06:17:05.287 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.287 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176010+0000", 2024-09-14T06:17:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00035447499999999998 2024-09-14T06:17:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178304+0000", 2024-09-14T06:17:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0022941379999999998 2024-09-14T06:17:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182467+0000", 2024-09-14T06:17:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041632020000000004 2024-09-14T06:17:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.288 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.289 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.289 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182491+0000", 2024-09-14T06:17:05.289 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3733000000000001e-05 2024-09-14T06:17:05.289 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.289 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.289 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.289 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182509+0000", 2024-09-14T06:17:05.289 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8113000000000001e-05 2024-09-14T06:17:05.289 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.289 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.289 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.289 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.289 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.289 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13249 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.290 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:05.290 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.080982851999998, 2024-09-14T06:17:05.290 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082037225000000005, 2024-09-14T06:17:05.290 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.290 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.290 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.290 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.290 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.290 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.290 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13249 2024-09-14T06:17:05.290 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.290 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.290 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.290 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.291 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:05.291 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.291 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.291 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.291 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.291 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:05.291 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.291 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.291 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.291 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.291 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:05.291 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4600000000000005e-07 2024-09-14T06:17:05.291 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.291 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.292 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.292 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100584+0000", 2024-09-14T06:17:05.292 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.947e-06 2024-09-14T06:17:05.292 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.292 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.292 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.292 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100593+0000", 2024-09-14T06:17:05.292 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9760000000000001e-06 2024-09-14T06:17:05.292 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.292 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.292 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.292 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100597+0000", 2024-09-14T06:17:05.292 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9689999999999996e-06 2024-09-14T06:17:05.293 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.293 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.293 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.293 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176461+0000", 2024-09-14T06:17:05.293 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075864132000000001 2024-09-14T06:17:05.293 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.293 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.293 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.293 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176516+0000", 2024-09-14T06:17:05.293 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4416999999999998e-05 2024-09-14T06:17:05.293 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.293 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.293 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.293 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176734+0000", 2024-09-14T06:17:05.294 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021859399999999999 2024-09-14T06:17:05.294 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.294 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.294 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.294 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178308+0000", 2024-09-14T06:17:05.294 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015733660000000001 2024-09-14T06:17:05.294 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.294 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.294 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.294 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182573+0000", 2024-09-14T06:17:05.294 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042649200000000002 2024-09-14T06:17:05.294 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.294 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.294 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.295 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182597+0000", 2024-09-14T06:17:05.295 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4355e-05 2024-09-14T06:17:05.295 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.295 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.295 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.295 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182615+0000", 2024-09-14T06:17:05.295 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8003e-05 2024-09-14T06:17:05.295 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.295 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.295 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.295 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.295 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.295 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13250 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:05.295 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:05.296 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 29.080818247, 2024-09-14T06:17:05.296 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081983890000000004, 2024-09-14T06:17:05.296 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.296 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.296 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.296 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.296 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.296 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.296 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13250 2024-09-14T06:17:05.296 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.296 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.296 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.296 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.297 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:05.297 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.297 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.297 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.297 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.297 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:05.297 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.297 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.297 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.297 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.297 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100743+0000", 2024-09-14T06:17:05.297 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:17:05.297 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.297 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.297 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.298 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100753+0000", 2024-09-14T06:17:05.298 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9979999999999998e-06 2024-09-14T06:17:05.298 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.298 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.298 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.298 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100772+0000", 2024-09-14T06:17:05.298 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8664e-05 2024-09-14T06:17:05.298 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.298 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.298 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.298 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100779+0000", 2024-09-14T06:17:05.298 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.046e-06 2024-09-14T06:17:05.298 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.298 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.299 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.299 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177049+0000", 2024-09-14T06:17:05.299 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076270819000000004 2024-09-14T06:17:05.299 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.299 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.299 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.299 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177140+0000", 2024-09-14T06:17:05.299 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0746000000000002e-05 2024-09-14T06:17:05.299 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.299 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.299 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:05.299 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177519+0000", 2024-09-14T06:17:05.299 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378656 2024-09-14T06:17:05.299 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.299 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.299 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.300 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178349+0000", 2024-09-14T06:17:05.300 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00083061599999999995 2024-09-14T06:17:05.300 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.300 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.300 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.300 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182679+0000", 2024-09-14T06:17:05.300 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043298379999999999 2024-09-14T06:17:05.300 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.300 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.300 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.300 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182705+0000", 2024-09-14T06:17:05.300 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5602999999999999e-05 2024-09-14T06:17:05.300 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.301 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.301 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.301 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182726+0000", 2024-09-14T06:17:05.301 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1297e-05 2024-09-14T06:17:05.301 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.301 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.301 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.301 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.301 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.301 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:05.302 INFO:teuthology.orchestra.run.smithi039.stdout:3383 trunc from 0xcd1d34f to 0x28c109f 2024-09-14T06:17:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:05.304 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:17:05.304 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:17:05.305 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-09-14T06:17:05.306 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-09-14T06:17:04.398273+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":726856229,"num_objects":720,"num_object_clones":122,"num_object_copies":1440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":720,"num_whiteouts":91,"num_read":48411,"num_read_kb":353309,"num_write":8708,"num_write_kb":222495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":16580809,"num_keys_recovered":20,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6981,"ondisk_log_size":6981,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":326400,"kb_used_data":43588,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754648320,"statfs":{"total":773094113280,"available":772759879680,"internally_reserved":0,"allocated":44634112,"data_stored":376947334,"data_compressed":12178032,"data_compressed_allocated":26361856,"data_compressed_original":363466782,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":165642336,"num_objects":32,"num_object_clones":11,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":1,"num_read":3381,"num_read_kb":34068,"num_write":581,"num_write_kb":18021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000951"},"pg_stats":[{"pgid":"3.1f","version":"55'280","reported_seq":1382,"reported_epoch":55,"state":"peering","last_fresh":"2024-09-14T06:17:01.131582+0000","last_change":"2024-09-14T06:17:01.184422+0000","last_active":"2024-09-14T06:17:01.184422+0000","last_peered":"2024-09-14T06:17:01.131582+0000","last_clean":"2024-09-14T06:16:59.040241+0000","last_became_active":"2024-09-14T06:17:00.187916+0000","last_became_peered":"2024-09-14T06:17:00.187916+0000","last_unstale":"2024-09-14T06:17:01.131582+0000","last_undegraded":"2024-09-14T06:17:01.131582+0000","last_fullsized":"2024-09-14T06:17:00.172596+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010466430000000001,"stat_sum":{"num_bytes":30765352,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":1,"num_read":2138,"num_read_kb":13321,"num_write":356,"num_write_kb":6653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"56'188","reported_seq":904,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.218294+0000","last_change":"2024-09-14T06:16:27.310639+0000","last_active":"2024-09-14T06:17:01.218294+0000","last_peered":"2024-09-14T06:17:01.218294+0000","last_clean":"2024-09-14T06:17:01.218294+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:17:01.218294+0000","last_undegraded":"2024-09-14T06:17:01.218294+0000","last_fullsized":"2024-09-14T06:17:01.218294+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:20:52.457557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095849699999999999,"stat_sum":{"num_bytes":29115872,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":1255,"num_read_kb":7674,"num_write":196,"num_write_kb":10926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"49'107","reported_seq":656,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.186311+0000","last_change":"2024-09-14T06:16:22.322492+0000","last_active":"2024-09-14T06:17:02.186311+0000","last_peered":"2024-09-14T06:17:02.186311+0000","last_clean":"2024-09-14T06:17:02.186311+0000","last_became_active":"2024-09-14T06:16:20.192807+0000","last_became_peered":"2024-09-14T06:16:20.192807+0000","last_unstale":"2024-09-14T06:17:02.186311+0000","last_undegraded":"2024-09-14T06:17:02.186311+0000","last_fullsized":"2024-09-14T06:17:02.186311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:46:23.310155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001319401,"stat_sum":{"num_bytes":22444329,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":2,"num_read":484,"num_read_kb":9487,"num_write":109,"num_write_kb":5035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"57'737","reported_seq":1612,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.569871+0000","last_change":"2024-09-14T06:16:22.201834+0000","last_active":"2024-09-14T06:17:02.569871+0000","last_peered":"2024-09-14T06:17:02.569871+0000","last_clean":"2024-09-14T06:17:02.569871+0000","last_became_active":"2024-09-14T06:16:20.190215+0000","last_became_peered":"2024-09-14T06:16:20.190215+0000","last_unstale":"2024-09-14T06:17:02.569871+0000","last_undegraded":"2024-09-14T06:17:02.569871+0000","last_fullsized":"2024-09-14T06:17:02.569871+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:36:10.080988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026065160000000001,"stat_sum":{"num_bytes":12365597,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":2,"num_read":4761,"num_read_kb":4977,"num_write":1148,"num_write_kb":2539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"55'129","reported_seq":603,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.180455+0000","last_change":"2024-09-14T06:16:22.193116+0000","last_active":"2024-09-14T06:17:01.180455+0000","last_peered":"2024-09-14T06:17:01.180455+0000","last_clean":"2024-09-14T06:17:01.180455+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:17:01.180455+0000","last_undegraded":"2024-09-14T06:17:01.180455+0000","last_fullsized":"2024-09-14T06:17:01.180455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:55:07.318171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014745120000000001,"stat_sum":{"num_bytes":32994688,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":3,"num_read":420,"num_read_kb":11922,"num_write":125,"num_write_kb":11647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"50'388","reported_seq":1736,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.582229+0000","last_change":"2024-09-14T06:16:22.201461+0000","last_active":"2024-09-14T06:17:02.582229+0000","last_peered":"2024-09-14T06:17:02.582229+0000","last_clean":"2024-09-14T06:17:02.582229+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:17:02.582229+0000","last_undegraded":"2024-09-14T06:17:02.582229+0000","last_fullsized":"2024-09-14T06:17:02.582229+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:23.114266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002267542,"stat_sum":{"num_bytes":9541739,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":4,"num_read":2754,"num_read_kb":9963,"num_write":547,"num_write_kb":6183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"52'364","reported_seq":1378,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.180175+0000","last_change":"2024-09-14T06:16:22.191773+0000","last_active":"2024-09-14T06:17:01.180175+0000","last_peered":"2024-09-14T06:17:01.180175+0000","last_clean":"2024-09-14T06:17:01.180175+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:17:01.180175+0000","last_undegraded":"2024-09-14T06:17:01.180175+0000","last_fullsized":"2024-09-14T06:17:01.180175+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:54:59.817800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073833899999999997,"stat_sum":{"num_bytes":10887305,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":4,"num_read":2376,"num_read_kb":8170,"num_write":567,"num_write_kb":6461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"56'184","reported_seq":1087,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.192557+0000","last_change":"2024-09-14T06:16:22.205296+0000","last_active":"2024-09-14T06:17:01.192557+0000","last_peered":"2024-09-14T06:17:01.192557+0000","last_clean":"2024-09-14T06:17:01.192557+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:17:01.192557+0000","last_undegraded":"2024-09-14T06:17:01.192557+0000","last_fullsized":"2024-09-14T06:17:01.192557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:59:20.625079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012474280000000001,"stat_sum":{"num_bytes":34514109,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":2,"num_read":1765,"num_read_kb":19853,"num_write":175,"num_write_kb":12034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"52'142","reported_seq":578,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.185044+0000","last_change":"2024-09-14T06:16:22.204961+0000","last_active":"2024-09-14T06:17:01.185044+0000","last_peered":"2024-09-14T06:17:01.185044+0000","last_clean":"2024-09-14T06:17:01.185044+0000","last_became_active":"2024-09-14T06:16:20.187444+0000","last_became_peered":"2024-09-14T06:16:20.187444+0000","last_unstale":"2024-09-14T06:17:01.185044+0000","last_undegraded":"2024-09-14T06:17:01.185044+0000","last_fullsized":"2024-09-14T06:17:01.185044+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:03:35.600597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001106627,"stat_sum":{"num_bytes":16289478,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":4,"num_read":582,"num_read_kb":16001,"num_write":176,"num_write_kb":13048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"57'155","reported_seq":657,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.585665+0000","last_change":"2024-09-14T06:16:33.544175+0000","last_active":"2024-09-14T06:17:02.585665+0000","last_peered":"2024-09-14T06:17:02.585665+0000","last_clean":"2024-09-14T06:17:02.585665+0000","last_became_active":"2024-09-14T06:16:33.543480+0000","last_became_peered":"2024-09-14T06:16:33.543480+0000","last_unstale":"2024-09-14T06:17:02.585665+0000","last_undegraded":"2024-09-14T06:17:02.585665+0000","last_fullsized":"2024-09-14T06:17:02.585665+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:15:55.981598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001557389,"stat_sum":{"num_bytes":24085328,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":4,"num_read":445,"num_read_kb":6201,"num_write":162,"num_write_kb":5381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"57'284","reported_seq":962,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.505743+0000","last_change":"2024-09-14T06:16:22.194304+0000","last_active":"2024-09-14T06:17:02.505743+0000","last_peered":"2024-09-14T06:17:02.505743+0000","last_clean":"2024-09-14T06:17:02.505743+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:17:02.505743+0000","last_undegraded":"2024-09-14T06:17:02.505743+0000","last_fullsized":"2024-09-14T06:17:02.505743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:12:32.883667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019733379999999998,"stat_sum":{"num_bytes":23986227,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":2,"num_read":1571,"num_read_kb":11333,"num_write":379,"num_write_kb":7264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"50'133","reported_seq":333,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.253317+0000","last_change":"2024-09-14T06:16:40.063145+0000","last_active":"2024-09-14T06:17:02.253317+0000","last_peered":"2024-09-14T06:17:02.253317+0000","last_clean":"2024-09-14T06:17:02.253317+0000","last_became_active":"2024-09-14T06:16:39.975752+0000","last_became_peered":"2024-09-14T06:16:39.975752+0000","last_unstale":"2024-09-14T06:17:02.253317+0000","last_undegraded":"2024-09-14T06:17:02.253317+0000","last_fullsized":"2024-09-14T06:17:02.253317+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:31:37.419339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001465247,"stat_sum":{"num_bytes":24408554,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1126,"num_read_kb":20997,"num_write":138,"num_write_kb":11501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":9287382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.186123+0000","last_change":"2024-09-14T06:16:12.179717+0000","last_active":"2024-09-14T06:17:02.186123+0000","last_peered":"2024-09-14T06:17:02.186123+0000","last_clean":"2024-09-14T06:17:02.186123+0000","last_became_active":"2024-09-14T06:16:10.150426+0000","last_became_peered":"2024-09-14T06:16:10.150426+0000","last_unstale":"2024-09-14T06:17:02.186123+0000","last_undegraded":"2024-09-14T06:17:02.186123+0000","last_fullsized":"2024-09-14T06:17:02.186123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:29:55.403337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010688049999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"52'280","reported_seq":739,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.180241+0000","last_change":"2024-09-14T06:16:22.192972+0000","last_active":"2024-09-14T06:17:01.180241+0000","last_peered":"2024-09-14T06:17:01.180241+0000","last_clean":"2024-09-14T06:17:01.180241+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:17:01.180241+0000","last_undegraded":"2024-09-14T06:17:01.180241+0000","last_fullsized":"2024-09-14T06:17:01.180241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:27:34.064391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086504599999999996,"stat_sum":{"num_bytes":27122218,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":4,"num_read":1229,"num_read_kb":9138,"num_write":341,"num_write_kb":8098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":88,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.180510+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:17:01.180510+0000","last_peered":"2024-09-14T06:17:01.180510+0000","last_clean":"2024-09-14T06:17:01.180510+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:17:01.180510+0000","last_undegraded":"2024-09-14T06:17:01.180510+0000","last_fullsized":"2024-09-14T06:17:01.180510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"50'169","reported_seq":1970,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.180478+0000","last_change":"2024-09-14T06:16:22.192683+0000","last_active":"2024-09-14T06:17:01.180478+0000","last_peered":"2024-09-14T06:17:01.180478+0000","last_clean":"2024-09-14T06:17:01.180478+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:17:01.180478+0000","last_undegraded":"2024-09-14T06:17:01.180478+0000","last_fullsized":"2024-09-14T06:17:01.180478+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:33:36.901307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001328716,"stat_sum":{"num_bytes":21871443,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":4,"num_read":2297,"num_read_kb":16326,"num_write":160,"num_write_kb":4259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.186603+0000","last_change":"2024-09-14T06:16:12.180261+0000","last_active":"2024-09-14T06:17:02.186603+0000","last_peered":"2024-09-14T06:17:02.186603+0000","last_clean":"2024-09-14T06:17:02.186603+0000","last_became_active":"2024-09-14T06:16:10.147337+0000","last_became_peered":"2024-09-14T06:16:10.147337+0000","last_unstale":"2024-09-14T06:17:02.186603+0000","last_undegraded":"2024-09-14T06:17:02.186603+0000","last_fullsized":"2024-09-14T06:17:02.186603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:54:30.677581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001722187,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"49'146","reported_seq":1164,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.188056+0000","last_change":"2024-09-14T06:16:22.213152+0000","last_active":"2024-09-14T06:17:01.188056+0000","last_peered":"2024-09-14T06:17:01.188056+0000","last_clean":"2024-09-14T06:17:01.188056+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:17:01.188056+0000","last_undegraded":"2024-09-14T06:17:01.188056+0000","last_fullsized":"2024-09-14T06:17:01.188056+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:20:40.134887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057590699999999996,"stat_sum":{"num_bytes":36266956,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":3,"num_read":964,"num_read_kb":10897,"num_write":141,"num_write_kb":5884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":88,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.183885+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:17:01.183885+0000","last_peered":"2024-09-14T06:17:01.183885+0000","last_clean":"2024-09-14T06:17:01.183885+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:17:01.183885+0000","last_undegraded":"2024-09-14T06:17:01.183885+0000","last_fullsized":"2024-09-14T06:17:01.183885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"50'98","reported_seq":1086,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.188261+0000","last_change":"2024-09-14T06:16:33.548583+0000","last_active":"2024-09-14T06:17:01.188261+0000","last_peered":"2024-09-14T06:17:01.188261+0000","last_clean":"2024-09-14T06:17:01.188261+0000","last_became_active":"2024-09-14T06:16:33.547925+0000","last_became_peered":"2024-09-14T06:16:33.547925+0000","last_unstale":"2024-09-14T06:17:01.188261+0000","last_undegraded":"2024-09-14T06:17:01.188261+0000","last_fullsized":"2024-09-14T06:17:01.188261+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:55.647390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089329700000000004,"stat_sum":{"num_bytes":30292586,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":984,"num_read_kb":20409,"num_write":115,"num_write_kb":8085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"57'92","reported_seq":437,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.569708+0000","last_change":"2024-09-14T06:16:27.282745+0000","last_active":"2024-09-14T06:17:02.569708+0000","last_peered":"2024-09-14T06:17:02.569708+0000","last_clean":"2024-09-14T06:17:02.569708+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:17:02.569708+0000","last_undegraded":"2024-09-14T06:17:02.569708+0000","last_fullsized":"2024-09-14T06:17:02.569708+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:28:53.070968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002354358,"stat_sum":{"num_bytes":22997810,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":258,"num_read_kb":3653,"num_write":76,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.185633+0000","last_change":"2024-09-14T06:16:12.182029+0000","last_active":"2024-09-14T06:17:02.185633+0000","last_peered":"2024-09-14T06:17:02.185633+0000","last_clean":"2024-09-14T06:17:02.185633+0000","last_became_active":"2024-09-14T06:16:10.196390+0000","last_became_peered":"2024-09-14T06:16:10.196390+0000","last_unstale":"2024-09-14T06:17:02.185633+0000","last_undegraded":"2024-09-14T06:17:02.185633+0000","last_fullsized":"2024-09-14T06:17:02.185633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:21.237087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034648000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"56'271","reported_seq":1404,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.209270+0000","last_change":"2024-09-14T06:16:22.205051+0000","last_active":"2024-09-14T06:17:01.209270+0000","last_peered":"2024-09-14T06:17:01.209270+0000","last_clean":"2024-09-14T06:17:01.209270+0000","last_became_active":"2024-09-14T06:16:20.196215+0000","last_became_peered":"2024-09-14T06:16:20.196215+0000","last_unstale":"2024-09-14T06:17:01.209270+0000","last_undegraded":"2024-09-14T06:17:01.209270+0000","last_fullsized":"2024-09-14T06:17:01.209270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:20:21.226839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001116559,"stat_sum":{"num_bytes":31348115,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":2,"num_read":1897,"num_read_kb":18707,"num_write":393,"num_write_kb":10096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.210037+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:17:01.210037+0000","last_peered":"2024-09-14T06:17:01.210037+0000","last_clean":"2024-09-14T06:17:01.210037+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:17:01.210037+0000","last_undegraded":"2024-09-14T06:17:01.210037+0000","last_fullsized":"2024-09-14T06:17:01.210037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"50'299","reported_seq":574,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.187942+0000","last_change":"2024-09-14T06:16:27.316499+0000","last_active":"2024-09-14T06:17:01.187942+0000","last_peered":"2024-09-14T06:17:01.187942+0000","last_clean":"2024-09-14T06:17:01.187942+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:17:01.187942+0000","last_undegraded":"2024-09-14T06:17:01.187942+0000","last_fullsized":"2024-09-14T06:17:01.187942+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:41:53.928895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052773999999999996,"stat_sum":{"num_bytes":9782492,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":2,"num_read":1219,"num_read_kb":4826,"num_write":441,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":90,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.185251+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:17:02.185251+0000","last_peered":"2024-09-14T06:17:02.185251+0000","last_clean":"2024-09-14T06:17:02.185251+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:17:02.185251+0000","last_undegraded":"2024-09-14T06:17:02.185251+0000","last_fullsized":"2024-09-14T06:17:02.185251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"56'177","reported_seq":792,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.180292+0000","last_change":"2024-09-14T06:16:27.302045+0000","last_active":"2024-09-14T06:17:01.180292+0000","last_peered":"2024-09-14T06:17:01.180292+0000","last_clean":"2024-09-14T06:17:01.180292+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:17:01.180292+0000","last_undegraded":"2024-09-14T06:17:01.180292+0000","last_fullsized":"2024-09-14T06:17:01.180292+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:13:10.694404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060103200000000004,"stat_sum":{"num_bytes":34716266,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":3,"num_read":600,"num_read_kb":9071,"num_write":174,"num_write_kb":5280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":111,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.185834+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:17:02.185834+0000","last_peered":"2024-09-14T06:17:02.185834+0000","last_clean":"2024-09-14T06:17:02.185834+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:17:02.185834+0000","last_undegraded":"2024-09-14T06:17:02.185834+0000","last_fullsized":"2024-09-14T06:17:02.185834+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":181,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.253571+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:17:02.253571+0000","last_peered":"2024-09-14T06:17:02.253571+0000","last_clean":"2024-09-14T06:17:02.253571+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:17:02.253571+0000","last_undegraded":"2024-09-14T06:17:02.253571+0000","last_fullsized":"2024-09-14T06:17:02.253571+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"52'127","reported_seq":330,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.180124+0000","last_change":"2024-09-14T06:16:22.191260+0000","last_active":"2024-09-14T06:17:01.180124+0000","last_peered":"2024-09-14T06:17:01.180124+0000","last_clean":"2024-09-14T06:17:01.180124+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:17:01.180124+0000","last_undegraded":"2024-09-14T06:17:01.180124+0000","last_fullsized":"2024-09-14T06:17:01.180124+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:19:08.910375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047498199999999999,"stat_sum":{"num_bytes":9165317,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":332,"num_read_kb":3423,"num_write":144,"num_write_kb":3018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"52'82","reported_seq":378,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.185417+0000","last_change":"2024-09-14T06:16:22.322878+0000","last_active":"2024-09-14T06:17:02.185417+0000","last_peered":"2024-09-14T06:17:02.185417+0000","last_clean":"2024-09-14T06:17:02.185417+0000","last_became_active":"2024-09-14T06:16:20.196837+0000","last_became_peered":"2024-09-14T06:16:20.196837+0000","last_unstale":"2024-09-14T06:17:02.185417+0000","last_undegraded":"2024-09-14T06:17:02.185417+0000","last_fullsized":"2024-09-14T06:17:02.185417+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:14:06.503881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418299,"stat_sum":{"num_bytes":29986833,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":2,"num_read":234,"num_read_kb":3414,"num_write":70,"num_write_kb":4055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"57'166","reported_seq":1120,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.569347+0000","last_change":"2024-09-14T06:16:22.323725+0000","last_active":"2024-09-14T06:17:02.569347+0000","last_peered":"2024-09-14T06:17:02.569347+0000","last_clean":"2024-09-14T06:17:02.569347+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:17:02.569347+0000","last_undegraded":"2024-09-14T06:17:02.569347+0000","last_fullsized":"2024-09-14T06:17:02.569347+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:54:03.911067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024221899999999998,"stat_sum":{"num_bytes":30606945,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1433,"num_read_kb":8551,"num_write":153,"num_write_kb":4440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"50'72","reported_seq":954,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.210037+0000","last_change":"2024-09-14T06:16:22.219322+0000","last_active":"2024-09-14T06:17:01.210037+0000","last_peered":"2024-09-14T06:17:01.210037+0000","last_clean":"2024-09-14T06:17:01.210037+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:17:01.210037+0000","last_undegraded":"2024-09-14T06:17:01.210037+0000","last_fullsized":"2024-09-14T06:17:01.210037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:23:54.631579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045017199999999999,"stat_sum":{"num_bytes":11504344,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":2,"num_read":829,"num_read_kb":11742,"num_write":75,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"50'56","reported_seq":1081,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.217199+0000","last_change":"2024-09-14T06:16:22.195705+0000","last_active":"2024-09-14T06:17:02.217199+0000","last_peered":"2024-09-14T06:17:02.217199+0000","last_clean":"2024-09-14T06:17:02.217199+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:17:02.217199+0000","last_undegraded":"2024-09-14T06:17:02.217199+0000","last_fullsized":"2024-09-14T06:17:02.217199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:09:20.682530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025578329999999998,"stat_sum":{"num_bytes":22004370,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":1,"num_read":951,"num_read_kb":14136,"num_write":80,"num_write_kb":5029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"52'495","reported_seq":1258,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.186505+0000","last_change":"2024-09-14T06:16:22.323126+0000","last_active":"2024-09-14T06:17:02.186505+0000","last_peered":"2024-09-14T06:17:02.186505+0000","last_clean":"2024-09-14T06:17:02.186505+0000","last_became_active":"2024-09-14T06:16:20.190580+0000","last_became_peered":"2024-09-14T06:16:20.190580+0000","last_unstale":"2024-09-14T06:17:02.186505+0000","last_undegraded":"2024-09-14T06:17:02.186505+0000","last_fullsized":"2024-09-14T06:17:02.186505+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:49:50.845278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001627671,"stat_sum":{"num_bytes":28396948,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":3,"num_read":2908,"num_read_kb":19334,"num_write":822,"num_write_kb":16796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"52'260","reported_seq":1573,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.217163+0000","last_change":"2024-09-14T06:16:33.547557+0000","last_active":"2024-09-14T06:17:02.217163+0000","last_peered":"2024-09-14T06:17:02.217163+0000","last_clean":"2024-09-14T06:17:02.217163+0000","last_became_active":"2024-09-14T06:16:33.546796+0000","last_became_peered":"2024-09-14T06:16:33.546796+0000","last_unstale":"2024-09-14T06:17:02.217163+0000","last_undegraded":"2024-09-14T06:17:02.217163+0000","last_fullsized":"2024-09-14T06:17:02.217163+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:25:25.206196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026669160000000001,"stat_sum":{"num_bytes":13460745,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":1,"num_read":3509,"num_read_kb":10097,"num_write":245,"num_write_kb":6854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"51'153","reported_seq":1379,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.217176+0000","last_change":"2024-09-14T06:16:22.195247+0000","last_active":"2024-09-14T06:17:02.217176+0000","last_peered":"2024-09-14T06:17:02.217176+0000","last_clean":"2024-09-14T06:17:02.217176+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:17:02.217176+0000","last_undegraded":"2024-09-14T06:17:02.217176+0000","last_fullsized":"2024-09-14T06:17:02.217176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:40:06.024842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022870590000000001,"stat_sum":{"num_bytes":28493429,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":3,"num_read":1236,"num_read_kb":13136,"num_write":147,"num_write_kb":9843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"52'150","reported_seq":807,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.253356+0000","last_change":"2024-09-14T06:16:22.200094+0000","last_active":"2024-09-14T06:17:02.253356+0000","last_peered":"2024-09-14T06:17:02.253356+0000","last_clean":"2024-09-14T06:17:02.253356+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:17:02.253356+0000","last_undegraded":"2024-09-14T06:17:02.253356+0000","last_fullsized":"2024-09-14T06:17:02.253356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:30:33.790500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011343239999999999,"stat_sum":{"num_bytes":13377782,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":4,"num_read":869,"num_read_kb":7535,"num_write":137,"num_write_kb":3559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"52'187","reported_seq":1212,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.185317+0000","last_change":"2024-09-14T06:16:27.331344+0000","last_active":"2024-09-14T06:17:02.185317+0000","last_peered":"2024-09-14T06:17:02.185317+0000","last_clean":"2024-09-14T06:17:02.185317+0000","last_became_active":"2024-09-14T06:16:27.251848+0000","last_became_peered":"2024-09-14T06:16:27.251848+0000","last_unstale":"2024-09-14T06:17:02.185317+0000","last_undegraded":"2024-09-14T06:17:02.185317+0000","last_fullsized":"2024-09-14T06:17:02.185317+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:33:50.573056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226324,"stat_sum":{"num_bytes":14705813,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":6,"num_read":1695,"num_read_kb":10357,"num_write":222,"num_write_kb":5967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":66525,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"52'274","reported_seq":866,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T06:17:01.187956+0000","last_change":"2024-09-14T06:16:22.213466+0000","last_active":"2024-09-14T06:17:01.187956+0000","last_peered":"2024-09-14T06:17:01.187956+0000","last_clean":"2024-09-14T06:17:01.187956+0000","last_became_active":"2024-09-14T06:16:20.197825+0000","last_became_peered":"2024-09-14T06:16:20.197825+0000","last_unstale":"2024-09-14T06:17:01.187956+0000","last_undegraded":"2024-09-14T06:17:01.187956+0000","last_fullsized":"2024-09-14T06:17:01.187956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:05:46.234808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082016299999999995,"stat_sum":{"num_bytes":16438397,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":4,"num_read":1455,"num_read_kb":6113,"num_write":344,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"52'261","reported_seq":2278,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-14T06:17:02.185742+0000","last_change":"2024-09-14T06:16:22.322160+0000","last_active":"2024-09-14T06:17:02.185742+0000","last_peered":"2024-09-14T06:17:02.185742+0000","last_clean":"2024-09-14T06:17:02.185742+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:17:02.185742+0000","last_undegraded":"2024-09-14T06:17:02.185742+0000","last_fullsized":"2024-09-14T06:17:02.185742+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:17:41.183118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011502319999999999,"stat_sum":{"num_bytes":22328455,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":5,"num_read":3769,"num_read_kb":12490,"num_write":247,"num_write_kb":4229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":726265842,"num_objects":716,"num_object_clones":122,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":716,"num_whiteouts":91,"num_read":48345,"num_read_kb":353254,"num_write":8605,"num_write_kb":220865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":16580809,"num_keys_recovered":20,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":39677952,"data_stored":373824778,"data_compressed":12157963,"data_compressed_allocated":26288128,"data_compressed_original":362381342,"omap_allocated":0,"internal_metadata":0},"log_size":6906,"ondisk_log_size":6906,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45844,"kb_used_data":6332,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325996,"statfs":{"total":96636764160,"available":96589819904,"internally_reserved":0,"allocated":6483968,"data_stored":58634775,"data_compressed":2154373,"data_compressed_allocated":4276224,"data_compressed_original":56998688,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46228,"kb_used_data":6712,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325612,"statfs":{"total":96636764160,"available":96589426688,"internally_reserved":0,"allocated":6873088,"data_stored":53760652,"data_compressed":1418150,"data_compressed_allocated":3813376,"data_compressed_original":51483794,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":5296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":5423104,"data_stored":48486406,"data_compressed":1435512,"data_compressed_allocated":3534848,"data_compressed_original":47146323,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45196,"kb_used_data":5684,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326644,"statfs":{"total":96636764160,"available":96590483456,"internally_reserved":0,"allocated":5820416,"data_stored":47482286,"data_compressed":1366427,"data_compressed_allocated":3256320,"data_compressed_original":45433045,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34748,"kb_used_data":3564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337092,"statfs":{"total":96636764160,"available":96601182208,"internally_reserved":0,"allocated":3649536,"data_stored":29524682,"data_compressed":1172489,"data_compressed_allocated":1990656,"data_compressed_original":28350455,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41052,"kb_used_data":5704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330788,"statfs":{"total":96636764160,"available":96594726912,"internally_reserved":0,"allocated":5840896,"data_stored":44862740,"data_compressed":1648751,"data_compressed_allocated":3219456,"data_compressed_original":42939976,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41584,"kb_used_data":6240,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330256,"statfs":{"total":96636764160,"available":96594182144,"internally_reserved":0,"allocated":6389760,"data_stored":59293166,"data_compressed":2038263,"data_compressed_allocated":4149248,"data_compressed_original":57702385,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39416,"kb_used_data":4056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332424,"statfs":{"total":96636764160,"available":96596402176,"internally_reserved":0,"allocated":4153344,"data_stored":34902627,"data_compressed":944067,"data_compressed_allocated":2121728,"data_compressed_original":33412116,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5414912,"data_stored":47446888,"data_compressed":1366590,"data_compressed_allocated":3260416,"data_compressed_original":45465813,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4698112,"data_stored":47584153,"data_compressed":1425207,"data_compressed_allocated":3489792,"data_compressed_original":46552403,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6221824,"data_stored":53473959,"data_compressed":1418150,"data_compressed_allocated":3813376,"data_compressed_original":51483794,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5840896,"data_stored":58377168,"data_compressed":2154504,"data_compressed_allocated":4280320,"data_compressed_original":57023264,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3514368,"data_stored":34621805,"data_compressed":944067,"data_compressed_allocated":2121728,"data_compressed_original":33412116,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5767168,"data_stored":58489154,"data_compressed":2028205,"data_compressed_allocated":4112384,"data_compressed_original":57153521,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5197824,"data_stored":44581899,"data_compressed":1648751,"data_compressed_allocated":3219456,"data_compressed_original":42939976,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3022848,"data_stored":29249752,"data_compressed":1172489,"data_compressed_allocated":1990656,"data_compressed_original":28350455,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:17:05.307 INFO:tasks.thrashosds.thrasher:Setting 3.9 to [3, 4, 6, 7] 2024-09-14T06:17:05.307 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.9', '3', '4', '6', '7'] 2024-09-14T06:17:05.308 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.9 3 4 6 7 2024-09-14T06:17:05.308 INFO:teuthology.orchestra.run.smithi039.stdout:3384 trunc from 0x28c109f to 0x8e4291b 2024-09-14T06:17:05.309 INFO:teuthology.orchestra.run.smithi039.stdout:3385 writesame 0x142060d thru 0x1423a98 data_size 0x4c(0x348c bytes) 2024-09-14T06:17:05.309 INFO:teuthology.orchestra.run.smithi039.stdout:3386 read 0x69b3fbd thru 0x69b5542 (0x1586 bytes) 2024-09-14T06:17:05.309 INFO:teuthology.orchestra.run.smithi039.stdout:3387 trunc from 0x8e4291b to 0x9da0cf4 2024-09-14T06:17:05.309 INFO:teuthology.orchestra.run.smithi039.stdout:3388 punch from 0x7d65dbe to 0x7d65e82, (0xc4 bytes) 2024-09-14T06:17:05.309 INFO:teuthology.orchestra.run.smithi039.stdout:3389 trunc from 0x9da0cf4 to 0xd7de39b 2024-09-14T06:17:05.310 INFO:teuthology.orchestra.run.smithi039.stdout:3390 write 0xddf71a3 thru 0xde06de8 (0xfc46 bytes) 2024-09-14T06:17:05.310 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:05.298+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:05.310 INFO:teuthology.orchestra.run.smithi039.stdout:3391 writesame 0x60b9210 thru 0x60c12f0 data_size 0x1(0x80e1 bytes) 2024-09-14T06:17:05.339 INFO:teuthology.orchestra.run.smithi039.stdout:3392 read 0x94c92db thru 0x94d5cf8 (0xca1e bytes) 2024-09-14T06:17:05.339 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:05.351 INFO:teuthology.orchestra.run.smithi039.stdout:3394 trunc from 0xde06de9 to 0x818f3f6 2024-09-14T06:17:05.368 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:05.368 INFO:teuthology.orchestra.run.smithi039.stdout:3397 read 0x582d620 thru 0x5837f1d (0xa8fe bytes) 2024-09-14T06:17:05.368 INFO:teuthology.orchestra.run.smithi039.stdout:3398 write 0x4297883 thru 0x42a62e7 (0xea65 bytes) 2024-09-14T06:17:05.372 INFO:teuthology.orchestra.run.smithi039.stdout:3399 write 0x2be31ab thru 0x2bed9ea (0xa840 bytes) 2024-09-14T06:17:05.377 INFO:teuthology.orchestra.run.smithi039.stdout:3400 punch from 0x30741aa to 0x308132a, (0xd180 bytes) 2024-09-14T06:17:05.390 INFO:teuthology.orchestra.run.smithi039.stdout:3401 write 0xb065d32 thru 0xb06adf9 (0x50c8 bytes) 2024-09-14T06:17:05.404 INFO:teuthology.orchestra.run.smithi039.stdout:3402 write 0xbdae6b6 thru 0xbdb3068 (0x49b3 bytes) 2024-09-14T06:17:05.404 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:05.402+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:05.428 INFO:teuthology.orchestra.run.smithi039.stdout:3403 trunc from 0xbdb3069 to 0x2a9a1cc 2024-09-14T06:17:05.443 INFO:teuthology.orchestra.run.smithi039.stdout:3404 punch from 0x113ece1 to 0x11400a9, (0x13c8 bytes) 2024-09-14T06:17:05.443 INFO:teuthology.orchestra.run.smithi039.stdout:3405 punch from 0x21f8a7e to 0x2205301, (0xc883 bytes) 2024-09-14T06:17:05.460 INFO:teuthology.orchestra.run.smithi039.stdout:3406 trunc from 0x2a9a1cc to 0xcb6bced 2024-09-14T06:17:05.468 INFO:teuthology.orchestra.run.smithi039.stdout:3407 write 0xa8e05d5 thru 0xa8ee305 (0xdd31 bytes) 2024-09-14T06:17:05.477 INFO:teuthology.orchestra.run.smithi039.stdout:3408 writesame 0x4420e89 thru 0x4425290 data_size 0x1c(0x4408 bytes) 2024-09-14T06:17:05.484 INFO:teuthology.orchestra.run.smithi039.stdout:3409 write 0x196c1f5 thru 0x196e259 (0x2065 bytes) 2024-09-14T06:17:05.485 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:05.485 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:05.485 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:05.485 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:05.490 INFO:teuthology.orchestra.run.smithi039.stdout:3410 read 0x4b8f1c thru 0x4c15ca (0x86af bytes) 2024-09-14T06:17:05.496 INFO:teuthology.orchestra.run.smithi039.stdout:3411 write 0xb189e68 thru 0xb18a979 (0xb12 bytes) 2024-09-14T06:17:05.501 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:17:05.505 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:05.502+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:05.515 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:05.525 INFO:teuthology.orchestra.run.smithi039.stdout:3414 write 0xcf44b48 thru 0xcf475e6 (0x2a9f bytes) 2024-09-14T06:17:05.530 INFO:teuthology.orchestra.run.smithi039.stdout:3415 read 0x63859ef thru 0x638a8d8 (0x4eea bytes) 2024-09-14T06:17:05.539 INFO:teuthology.orchestra.run.smithi039.stdout:3416 trunc from 0xcf475e7 to 0x69cd4b1 2024-09-14T06:17:05.553 INFO:teuthology.orchestra.run.smithi039.stdout:3417 write 0x105f615 thru 0x10601d6 (0xbc2 bytes) 2024-09-14T06:17:05.569 INFO:teuthology.orchestra.run.smithi039.stdout:3418 punch from 0x2a84d5e to 0x2a86e8e, (0x2130 bytes) 2024-09-14T06:17:05.585 INFO:teuthology.orchestra.run.smithi039.stdout:3420 write 0xd90fd14 thru 0xd910066 (0x353 bytes) 2024-09-14T06:17:05.590 INFO:teuthology.orchestra.run.smithi039.stdout:3421 read 0x75a6fc5 thru 0x75a9e97 (0x2ed3 bytes) 2024-09-14T06:17:05.596 INFO:teuthology.orchestra.run.smithi039.stdout:3422 trunc from 0xd910067 to 0x9981f63 2024-09-14T06:17:05.605 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:05.602+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:05.608 INFO:teuthology.orchestra.run.smithi039.stdout:3423 punch from 0x4b61bc to 0x4c2392, (0xc1d6 bytes) 2024-09-14T06:17:05.608 INFO:teuthology.orchestra.run.smithi039.stdout:3424 write 0x27cf593 thru 0x27d29e2 (0x3450 bytes) 2024-09-14T06:17:05.624 INFO:teuthology.orchestra.run.smithi039.stdout:3425 clone 19 order 18 su 131072 sc 6 2024-09-14T06:17:05.669 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:05.670 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:05.670 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:05.670 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:05.670 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:05.683 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:17:05.727 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:05.726+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:05.807 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:05.806+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:05.849 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:05.850 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:05.850 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:05.850 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:05.850 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.850 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:30547 3.1f 3:f84dfb53:::rbd_data.11529eb229bd.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write 16226533~50642] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T06:17:05.850 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:05.850 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 6.6594857630000002, 2024-09-14T06:17:05.850 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0079770020000001, 2024-09-14T06:17:05.850 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.850 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.851 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.851 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.851 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.851 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.851 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 30547 2024-09-14T06:17:05.851 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.851 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.851 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.851 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.851 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:05.851 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.851 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.851 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.851 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.852 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:05.852 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.852 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.852 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.852 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.852 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189811+0000", 2024-09-14T06:17:05.852 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2060000000000001e-06 2024-09-14T06:17:05.852 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.852 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.852 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.852 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189871+0000", 2024-09-14T06:17:05.852 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9862999999999999e-05 2024-09-14T06:17:05.853 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.853 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.853 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.853 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189899+0000", 2024-09-14T06:17:05.853 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7849000000000001e-05 2024-09-14T06:17:05.853 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.853 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.853 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.853 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189915+0000", 2024-09-14T06:17:05.853 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:17:05.853 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.853 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.853 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.854 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190024+0000", 2024-09-14T06:17:05.854 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000109202 2024-09-14T06:17:05.854 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.854 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.854 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:17:05.854 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190041+0000", 2024-09-14T06:17:05.854 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7665000000000001e-05 2024-09-14T06:17:05.854 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.854 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.854 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.854 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188512+0000", 2024-09-14T06:17:05.854 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.99847100700000002 2024-09-14T06:17:05.854 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.854 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.855 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for missing object", 2024-09-14T06:17:05.855 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188646+0000", 2024-09-14T06:17:05.855 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000133986 2024-09-14T06:17:05.855 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.855 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.855 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.855 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193097+0000", 2024-09-14T06:17:05.855 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044507059999999996 2024-09-14T06:17:05.855 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.855 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.855 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.855 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193183+0000", 2024-09-14T06:17:05.855 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5496999999999998e-05 2024-09-14T06:17:05.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.856 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.856 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:17:05.856 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193307+0000", 2024-09-14T06:17:05.856 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000124611 2024-09-14T06:17:05.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.856 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.856 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.856 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.194318+0000", 2024-09-14T06:17:05.856 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0010106 2024-09-14T06:17:05.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.856 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.857 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.857 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197670+0000", 2024-09-14T06:17:05.857 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033523170000000001 2024-09-14T06:17:05.857 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.857 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.857 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.857 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197712+0000", 2024-09-14T06:17:05.857 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1681000000000003e-05 2024-09-14T06:17:05.857 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.857 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.857 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.857 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197786+0000", 2024-09-14T06:17:05.858 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4148999999999998e-05 2024-09-14T06:17:05.858 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.858 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.858 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.858 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.858 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.858 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32015 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.858 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.867830+0000", 2024-09-14T06:17:05.858 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.981464675, 2024-09-14T06:17:05.858 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.091932734000000002, 2024-09-14T06:17:05.858 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.858 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.858 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.859 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.859 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.859 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.859 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32015 2024-09-14T06:17:05.859 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.859 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.859 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.859 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.859 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867830+0000", 2024-09-14T06:17:05.859 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.859 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.859 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.859 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.860 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867830+0000", 2024-09-14T06:17:05.860 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.860 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.860 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.860 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.860 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867831+0000", 2024-09-14T06:17:05.860 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.057e-06 2024-09-14T06:17:05.860 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.860 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.860 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.860 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867834+0000", 2024-09-14T06:17:05.860 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:17:05.860 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.861 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.861 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.861 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867843+0000", 2024-09-14T06:17:05.861 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2399999999999996e-06 2024-09-14T06:17:05.861 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.861 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.861 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.861 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867850+0000", 2024-09-14T06:17:05.861 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9870000000000002e-06 2024-09-14T06:17:05.861 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.861 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.861 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.862 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.959216+0000", 2024-09-14T06:17:05.862 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.091366050000000004 2024-09-14T06:17:05.862 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.862 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.862 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.862 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.959361+0000", 2024-09-14T06:17:05.862 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014463599999999999 2024-09-14T06:17:05.862 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.862 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.862 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.862 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.959763+0000", 2024-09-14T06:17:05.863 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040202499999999997 2024-09-14T06:17:05.863 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.863 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.863 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.863 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.863 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.863 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32016 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6127616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.863 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.867859+0000", 2024-09-14T06:17:05.863 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.981435456, 2024-09-14T06:17:05.863 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.092554549999999999, 2024-09-14T06:17:05.863 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.863 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.863 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.863 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.864 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.864 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.864 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32016 2024-09-14T06:17:05.864 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.864 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.864 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.864 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.864 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867859+0000", 2024-09-14T06:17:05.864 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.864 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.864 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.864 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.864 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867859+0000", 2024-09-14T06:17:05.864 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.865 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.865 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.865 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.865 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867860+0000", 2024-09-14T06:17:05.865 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9599999999999999e-07 2024-09-14T06:17:05.865 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.865 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.865 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.865 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867874+0000", 2024-09-14T06:17:05.865 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3550000000000001e-05 2024-09-14T06:17:05.865 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.865 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.866 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.866 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867886+0000", 2024-09-14T06:17:05.866 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2213e-05 2024-09-14T06:17:05.866 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.866 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.866 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.866 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867901+0000", 2024-09-14T06:17:05.866 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5429000000000001e-05 2024-09-14T06:17:05.866 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.866 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.866 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.866 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.959848+0000", 2024-09-14T06:17:05.866 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.091946813000000002 2024-09-14T06:17:05.866 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.867 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.867 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.867 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.959986+0000", 2024-09-14T06:17:05.867 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000137553 2024-09-14T06:17:05.867 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.867 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.867 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.867 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.960414+0000", 2024-09-14T06:17:05.867 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00042829599999999999 2024-09-14T06:17:05.867 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.867 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.867 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.867 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.867 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.868 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32017 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5668864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.868 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.867911+0000", 2024-09-14T06:17:05.868 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.981383772, 2024-09-14T06:17:05.868 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.093148402000000005, 2024-09-14T06:17:05.868 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.868 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.868 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.868 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.868 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.868 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.868 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32017 2024-09-14T06:17:05.868 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.868 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.869 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.869 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.869 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867911+0000", 2024-09-14T06:17:05.869 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.869 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.869 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.869 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.869 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867911+0000", 2024-09-14T06:17:05.869 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.869 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.869 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.869 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.869 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867912+0000", 2024-09-14T06:17:05.870 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8700000000000005e-07 2024-09-14T06:17:05.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.870 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.870 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867915+0000", 2024-09-14T06:17:05.870 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:17:05.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.870 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.870 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867924+0000", 2024-09-14T06:17:05.870 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.696e-06 2024-09-14T06:17:05.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.871 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.871 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867931+0000", 2024-09-14T06:17:05.871 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0040000000000002e-06 2024-09-14T06:17:05.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.871 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.871 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.871 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.960502+0000", 2024-09-14T06:17:05.871 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.092571102000000002 2024-09-14T06:17:05.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.871 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.871 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.871 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.960651+0000", 2024-09-14T06:17:05.871 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014883600000000001 2024-09-14T06:17:05.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.872 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.872 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.872 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.961059+0000", 2024-09-14T06:17:05.872 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000408121 2024-09-14T06:17:05.872 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.872 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.872 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.872 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.872 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.872 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32018 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.872 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.867946+0000", 2024-09-14T06:17:05.872 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.981348874, 2024-09-14T06:17:05.872 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.093782922000000005, 2024-09-14T06:17:05.872 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.873 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.873 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.873 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.873 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.873 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.873 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32018 2024-09-14T06:17:05.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.873 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.873 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.873 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.873 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867946+0000", 2024-09-14T06:17:05.873 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.874 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.874 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867946+0000", 2024-09-14T06:17:05.874 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.874 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.874 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867947+0000", 2024-09-14T06:17:05.874 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0270000000000001e-06 2024-09-14T06:17:05.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.874 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.874 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867952+0000", 2024-09-14T06:17:05.875 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8069999999999999e-06 2024-09-14T06:17:05.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.875 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.875 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.875 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867970+0000", 2024-09-14T06:17:05.875 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8318999999999999e-05 2024-09-14T06:17:05.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.875 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.875 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.875 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.867983+0000", 2024-09-14T06:17:05.876 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2772000000000001e-05 2024-09-14T06:17:05.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.876 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.876 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.876 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.961149+0000", 2024-09-14T06:17:05.876 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.093166104 2024-09-14T06:17:05.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.876 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.876 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.876 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.961286+0000", 2024-09-14T06:17:05.876 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013708700000000001 2024-09-14T06:17:05.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.876 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.876 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.877 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.961729+0000", 2024-09-14T06:17:05.877 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00044280600000000002 2024-09-14T06:17:05.877 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.877 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.877 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.877 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.877 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.877 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32019 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.877 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868002+0000", 2024-09-14T06:17:05.877 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.98129245, 2024-09-14T06:17:05.877 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.094365927000000002, 2024-09-14T06:17:05.877 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.877 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.878 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.878 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.878 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.878 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.878 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32019 2024-09-14T06:17:05.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.878 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.878 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.878 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.878 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868002+0000", 2024-09-14T06:17:05.878 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.879 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.879 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868002+0000", 2024-09-14T06:17:05.879 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.879 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.879 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868004+0000", 2024-09-14T06:17:05.879 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.68e-06 2024-09-14T06:17:05.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.879 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868009+0000", 2024-09-14T06:17:05.880 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2830000000000001e-06 2024-09-14T06:17:05.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.880 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.880 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868031+0000", 2024-09-14T06:17:05.880 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1976000000000001e-05 2024-09-14T06:17:05.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.880 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.880 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868045+0000", 2024-09-14T06:17:05.881 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3599e-05 2024-09-14T06:17:05.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.881 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.881 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.881 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.961816+0000", 2024-09-14T06:17:05.881 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.093771418999999995 2024-09-14T06:17:05.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.881 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.881 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.881 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.961960+0000", 2024-09-14T06:17:05.881 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014337700000000001 2024-09-14T06:17:05.882 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.882 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.882 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.882 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.962368+0000", 2024-09-14T06:17:05.882 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040859300000000001 2024-09-14T06:17:05.882 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.882 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.882 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.882 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.882 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.882 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32020 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.882 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868068+0000", 2024-09-14T06:17:05.882 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.9812263530000001, 2024-09-14T06:17:05.882 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.094962112000000001, 2024-09-14T06:17:05.882 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.883 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.883 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.883 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.883 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.883 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.883 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32020 2024-09-14T06:17:05.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.883 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.883 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.883 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.883 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868068+0000", 2024-09-14T06:17:05.883 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.884 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.884 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.884 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868068+0000", 2024-09-14T06:17:05.884 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.884 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.884 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.884 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868070+0000", 2024-09-14T06:17:05.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.283e-06 2024-09-14T06:17:05.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868075+0000", 2024-09-14T06:17:05.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7470000000000002e-06 2024-09-14T06:17:05.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868096+0000", 2024-09-14T06:17:05.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0729000000000001e-05 2024-09-14T06:17:05.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.886 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.886 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868113+0000", 2024-09-14T06:17:05.886 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6909e-05 2024-09-14T06:17:05.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.886 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.886 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.962454+0000", 2024-09-14T06:17:05.886 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.094340757999999997 2024-09-14T06:17:05.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.886 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.886 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.962592+0000", 2024-09-14T06:17:05.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013838000000000001 2024-09-14T06:17:05.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.887 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.887 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.887 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963030+0000", 2024-09-14T06:17:05.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00043830600000000001 2024-09-14T06:17:05.887 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.887 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.887 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.887 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.887 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32021 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5767168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.887 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868136+0000", 2024-09-14T06:17:05.887 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.981158733, 2024-09-14T06:17:05.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095287032999999993, 2024-09-14T06:17:05.888 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.888 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.888 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.888 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.888 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.888 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.889 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32021 2024-09-14T06:17:05.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.889 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.889 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.889 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.889 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868136+0000", 2024-09-14T06:17:05.889 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.889 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.889 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.889 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868136+0000", 2024-09-14T06:17:05.889 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.890 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.890 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.890 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.890 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868137+0000", 2024-09-14T06:17:05.890 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2440000000000001e-06 2024-09-14T06:17:05.890 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.890 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.890 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.890 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868142+0000", 2024-09-14T06:17:05.890 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.079e-06 2024-09-14T06:17:05.890 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.890 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.890 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.891 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868164+0000", 2024-09-14T06:17:05.891 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1939000000000001e-05 2024-09-14T06:17:05.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.891 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.891 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.891 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868175+0000", 2024-09-14T06:17:05.891 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.046e-05 2024-09-14T06:17:05.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.891 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.891 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.891 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963120+0000", 2024-09-14T06:17:05.891 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.094945698999999995 2024-09-14T06:17:05.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.891 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.892 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.892 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963265+0000", 2024-09-14T06:17:05.892 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000144926 2024-09-14T06:17:05.892 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.892 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.892 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.892 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963423+0000", 2024-09-14T06:17:05.892 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015768600000000001 2024-09-14T06:17:05.892 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.892 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.892 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.892 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.892 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.892 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32022 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.981105552, 2024-09-14T06:17:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095598104000000003, 2024-09-14T06:17:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32022 2024-09-14T06:17:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.893 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868190+0000", 2024-09-14T06:17:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6000000000000003e-07 2024-09-14T06:17:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.894 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.895 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868243+0000", 2024-09-14T06:17:05.895 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3443999999999999e-05 2024-09-14T06:17:05.895 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.895 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.895 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.895 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868260+0000", 2024-09-14T06:17:05.895 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6799e-05 2024-09-14T06:17:05.895 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.895 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.895 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.895 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868271+0000", 2024-09-14T06:17:05.895 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0523000000000001e-05 2024-09-14T06:17:05.895 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.895 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.896 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.896 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963502+0000", 2024-09-14T06:17:05.896 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095231466000000001 2024-09-14T06:17:05.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.896 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.896 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.896 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963636+0000", 2024-09-14T06:17:05.896 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013388699999999999 2024-09-14T06:17:05.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.897 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.897 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.897 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963787+0000", 2024-09-14T06:17:05.897 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000151225 2024-09-14T06:17:05.897 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.897 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.897 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.897 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.897 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32023 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5816320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.897 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:05.897 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.9810104260000001, 2024-09-14T06:17:05.897 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095879753999999998, 2024-09-14T06:17:05.898 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.898 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.898 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.898 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.898 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.898 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.898 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32023 2024-09-14T06:17:05.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.898 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.898 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.898 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.898 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:05.898 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.899 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.899 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.899 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:05.899 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.899 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.899 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.899 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868285+0000", 2024-09-14T06:17:05.899 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6299999999999993e-07 2024-09-14T06:17:05.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.899 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.899 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.899 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868288+0000", 2024-09-14T06:17:05.900 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.18e-06 2024-09-14T06:17:05.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.900 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.900 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.900 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868298+0000", 2024-09-14T06:17:05.900 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0029e-05 2024-09-14T06:17:05.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.900 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.900 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.900 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868307+0000", 2024-09-14T06:17:05.900 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7229999999999993e-06 2024-09-14T06:17:05.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.900 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.900 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.901 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963871+0000", 2024-09-14T06:17:05.901 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095563876000000006 2024-09-14T06:17:05.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.901 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.901 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964015+0000", 2024-09-14T06:17:05.901 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143573 2024-09-14T06:17:05.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.901 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.901 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964164+0000", 2024-09-14T06:17:05.901 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014940999999999999 2024-09-14T06:17:05.902 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.902 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.902 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.902 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.902 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32024 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.902 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:05.902 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.980977282, 2024-09-14T06:17:05.902 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096230231999999999, 2024-09-14T06:17:05.902 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.902 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.902 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.902 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.902 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.903 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.903 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32024 2024-09-14T06:17:05.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.903 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.903 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:05.903 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.903 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:05.903 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.904 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.904 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.904 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868318+0000", 2024-09-14T06:17:05.904 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1699999999999997e-07 2024-09-14T06:17:05.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.904 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.904 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.904 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868321+0000", 2024-09-14T06:17:05.904 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6769999999999999e-06 2024-09-14T06:17:05.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.905 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.905 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.905 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868334+0000", 2024-09-14T06:17:05.905 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2928999999999999e-05 2024-09-14T06:17:05.905 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.905 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.905 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.905 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868341+0000", 2024-09-14T06:17:05.905 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9800000000000001e-06 2024-09-14T06:17:05.905 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.906 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.906 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.906 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964258+0000", 2024-09-14T06:17:05.906 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095917099000000006 2024-09-14T06:17:05.906 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.906 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.906 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.906 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964392+0000", 2024-09-14T06:17:05.906 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013396999999999999 2024-09-14T06:17:05.906 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.906 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.906 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.906 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964548+0000", 2024-09-14T06:17:05.906 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015585999999999999 2024-09-14T06:17:05.906 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.907 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.907 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.907 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.907 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.907 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32025 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.907 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:05.907 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.9809446259999999, 2024-09-14T06:17:05.907 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096576591000000003, 2024-09-14T06:17:05.907 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.907 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.907 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.908 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.908 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.908 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.908 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32025 2024-09-14T06:17:05.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.908 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.908 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.908 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.908 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:05.908 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.909 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:05.909 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.910 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868351+0000", 2024-09-14T06:17:05.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1600000000000001e-07 2024-09-14T06:17:05.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.910 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.910 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868353+0000", 2024-09-14T06:17:05.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.627e-06 2024-09-14T06:17:05.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.910 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.910 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868364+0000", 2024-09-14T06:17:05.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0893000000000001e-05 2024-09-14T06:17:05.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.910 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.911 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868381+0000", 2024-09-14T06:17:05.911 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6769000000000002e-05 2024-09-14T06:17:05.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.911 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.911 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.911 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964632+0000", 2024-09-14T06:17:05.911 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096250589999999997 2024-09-14T06:17:05.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.911 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.911 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.911 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964772+0000", 2024-09-14T06:17:05.911 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014045299999999999 2024-09-14T06:17:05.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.912 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.912 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.912 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964927+0000", 2024-09-14T06:17:05.912 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015454299999999999 2024-09-14T06:17:05.912 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.912 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.912 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.912 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.912 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.912 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32026 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.912 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:05.912 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.980903675, 2024-09-14T06:17:05.912 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096901622000000007, 2024-09-14T06:17:05.912 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.912 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.913 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.913 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.913 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.913 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.913 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32026 2024-09-14T06:17:05.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.913 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.913 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.913 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.913 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:05.913 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.913 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:05.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868392+0000", 2024-09-14T06:17:05.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0699999999999996e-07 2024-09-14T06:17:05.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868395+0000", 2024-09-14T06:17:05.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9859999999999999e-06 2024-09-14T06:17:05.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.915 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.915 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868405+0000", 2024-09-14T06:17:05.915 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9229999999999997e-06 2024-09-14T06:17:05.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.915 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.915 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868412+0000", 2024-09-14T06:17:05.915 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9670000000000004e-06 2024-09-14T06:17:05.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.915 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.915 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965010+0000", 2024-09-14T06:17:05.915 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096598562999999998 2024-09-14T06:17:05.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.915 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965143+0000", 2024-09-14T06:17:05.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013247700000000001 2024-09-14T06:17:05.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965293+0000", 2024-09-14T06:17:05.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000149899 2024-09-14T06:17:05.916 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.916 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.916 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.916 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32027 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.916 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:05.917 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.9808353379999999, 2024-09-14T06:17:05.917 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097211306999999997, 2024-09-14T06:17:05.917 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.917 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.917 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.917 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.917 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.917 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.917 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32027 2024-09-14T06:17:05.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.917 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.917 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.917 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.918 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:05.918 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.918 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.918 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.918 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:05.918 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.918 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.918 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.918 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868461+0000", 2024-09-14T06:17:05.918 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.296e-06 2024-09-14T06:17:05.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.919 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.919 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.919 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868465+0000", 2024-09-14T06:17:05.919 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9570000000000004e-06 2024-09-14T06:17:05.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.919 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.919 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.919 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868476+0000", 2024-09-14T06:17:05.919 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1365999999999999e-05 2024-09-14T06:17:05.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.919 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.920 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868483+0000", 2024-09-14T06:17:05.920 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2529999999999998e-06 2024-09-14T06:17:05.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.920 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.920 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965375+0000", 2024-09-14T06:17:05.920 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096891973000000006 2024-09-14T06:17:05.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.920 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.920 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965518+0000", 2024-09-14T06:17:05.920 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143017 2024-09-14T06:17:05.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.921 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.921 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965671+0000", 2024-09-14T06:17:05.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015244499999999999 2024-09-14T06:17:05.921 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.921 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.921 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.921 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32028 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.921 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:05.921 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.9808005099999999, 2024-09-14T06:17:05.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097841480999999994, 2024-09-14T06:17:05.922 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.922 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.922 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.922 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.922 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.922 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.922 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32028 2024-09-14T06:17:05.922 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.922 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.922 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.922 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.922 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:05.922 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.923 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.923 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.923 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:05.923 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.923 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.923 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.923 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868495+0000", 2024-09-14T06:17:05.923 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6400000000000001e-07 2024-09-14T06:17:05.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.923 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.923 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.924 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868498+0000", 2024-09-14T06:17:05.924 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7760000000000002e-06 2024-09-14T06:17:05.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.924 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.924 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.924 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868507+0000", 2024-09-14T06:17:05.924 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1530000000000003e-06 2024-09-14T06:17:05.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.924 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.924 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.924 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868516+0000", 2024-09-14T06:17:05.924 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8929999999999994e-06 2024-09-14T06:17:05.925 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965753+0000", 2024-09-14T06:17:05.925 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097236970000000006 2024-09-14T06:17:05.925 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965890+0000", 2024-09-14T06:17:05.925 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013677599999999999 2024-09-14T06:17:05.925 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.926 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966336+0000", 2024-09-14T06:17:05.926 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00044614900000000001 2024-09-14T06:17:05.926 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.926 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.926 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.926 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32029 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.926 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:05.926 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.980769435, 2024-09-14T06:17:05.926 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098577779000000004, 2024-09-14T06:17:05.926 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.926 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.926 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.926 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.927 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.927 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.927 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32029 2024-09-14T06:17:05.927 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.927 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.927 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.927 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.927 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:05.927 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.927 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.927 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.927 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.927 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:05.927 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.927 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.928 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.928 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.928 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868526+0000", 2024-09-14T06:17:05.928 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9300000000000006e-07 2024-09-14T06:17:05.928 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.928 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.928 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.928 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868529+0000", 2024-09-14T06:17:05.928 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5969999999999999e-06 2024-09-14T06:17:05.928 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.928 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.928 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.928 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868538+0000", 2024-09-14T06:17:05.928 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1700000000000003e-06 2024-09-14T06:17:05.928 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868560+0000", 2024-09-14T06:17:05.929 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1596000000000001e-05 2024-09-14T06:17:05.929 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966423+0000", 2024-09-14T06:17:05.929 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097863396000000005 2024-09-14T06:17:05.929 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966578+0000", 2024-09-14T06:17:05.929 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000154792 2024-09-14T06:17:05.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.930 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967103+0000", 2024-09-14T06:17:05.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00052523499999999998 2024-09-14T06:17:05.930 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.930 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.930 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.930 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.930 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32030 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.930 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:05.930 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.9807245170000001, 2024-09-14T06:17:05.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099324030999999993, 2024-09-14T06:17:05.931 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.931 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.931 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.931 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.931 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.931 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.931 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32030 2024-09-14T06:17:05.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.931 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.931 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.931 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.931 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:05.932 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.932 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.932 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.932 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:05.932 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.932 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.932 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.932 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868571+0000", 2024-09-14T06:17:05.932 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9699999999999995e-07 2024-09-14T06:17:05.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.933 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.933 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868574+0000", 2024-09-14T06:17:05.933 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9529999999999999e-06 2024-09-14T06:17:05.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.933 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.933 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868584+0000", 2024-09-14T06:17:05.933 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7860000000000008e-06 2024-09-14T06:17:05.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.933 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.933 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868591+0000", 2024-09-14T06:17:05.933 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0500000000000003e-06 2024-09-14T06:17:05.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967191+0000", 2024-09-14T06:17:05.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098600035000000003 2024-09-14T06:17:05.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967333+0000", 2024-09-14T06:17:05.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014190599999999999 2024-09-14T06:17:05.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.935 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967894+0000", 2024-09-14T06:17:05.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000561504 2024-09-14T06:17:05.935 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.935 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.935 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.935 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.935 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32031 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.935 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:05.935 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.9806949550000001, 2024-09-14T06:17:05.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.10010279, 2024-09-14T06:17:05.936 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.936 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.936 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.936 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.936 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.936 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.936 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32031 2024-09-14T06:17:05.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.936 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.936 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.936 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.936 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:05.936 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.937 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.937 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:05.937 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.937 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.937 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:05.937 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:17:05.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.937 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868603+0000", 2024-09-14T06:17:05.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.599e-06 2024-09-14T06:17:05.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868612+0000", 2024-09-14T06:17:05.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.986e-06 2024-09-14T06:17:05.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868619+0000", 2024-09-14T06:17:05.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8900000000000001e-06 2024-09-14T06:17:05.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967983+0000", 2024-09-14T06:17:05.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099363612000000004 2024-09-14T06:17:05.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968140+0000", 2024-09-14T06:17:05.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015788600000000001 2024-09-14T06:17:05.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968703+0000", 2024-09-14T06:17:05.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00056208300000000001 2024-09-14T06:17:05.940 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.940 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.940 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.940 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.940 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32032 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6258688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.940 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:05.940 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.98066648, 2024-09-14T06:17:05.940 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100488088, 2024-09-14T06:17:05.940 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.940 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.940 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:05.940 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.941 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.941 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.941 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32032 2024-09-14T06:17:05.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.941 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:05.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:05.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868629+0000", 2024-09-14T06:17:05.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:17:05.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868631+0000", 2024-09-14T06:17:05.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4099999999999998e-06 2024-09-14T06:17:05.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868640+0000", 2024-09-14T06:17:05.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0939999999999997e-06 2024-09-14T06:17:05.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.943 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.943 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868647+0000", 2024-09-14T06:17:05.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9460000000000001e-06 2024-09-14T06:17:05.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.943 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.943 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968797+0000", 2024-09-14T06:17:05.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100149867 2024-09-14T06:17:05.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.943 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.944 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968937+0000", 2024-09-14T06:17:05.944 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013981399999999999 2024-09-14T06:17:05.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.944 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.944 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969116+0000", 2024-09-14T06:17:05.944 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017927399999999999 2024-09-14T06:17:05.944 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.944 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.944 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.944 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32044 3.1e 3:7e6619d4:::rbd_data.115274ed0aa8.0000000000000068:head [set-alloc-hint object_size 1048576 write_size 1048576,write 835584~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:05.945 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:05.945 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 1.972977397, 2024-09-14T06:17:05.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096039367, 2024-09-14T06:17:05.945 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:05.945 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:05.945 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:05.945 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:05.945 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:05.945 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:05.945 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32044 2024-09-14T06:17:05.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.945 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:05.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:05.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:05.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:05.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:05.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:05.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:05.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876320+0000", 2024-09-14T06:17:05.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8270000000000002e-06 2024-09-14T06:17:05.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:05.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876397+0000", 2024-09-14T06:17:05.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6985999999999998e-05 2024-09-14T06:17:05.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:05.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876431+0000", 2024-09-14T06:17:05.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3719e-05 2024-09-14T06:17:05.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:05.948 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876452+0000", 2024-09-14T06:17:05.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0891999999999999e-05 2024-09-14T06:17:05.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.948 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.948 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:05.948 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969186+0000", 2024-09-14T06:17:05.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.092734681999999999 2024-09-14T06:17:05.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.948 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.948 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:05.948 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969550+0000", 2024-09-14T06:17:05.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036402199999999998 2024-09-14T06:17:05.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.949 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:17:05.949 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969987+0000", 2024-09-14T06:17:05.949 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000436759 2024-09-14T06:17:05.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.949 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:05.949 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971569+0000", 2024-09-14T06:17:05.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001581821 2024-09-14T06:17:05.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:05.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971947+0000", 2024-09-14T06:17:05.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378446 2024-09-14T06:17:05.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:05.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972146+0000", 2024-09-14T06:17:05.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019819 2024-09-14T06:17:05.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:05.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:05.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:05.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972357+0000", 2024-09-14T06:17:05.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021102299999999999 2024-09-14T06:17:05.951 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.951 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.951 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.951 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:05.951 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:05.951 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:05.951 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:17:05.953 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:17:05.954 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:05.954+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:06.056 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:06.054+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:06.112 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:06.112 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:06.121 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:06.121 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:06.121 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:06.121 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:06.129 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:06.129 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:06.135 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:17:06.146 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:06.146 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:06.157 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:06.154+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:06.162 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:06.163 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:06.181 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:06.181 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:06.200 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:06.200 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:06.218 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:06.218 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:06.237 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:06.237 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:06.246 INFO:teuthology.orchestra.run.smithi039.stderr:set 3.9 pg_upmap_items mapping to [3->4,6->7] 2024-09-14T06:17:06.258 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:06.254+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:06.309 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:06.309 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:06.323 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:17:06.333 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone18 from 0x9981f63 (overlap 0x28c109f) to 0x1c87208 2024-09-14T06:17:06.367 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:06.366+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:06.460 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:06.458+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:06.494 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:06.495 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:06.509 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:17:06.510 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-14T06:17:06.560 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:06.558+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:06.661 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:06.658+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:06.667 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:06.667 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:06.684 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:17:06.703 INFO:teuthology.orchestra.run.smithi039.stdout:3426 read 0x7632306 thru 0x76330d8 (0xdd3 bytes) 2024-09-14T06:17:06.712 INFO:teuthology.orchestra.run.smithi039.stdout:3429 read 0x11d0ab8 thru 0x11dad51 (0xa29a bytes) 2024-09-14T06:17:06.715 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:06.739 INFO:teuthology.orchestra.run.smithi039.stdout:3431 trunc from 0x9981f63 to 0x10c6da3 2024-09-14T06:17:06.775 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:06.774+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:06.832 INFO:teuthology.orchestra.run.smithi039.stdout:3432 write 0xd5c0800 thru 0xd5cb353 (0xab54 bytes) 2024-09-14T06:17:06.840 INFO:teuthology.orchestra.run.smithi039.stdout:3433 read 0xc04b960 thru 0xc056918 (0xafb9 bytes) 2024-09-14T06:17:06.841 INFO:teuthology.orchestra.run.smithi039.stdout:3434 write 0xbad2310 thru 0xbada02d (0x7d1e bytes) 2024-09-14T06:17:06.849 INFO:teuthology.orchestra.run.smithi039.stdout:3435 writesame 0x4242521 thru 0x424c1a7 data_size 0x3(0x9c87 bytes) 2024-09-14T06:17:06.857 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:06.858 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:06.863 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:06.862+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:06.872 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:17:06.873 INFO:teuthology.orchestra.run.smithi039.stdout:3436 read 0x75a8d87 thru 0x75b6549 (0xd7c3 bytes) 2024-09-14T06:17:06.873 INFO:teuthology.orchestra.run.smithi039.stdout:3437 read 0x38f57a8 thru 0x39038b9 (0xe112 bytes) 2024-09-14T06:17:06.873 INFO:teuthology.orchestra.run.smithi039.stdout:3438 punch from 0x86fa924 to 0x87032cb, (0x89a7 bytes) 2024-09-14T06:17:06.873 INFO:teuthology.orchestra.run.smithi039.stdout:3439 write 0x9e43aac thru 0x9e4e3a0 (0xa8f5 bytes) 2024-09-14T06:17:06.881 INFO:teuthology.orchestra.run.smithi039.stdout:3441 punch from 0x6ccd9ca to 0x6cd3584, (0x5bba bytes) 2024-09-14T06:17:06.882 INFO:teuthology.orchestra.run.smithi039.stdout:3442 punch from 0x816b7f0 to 0x81704ca, (0x4cda bytes) 2024-09-14T06:17:06.882 INFO:teuthology.orchestra.run.smithi039.stdout:3445 read 0xc62eb0e thru 0xc6373cc (0x88bf bytes) 2024-09-14T06:17:06.882 INFO:teuthology.orchestra.run.smithi039.stdout:3446 read 0x8e2b2c7 thru 0x8e3427c (0x8fb6 bytes) 2024-09-14T06:17:06.882 INFO:teuthology.orchestra.run.smithi039.stdout:3448 write 0x833eb6e thru 0x833fe1e (0x12b1 bytes) 2024-09-14T06:17:06.889 INFO:teuthology.orchestra.run.smithi039.stdout:3449 punch from 0x2dd048c to 0x2dd229a, (0x1e0e bytes) 2024-09-14T06:17:06.889 INFO:teuthology.orchestra.run.smithi039.stdout:3450 writesame 0x921851f thru 0x9222aae data_size 0x18(0xa590 bytes) 2024-09-14T06:17:06.899 INFO:teuthology.orchestra.run.smithi039.stdout:3452 write 0x4898772 thru 0x489a367 (0x1bf6 bytes) 2024-09-14T06:17:06.937 INFO:teuthology.orchestra.run.smithi039.stdout:3453 trunc from 0xd5cb354 to 0x667b2b 2024-09-14T06:17:06.964 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:06.962+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:07.041 INFO:teuthology.orchestra.run.smithi039.stdout:3456 punch from 0x39e67c to 0x3ad18e, (0xeb12 bytes) 2024-09-14T06:17:07.041 INFO:teuthology.orchestra.run.smithi039.stdout:3457 read 0x2f56ce thru 0x2fef7b (0x98ae bytes) 2024-09-14T06:17:07.045 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:07.045 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:07.062 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:17:07.065 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:07.062+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:07.080 INFO:teuthology.orchestra.run.smithi039.stdout:3458 writesame 0x976d326 thru 0x9773e13 data_size 0x1(0x6aee bytes) 2024-09-14T06:17:07.090 INFO:teuthology.orchestra.run.smithi039.stdout:3459 trunc from 0x9773e14 to 0xe43d7d2 2024-09-14T06:17:07.102 INFO:teuthology.orchestra.run.smithi039.stdout:3460 trunc from 0xe43d7d2 to 0xb983570 2024-09-14T06:17:07.110 INFO:teuthology.orchestra.run.smithi039.stdout:3461 read 0x220fab5 thru 0x2217797 (0x7ce3 bytes) 2024-09-14T06:17:07.110 INFO:teuthology.orchestra.run.smithi039.stdout:3462 read 0x2ddaacb thru 0x2de71bc (0xc6f2 bytes) 2024-09-14T06:17:07.110 INFO:teuthology.orchestra.run.smithi039.stdout:3464 write 0x8ca7413 thru 0x8caa366 (0x2f54 bytes) 2024-09-14T06:17:07.126 INFO:teuthology.orchestra.run.smithi039.stdout:3465 write 0xe186776 thru 0xe192885 (0xc110 bytes) 2024-09-14T06:17:07.132 INFO:teuthology.orchestra.run.smithi039.stdout:3466 read 0xa99de2 thru 0xaa05ad (0x67cc bytes) 2024-09-14T06:17:07.132 INFO:teuthology.orchestra.run.smithi039.stdout:3467 read 0x35b3ae2 thru 0x35bdcf6 (0xa215 bytes) 2024-09-14T06:17:07.132 INFO:teuthology.orchestra.run.smithi039.stdout:3468 write 0x3bd79d7 thru 0x3bdb7a0 (0x3dca bytes) 2024-09-14T06:17:07.136 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:07.137 INFO:teuthology.orchestra.run.smithi039.stdout:3472 read 0x8354e51 thru 0x8359898 (0x4a48 bytes) 2024-09-14T06:17:07.137 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:07.137 INFO:teuthology.orchestra.run.smithi039.stdout:3474 punch from 0xdb25898 to 0xdb2cd75, (0x74dd bytes) 2024-09-14T06:17:07.151 INFO:teuthology.orchestra.run.smithi039.stdout:3475 trunc from 0xe192886 to 0x57f45aa 2024-09-14T06:17:07.165 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:07.162+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:07.168 INFO:teuthology.orchestra.run.smithi039.stdout:3476 read 0x45a605d thru 0x45ac507 (0x64ab bytes) 2024-09-14T06:17:07.187 INFO:teuthology.orchestra.run.smithi039.stdout:3477 write 0xd2e7d33 thru 0xd2eac37 (0x2f05 bytes) 2024-09-14T06:17:07.191 INFO:teuthology.orchestra.run.smithi039.stdout:3478 writesame 0xf94368 thru 0xf98d37 data_size 0x8(0x49d0 bytes) 2024-09-14T06:17:07.198 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:07.198 INFO:teuthology.orchestra.run.smithi039.stdout:3482 read 0x21a2bde thru 0x21ab07c (0x849f bytes) 2024-09-14T06:17:07.198 INFO:teuthology.orchestra.run.smithi039.stdout:3483 write 0x44f761 thru 0x45d401 (0xdca1 bytes) 2024-09-14T06:17:07.210 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:07.222 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:07.222 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:07.228 INFO:teuthology.orchestra.run.smithi039.stdout:3485 trunc from 0xd2eac38 to 0xd7ea55 2024-09-14T06:17:07.242 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:17:07.258 INFO:teuthology.orchestra.run.smithi039.stdout:3487 write 0x5d0f6a9 thru 0x5d151b9 (0x5b11 bytes) 2024-09-14T06:17:07.273 INFO:teuthology.orchestra.run.smithi039.stdout:3488 trunc from 0x5d151ba to 0x672e81 2024-09-14T06:17:07.273 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:07.270+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:07.288 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:07.288 INFO:teuthology.orchestra.run.smithi039.stdout:3490 punch from 0x5f8740 to 0x6034c6, (0xad86 bytes) 2024-09-14T06:17:07.300 INFO:teuthology.orchestra.run.smithi039.stdout:3491 write 0x41eb4ad thru 0x41fa9c5 (0xf519 bytes) 2024-09-14T06:17:07.305 INFO:teuthology.orchestra.run.smithi039.stdout:3492 read 0x2bd3861 thru 0x2be3031 (0xf7d1 bytes) 2024-09-14T06:17:07.305 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:07.310 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:17:07.318 INFO:teuthology.orchestra.run.smithi039.stdout:3494 write 0x84d8e64 thru 0x84e7419 (0xe5b6 bytes) 2024-09-14T06:17:07.333 INFO:teuthology.orchestra.run.smithi039.stdout:3495 writesame 0x9e49daf thru 0x9e4e5a5 data_size 0x45(0x47f7 bytes) 2024-09-14T06:17:07.333 INFO:teuthology.orchestra.run.smithi039.stdout:3497 punch from 0x996a674 to 0x99743fe, (0x9d8a bytes) 2024-09-14T06:17:07.333 INFO:teuthology.orchestra.run.smithi039.stdout:3498 write 0x18c99bb thru 0x18cfbfc (0x6242 bytes) 2024-09-14T06:17:07.339 INFO:teuthology.orchestra.run.smithi039.stdout:3499 trunc from 0x9e4e5a6 to 0xad386e8 2024-09-14T06:17:07.346 INFO:teuthology.orchestra.run.smithi039.stdout:3503 read 0x1b13bad thru 0x1b20a52 (0xcea6 bytes) 2024-09-14T06:17:07.346 INFO:teuthology.orchestra.run.smithi039.stdout:3504 trunc from 0xad386e8 to 0xaa202ef 2024-09-14T06:17:07.354 INFO:teuthology.orchestra.run.smithi039.stdout:3505 trunc from 0xaa202ef to 0x977a0fa 2024-09-14T06:17:07.367 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:07.366+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:3506 write 0xbfde507 thru 0xbfe38bb (0x53b5 bytes) 2024-09-14T06:17:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:3507 punch from 0x2f93f44 to 0x2f98dd9, (0x4e95 bytes) 2024-09-14T06:17:07.377 INFO:teuthology.orchestra.run.smithi039.stdout:3508 read 0x4c16c6a thru 0x4c23302 (0xc699 bytes) 2024-09-14T06:17:07.388 INFO:teuthology.orchestra.run.smithi039.stdout:3509 write 0xedef114 thru 0xedf573b (0x6628 bytes) 2024-09-14T06:17:07.392 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:07.392 INFO:teuthology.orchestra.run.smithi039.stdout:3511 writesame 0xc9c5dbe thru 0xc9cabce data_size 0x7(0x4e11 bytes) 2024-09-14T06:17:07.419 INFO:teuthology.orchestra.run.smithi039.stdout:3512 trunc from 0xedf573c to 0x28092bd 2024-09-14T06:17:07.425 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:07.425 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:07.440 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:07.440 INFO:teuthology.orchestra.run.smithi039.stdout:3514 read 0x1c3322c thru 0x1c3347d (0x252 bytes) 2024-09-14T06:17:07.441 INFO:teuthology.orchestra.run.smithi039.stdout:3515 punch from 0x166cfef to 0x167caee, (0xfaff bytes) 2024-09-14T06:17:07.441 INFO:teuthology.orchestra.run.smithi039.stdout:3516 punch from 0xb7977 to 0xbddcb, (0x6454 bytes) 2024-09-14T06:17:07.441 INFO:teuthology.orchestra.run.smithi039.stdout:3518 read 0x65c192 thru 0x669f81 (0xddf0 bytes) 2024-09-14T06:17:07.441 INFO:teuthology.orchestra.run.smithi039.stdout:3519 write 0x19adfca thru 0x19bbf37 (0xdf6e bytes) 2024-09-14T06:17:07.444 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:17:07.453 INFO:teuthology.orchestra.run.smithi039.stdout:3520 write 0x31e4206 thru 0x31eb773 (0x756e bytes) 2024-09-14T06:17:07.456 INFO:teuthology.orchestra.run.smithi039.stdout:3521 read 0xd4b182 thru 0xd53ebf (0x8d3e bytes) 2024-09-14T06:17:07.456 INFO:teuthology.orchestra.run.smithi039.stdout:3522 writesame 0x60adae thru 0x60d89b data_size 0x23(0x2aee bytes) 2024-09-14T06:17:07.467 INFO:teuthology.orchestra.run.smithi039.stdout:3523 trunc from 0x31eb774 to 0x782b442 2024-09-14T06:17:07.468 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:07.466+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:07.473 INFO:teuthology.orchestra.run.smithi039.stdout:3524 read 0x2ea8c77 thru 0x2eacc4d (0x3fd7 bytes) 2024-09-14T06:17:07.488 INFO:teuthology.orchestra.run.smithi039.stdout:3525 writesame 0xdadd19f thru 0xdadfd8d data_size 0x3(0x2bef bytes) 2024-09-14T06:17:07.488 INFO:teuthology.orchestra.run.smithi039.stdout:3526 write 0x4e2fac1 thru 0x4e3fa2b (0xff6b bytes) 2024-09-14T06:17:07.493 INFO:teuthology.orchestra.run.smithi039.stdout:3527 punch from 0x3881305 to 0x3890357, (0xf052 bytes) 2024-09-14T06:17:07.494 INFO:teuthology.orchestra.run.smithi039.stdout:3528 read 0x2b9e496 thru 0x2baa452 (0xbfbd bytes) 2024-09-14T06:17:07.496 INFO:teuthology.orchestra.run.smithi039.stdout:3529 trunc from 0xdadfd8e to 0xc03cef7 2024-09-14T06:17:07.502 INFO:teuthology.orchestra.run.smithi039.stdout:3530 trunc from 0xc03cef7 to 0xc67d0f4 2024-09-14T06:17:07.508 INFO:teuthology.orchestra.run.smithi039.stdout:3531 write 0x10eb888 thru 0x10f2eed (0x7666 bytes) 2024-09-14T06:17:07.511 INFO:teuthology.orchestra.run.smithi039.stdout:3532 punch from 0xc3277b5 to 0xc334071, (0xc8bc bytes) 2024-09-14T06:17:07.512 INFO:teuthology.orchestra.run.smithi039.stdout:3533 writesame 0x30bbed2 thru 0x30c0095 data_size 0x6(0x41c4 bytes) 2024-09-14T06:17:07.517 INFO:teuthology.orchestra.run.smithi039.stdout:3534 read 0x9a36f84 thru 0x9a3c122 (0x519f bytes) 2024-09-14T06:17:07.517 INFO:teuthology.orchestra.run.smithi039.stdout:3535 write 0xc62e1dd thru 0xc63bd7e (0xdba2 bytes) 2024-09-14T06:17:07.522 INFO:teuthology.orchestra.run.smithi039.stdout:3537 write 0x944e75c thru 0x9450884 (0x2129 bytes) 2024-09-14T06:17:07.527 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:07.527 INFO:teuthology.orchestra.run.smithi039.stdout:3540 punch from 0x37b9537 to 0x37c02b8, (0x6d81 bytes) 2024-09-14T06:17:07.527 INFO:teuthology.orchestra.run.smithi039.stdout:3541 read 0x2221df6 thru 0x2222f59 (0x1164 bytes) 2024-09-14T06:17:07.533 INFO:teuthology.orchestra.run.smithi039.stdout:3542 writesame 0xd40b1ae thru 0xd41869c data_size 0x13(0xd4ef bytes) 2024-09-14T06:17:07.534 INFO:teuthology.orchestra.run.smithi039.stdout:3543 punch from 0xa3a7a34 to 0xa3a9f06, (0x24d2 bytes) 2024-09-14T06:17:07.539 INFO:teuthology.orchestra.run.smithi039.stdout:3544 write 0xda7799f thru 0xda863aa (0xea0c bytes) 2024-09-14T06:17:07.545 INFO:teuthology.orchestra.run.smithi039.stdout:3546 punch from 0xadfc4e8 to 0xae046d7, (0x81ef bytes) 2024-09-14T06:17:07.546 INFO:teuthology.orchestra.run.smithi039.stdout:3548 punch from 0x7f0a000 to 0x7f10150, (0x6150 bytes) 2024-09-14T06:17:07.546 INFO:teuthology.orchestra.run.smithi039.stdout:3549 write 0x79b6fe8 thru 0x79c3d1d (0xcd36 bytes) 2024-09-14T06:17:07.551 INFO:teuthology.orchestra.run.smithi039.stdout:3550 writesame 0xb388c49 thru 0xb394eb2 data_size 0x7e(0xc26a bytes) 2024-09-14T06:17:07.557 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:07.557 INFO:teuthology.orchestra.run.smithi039.stdout:3552 read 0xcc730e thru 0xcd0235 (0x8f28 bytes) 2024-09-14T06:17:07.557 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:07.558 INFO:teuthology.orchestra.run.smithi039.stdout:3555 trunc from 0xda863ab to 0xed1ef1f 2024-09-14T06:17:07.566 INFO:teuthology.orchestra.run.smithi039.stdout:3556 write 0x6f50a99 thru 0x6f57eda (0x7442 bytes) 2024-09-14T06:17:07.568 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:07.566+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:07.571 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:07.585 INFO:teuthology.orchestra.run.smithi039.stdout:3558 trunc from 0xed1ef1f to 0x57bd6a8 2024-09-14T06:17:07.596 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:07.596 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:07.605 INFO:teuthology.orchestra.run.smithi039.stdout:3559 trunc from 0x57bd6a8 to 0x2a3ce55 2024-09-14T06:17:07.617 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:17:07.620 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:07.620 INFO:teuthology.orchestra.run.smithi039.stdout:3561 read 0x1b0dec4 thru 0x1b0f33e (0x147b bytes) 2024-09-14T06:17:07.626 INFO:teuthology.orchestra.run.smithi039.stdout:3562 write 0x2b330ea thru 0x2b35ebf (0x2dd6 bytes) 2024-09-14T06:17:07.630 INFO:teuthology.orchestra.run.smithi039.stdout:3563 punch from 0x1a63ca3 to 0x1a711e8, (0xd545 bytes) 2024-09-14T06:17:07.652 INFO:teuthology.orchestra.run.smithi039.stdout:3564 write 0xcc7ca68 thru 0xcc8251f (0x5ab8 bytes) 2024-09-14T06:17:07.658 INFO:teuthology.orchestra.run.smithi039.stdout:3565 write 0x68896a thru 0x693c39 (0xb2d0 bytes) 2024-09-14T06:17:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:3566 read 0xc7badc5 thru 0xc7c6658 (0xb894 bytes) 2024-09-14T06:17:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:3568 flatten 2024-09-14T06:17:07.669 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:07.666+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:07.761 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:07.761 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:07.770 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:07.766+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:07.780 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:17:07.871 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:07.870+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:07.888 INFO:teuthology.orchestra.run.smithi039.stdout:3569 read 0x9babac6 thru 0x9bb4307 (0x8842 bytes) 2024-09-14T06:17:07.888 INFO:teuthology.orchestra.run.smithi039.stdout:3571 read 0xbc2506a thru 0xbc2c44c (0x73e3 bytes) 2024-09-14T06:17:07.888 INFO:teuthology.orchestra.run.smithi039.stdout:3572 writesame 0x56efc73 thru 0x56f49a9 data_size 0xb(0x4d37 bytes) 2024-09-14T06:17:07.894 INFO:teuthology.orchestra.run.smithi039.stdout:3573 punch from 0x1474014 to 0x1478143, (0x412f bytes) 2024-09-14T06:17:07.894 INFO:teuthology.orchestra.run.smithi039.stdout:3574 punch from 0x52c19d5 to 0x52cc856, (0xae81 bytes) 2024-09-14T06:17:07.894 INFO:teuthology.orchestra.run.smithi039.stdout:3578 read 0x4581479 thru 0x458f801 (0xe389 bytes) 2024-09-14T06:17:07.894 INFO:teuthology.orchestra.run.smithi039.stdout:3579 punch from 0x35ece64 to 0x35f7408, (0xa5a4 bytes) 2024-09-14T06:17:07.894 INFO:teuthology.orchestra.run.smithi039.stdout:3580 punch from 0x3e7ce95 to 0x3e847ab, (0x7916 bytes) 2024-09-14T06:17:07.894 INFO:teuthology.orchestra.run.smithi039.stdout:3581 read 0xa47688e thru 0xa486805 (0xff78 bytes) 2024-09-14T06:17:07.894 INFO:teuthology.orchestra.run.smithi039.stdout:3582 write 0xac098eb thru 0xac0d540 (0x3c56 bytes) 2024-09-14T06:17:07.899 INFO:teuthology.orchestra.run.smithi039.stdout:3583 writesame 0x53d6c34 thru 0x53de7de data_size 0x3d(0x7bab bytes) 2024-09-14T06:17:07.913 INFO:teuthology.orchestra.run.smithi039.stdout:3584 write 0xd398b39 thru 0xd39e146 (0x560e bytes) 2024-09-14T06:17:07.924 INFO:teuthology.orchestra.run.smithi039.stdout:3585 trunc from 0xd39e147 to 0x8b972e7 2024-09-14T06:17:07.942 INFO:teuthology.orchestra.run.smithi039.stdout:3589 punch from 0x6a30e58 to 0x6a31cb8, (0xe60 bytes) 2024-09-14T06:17:07.942 INFO:teuthology.orchestra.run.smithi039.stdout:3590 writesame 0x5d3faa8 thru 0x5d41d4f data_size 0x4(0x22a8 bytes) 2024-09-14T06:17:07.950 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:07.960 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:07.960 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:07.963 INFO:teuthology.orchestra.run.smithi039.stdout:3593 writesame 0x9976f87 thru 0x9986a66 data_size 0x90(0xfae0 bytes) 2024-09-14T06:17:07.963 INFO:teuthology.orchestra.run.smithi039.stdout:3594 write 0x3954080 thru 0x3963832 (0xf7b3 bytes) 2024-09-14T06:17:07.971 INFO:teuthology.orchestra.run.smithi039.stdout:3595 read 0x931b1ce thru 0x9325008 (0x9e3b bytes) 2024-09-14T06:17:07.971 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:07.970+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:07.973 INFO:teuthology.orchestra.run.smithi039.stdout:3596 trunc from 0x9986a67 to 0x8cbed46 2024-09-14T06:17:07.978 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:17:07.995 INFO:teuthology.orchestra.run.smithi039.stdout:3597 trunc from 0x8cbed46 to 0x42bb28 2024-09-14T06:17:08.027 INFO:teuthology.orchestra.run.smithi039.stdout:3600 write 0x15b22f4 thru 0x15bbd98 (0x9aa5 bytes) 2024-09-14T06:17:08.033 INFO:teuthology.orchestra.run.smithi039.stdout:3601 trunc from 0x15bbd99 to 0x89566a 2024-09-14T06:17:08.046 INFO:teuthology.orchestra.run.smithi039.stdout:3603 writesame 0xe1852 thru 0xf067e data_size 0x17(0xee2d bytes) 2024-09-14T06:17:08.073 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:08.070+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:08.080 INFO:teuthology.orchestra.run.smithi039.stdout:3607 write 0x90877f9 thru 0x9092c7e (0xb486 bytes) 2024-09-14T06:17:08.084 INFO:teuthology.orchestra.run.smithi039.stdout:3609 read 0x2a6b70c thru 0x2a7636c (0xac61 bytes) 2024-09-14T06:17:08.101 INFO:teuthology.orchestra.run.smithi039.stdout:3611 write 0xe553d0b thru 0xe5627fe (0xeaf4 bytes) 2024-09-14T06:17:08.112 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:08.131 INFO:teuthology.orchestra.run.smithi039.stdout:3614 trunc from 0xe5627ff to 0x144b5d6 2024-09-14T06:17:08.140 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:08.140 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:08.149 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:08.149 INFO:teuthology.orchestra.run.smithi039.stdout:3616 punch from 0x15d1eb to 0x1646f4, (0x7509 bytes) 2024-09-14T06:17:08.149 INFO:teuthology.orchestra.run.smithi039.stdout:3617 read 0xe07cee thru 0xe13a8b (0xbd9e bytes) 2024-09-14T06:17:08.149 INFO:teuthology.orchestra.run.smithi039.stdout:3618 read 0xb1a050 thru 0xb1e327 (0x42d8 bytes) 2024-09-14T06:17:08.158 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:17:08.178 INFO:teuthology.orchestra.run.smithi039.stdout:3619 writesame 0xda83f69 thru 0xda85b27 data_size 0x1(0x1bbf bytes) 2024-09-14T06:17:08.179 INFO:teuthology.orchestra.run.smithi039.stdout:3620 writesame 0xc0198a8 thru 0xc01d21b data_size 0x2(0x3974 bytes) 2024-09-14T06:17:08.179 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:08.174+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:08.185 INFO:teuthology.orchestra.run.smithi039.stdout:3623 write 0x8f69ace thru 0x8f759d1 (0xbf04 bytes) 2024-09-14T06:17:08.191 INFO:teuthology.orchestra.run.smithi039.stdout:3624 read 0xc08ecf8 thru 0xc09db8c (0xee95 bytes) 2024-09-14T06:17:08.191 INFO:teuthology.orchestra.run.smithi039.stdout:3625 writesame 0xbb1a2df thru 0xbb1f32b data_size 0x1(0x504d bytes) 2024-09-14T06:17:08.208 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:08.225 INFO:teuthology.orchestra.run.smithi039.stdout:3629 trunc from 0xda85b28 to 0x3889630 2024-09-14T06:17:08.258 INFO:teuthology.orchestra.run.smithi039.stdout:3630 write 0x8aafd3d thru 0x8ab194d (0x1c11 bytes) 2024-09-14T06:17:08.259 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:17:08.262 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-09-14T06:17:08.278 INFO:teuthology.orchestra.run.smithi039.stdout:3637 write 0x9cf0265 thru 0x9cfdd1b (0xdab7 bytes) 2024-09-14T06:17:08.278 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:08.274+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:08.288 INFO:teuthology.orchestra.run.smithi039.stdout:3638 write 0x9a4ada5 thru 0x9a52735 (0x7991 bytes) 2024-09-14T06:17:08.294 INFO:teuthology.orchestra.run.smithi039.stdout:3639 read 0x3481c3c thru 0x34886ad (0x6a72 bytes) 2024-09-14T06:17:08.294 INFO:teuthology.orchestra.run.smithi039.stdout:3640 read 0x448762b thru 0x448888b (0x1261 bytes) 2024-09-14T06:17:08.294 INFO:teuthology.orchestra.run.smithi039.stdout:3641 write 0x4252d92 thru 0x42579cd (0x4c3c bytes) 2024-09-14T06:17:08.298 INFO:teuthology.orchestra.run.smithi039.stdout:3642 read 0x6a91302 thru 0x6a9f8bc (0xe5bb bytes) 2024-09-14T06:17:08.298 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:08.298 INFO:teuthology.orchestra.run.smithi039.stdout:3644 punch from 0x3d38a2c to 0x3d3e30a, (0x58de bytes) 2024-09-14T06:17:08.298 INFO:teuthology.orchestra.run.smithi039.stdout:3645 writesame 0x2a1c7fd thru 0x2a2a914 data_size 0xe(0xe118 bytes) 2024-09-14T06:17:08.304 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:08.304 INFO:teuthology.orchestra.run.smithi039.stdout:3648 punch from 0x63632cb to 0x6366943, (0x3678 bytes) 2024-09-14T06:17:08.304 INFO:teuthology.orchestra.run.smithi039.stdout:3649 read 0x6a01eeb thru 0x6a048f3 (0x2a09 bytes) 2024-09-14T06:17:08.304 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:08.304 INFO:teuthology.orchestra.run.smithi039.stdout:3651 punch from 0x886a3fb to 0x8873724, (0x9329 bytes) 2024-09-14T06:17:08.304 INFO:teuthology.orchestra.run.smithi039.stdout:3655 read 0x82c895e thru 0x82cb14f (0x27f2 bytes) 2024-09-14T06:17:08.305 INFO:teuthology.orchestra.run.smithi039.stdout:3656 write 0x5569800 thru 0x556c44a (0x2c4b bytes) 2024-09-14T06:17:08.309 INFO:teuthology.orchestra.run.smithi039.stdout:3657 read 0x7b782ff thru 0x7b802fa (0x7ffc bytes) 2024-09-14T06:17:08.320 INFO:teuthology.orchestra.run.smithi039.stdout:3659 writesame 0xdb1bf39 thru 0xdb2a7bf data_size 0x13(0xe887 bytes) 2024-09-14T06:17:08.321 INFO:teuthology.orchestra.run.smithi039.stdout:3660 write 0x13ed5e6 thru 0x13fa5ad (0xcfc8 bytes) 2024-09-14T06:17:08.324 INFO:teuthology.orchestra.run.smithi039.stdout:3662 punch from 0x3bf0b4e to 0x3bf9656, (0x8b08 bytes) 2024-09-14T06:17:08.325 INFO:teuthology.orchestra.run.smithi039.stdout:3665 punch from 0xcf847f6 to 0xcf8593b, (0x1145 bytes) 2024-09-14T06:17:08.325 INFO:teuthology.orchestra.run.smithi039.stdout:3666 read 0x62d064a thru 0x62d7fe0 (0x7997 bytes) 2024-09-14T06:17:08.325 INFO:teuthology.orchestra.run.smithi039.stdout:3667 writesame 0x2e343bc thru 0x2e42881 data_size 0x81(0xe4c6 bytes) 2024-09-14T06:17:08.330 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:08.330 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:08.330 INFO:teuthology.orchestra.run.smithi039.stdout:3668 punch from 0xc72097a to 0xc7222af, (0x1935 bytes) 2024-09-14T06:17:08.330 INFO:teuthology.orchestra.run.smithi039.stdout:3669 read 0x5513cf4 thru 0x551bb2f (0x7e3c bytes) 2024-09-14T06:17:08.331 INFO:teuthology.orchestra.run.smithi039.stdout:3670 writesame 0x8103a47 thru 0x8113530 data_size 0x1(0xfaea bytes) 2024-09-14T06:17:08.346 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:17:08.375 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:08.374+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:08.412 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:08.413 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:08.430 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:08.431 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:08.435 INFO:teuthology.orchestra.run.smithi039.stdout:3671 trunc from 0xdb2a7c0 to 0xc2d706 2024-09-14T06:17:08.453 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:08.454 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:08.471 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:08.471 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:08.474 INFO:teuthology.orchestra.run.smithi039.stdout:3672 writesame 0xb6df057 thru 0xb6e5c00 data_size 0x1(0x6baa bytes) 2024-09-14T06:17:08.475 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:08.474+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:08.485 INFO:teuthology.orchestra.run.smithi039.stdout:3673 write 0xe8d65e4 thru 0xe8da059 (0x3a76 bytes) 2024-09-14T06:17:08.491 INFO:teuthology.orchestra.run.smithi039.stdout:3674 punch from 0xa4702f4 to 0xa47c1c9, (0xbed5 bytes) 2024-09-14T06:17:08.492 INFO:teuthology.orchestra.run.smithi039.stdout:3675 read 0x6b34ae7 thru 0x6b42521 (0xda3b bytes) 2024-09-14T06:17:08.492 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:08.492 INFO:teuthology.orchestra.run.smithi039.stdout:3678 read 0x2d81a4f thru 0x2d90253 (0xe805 bytes) 2024-09-14T06:17:08.492 INFO:teuthology.orchestra.run.smithi039.stdout:3679 read 0x2f1b957 thru 0x2f1c199 (0x843 bytes) 2024-09-14T06:17:08.492 INFO:teuthology.orchestra.run.smithi039.stdout:3680 read 0xbc973bc thru 0xbc99020 (0x1c65 bytes) 2024-09-14T06:17:08.492 INFO:teuthology.orchestra.run.smithi039.stdout:3681 write 0x59b3d2c thru 0x59b86d3 (0x49a8 bytes) 2024-09-14T06:17:08.494 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:08.495 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:08.496 INFO:teuthology.orchestra.run.smithi039.stdout:3684 read 0xa596403 thru 0xa59f918 (0x9516 bytes) 2024-09-14T06:17:08.496 INFO:teuthology.orchestra.run.smithi039.stdout:3686 read 0x73313b5 thru 0x733b0a9 (0x9cf5 bytes) 2024-09-14T06:17:08.496 INFO:teuthology.orchestra.run.smithi039.stdout:3688 write 0x5d23067 thru 0x5d3081e (0xd7b8 bytes) 2024-09-14T06:17:08.501 INFO:teuthology.orchestra.run.smithi039.stdout:3689 read 0xdd72fbd thru 0xdd7c946 (0x998a bytes) 2024-09-14T06:17:08.501 INFO:teuthology.orchestra.run.smithi039.stdout:3690 writesame 0x6d78679 thru 0x6d7e541 data_size 0x17(0x5ec9 bytes) 2024-09-14T06:17:08.506 INFO:teuthology.orchestra.run.smithi039.stdout:3691 punch from 0x8f83a89 to 0x8f9324f, (0xf7c6 bytes) 2024-09-14T06:17:08.506 INFO:teuthology.orchestra.run.smithi039.stdout:3692 read 0x98639af thru 0x9865a17 (0x2069 bytes) 2024-09-14T06:17:08.506 INFO:teuthology.orchestra.run.smithi039.stdout:3693 write 0x12cf008 thru 0x12d441c (0x5415 bytes) 2024-09-14T06:17:08.510 INFO:teuthology.orchestra.run.smithi039.stdout:3694 read 0x62b6d33 thru 0x62bd1c1 (0x648f bytes) 2024-09-14T06:17:08.511 INFO:teuthology.orchestra.run.smithi039.stdout:3701 write 0x3c7bde6 thru 0x3c7c0cb (0x2e6 bytes) 2024-09-14T06:17:08.515 INFO:teuthology.orchestra.run.smithi039.stdout:3702 punch from 0x956da8c to 0x957482a, (0x6d9e bytes) 2024-09-14T06:17:08.515 INFO:teuthology.orchestra.run.smithi039.stdout:3705 read 0x5d3253 thru 0x5d911f (0x5ecd bytes) 2024-09-14T06:17:08.515 INFO:teuthology.orchestra.run.smithi039.stdout:3706 punch from 0x8608b0e to 0x8615564, (0xca56 bytes) 2024-09-14T06:17:08.515 INFO:teuthology.orchestra.run.smithi039.stdout:3707 punch from 0x201109f to 0x2020dcb, (0xfd2c bytes) 2024-09-14T06:17:08.515 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:08.516 INFO:teuthology.orchestra.run.smithi039.stdout:3709 read 0x3f205af thru 0x3f2e867 (0xe2b9 bytes) 2024-09-14T06:17:08.516 INFO:teuthology.orchestra.run.smithi039.stdout:3710 writesame 0xd76c4ed thru 0xd774c33 data_size 0x1(0x8747 bytes) 2024-09-14T06:17:08.517 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:08.518 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:08.532 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:08.532 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:08.532 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:08.532 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:08.532 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:08.539 INFO:teuthology.orchestra.run.smithi039.stdout:3712 write 0x188bff6 thru 0x18967f4 (0xa7ff bytes) 2024-09-14T06:17:08.546 INFO:teuthology.orchestra.run.smithi039.stdout:3713 punch from 0x9f7eb5f to 0x9f8b870, (0xcd11 bytes) 2024-09-14T06:17:08.546 INFO:teuthology.orchestra.run.smithi039.stdout:3714 read 0x8ac1046 thru 0x8aca929 (0x98e4 bytes) 2024-09-14T06:17:08.546 INFO:teuthology.orchestra.run.smithi039.stdout:3715 punch from 0x684cdbd to 0x6859470, (0xc6b3 bytes) 2024-09-14T06:17:08.546 INFO:teuthology.orchestra.run.smithi039.stdout:3717 read 0xdba7823 thru 0xdbab8f9 (0x40d7 bytes) 2024-09-14T06:17:08.546 INFO:teuthology.orchestra.run.smithi039.stdout:3718 read 0xdb939ca thru 0xdba3575 (0xfbac bytes) 2024-09-14T06:17:08.547 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:08.547 INFO:teuthology.orchestra.run.smithi039.stdout:3720 read 0x81c09c2 thru 0x81cbb11 (0xb150 bytes) 2024-09-14T06:17:08.547 INFO:teuthology.orchestra.run.smithi039.stdout:3721 punch from 0x69b1356 to 0x69bfa1a, (0xe6c4 bytes) 2024-09-14T06:17:08.547 INFO:teuthology.orchestra.run.smithi039.stdout:3722 write 0x97a70ea thru 0x97b0482 (0x9399 bytes) 2024-09-14T06:17:08.548 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:08.548 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:08.549 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:17:08.550 INFO:teuthology.orchestra.run.smithi039.stdout:3723 write 0x439e4bb thru 0x43a888b (0xa3d1 bytes) 2024-09-14T06:17:08.571 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:08.571 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:08.576 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:08.574+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:08.578 INFO:teuthology.orchestra.run.smithi039.stdout:3726 trunc from 0xe8da05a to 0x93f0b 2024-09-14T06:17:08.619 INFO:teuthology.orchestra.run.smithi039.stdout:3727 write 0x8525549 thru 0x8527af6 (0x25ae bytes) 2024-09-14T06:17:08.624 INFO:teuthology.orchestra.run.smithi039.stdout:3728 punch from 0x32cc71c to 0x32d1df2, (0x56d6 bytes) 2024-09-14T06:17:08.634 INFO:teuthology.orchestra.run.smithi039.stdout:3729 write 0xb586efc thru 0xb594661 (0xd766 bytes) 2024-09-14T06:17:08.649 INFO:teuthology.orchestra.run.smithi039.stdout:3731 write 0xd3f441d thru 0xd3f6109 (0x1ced bytes) 2024-09-14T06:17:08.653 INFO:teuthology.orchestra.run.smithi039.stdout:3733 read 0x93b5237 thru 0x93c13aa (0xc174 bytes) 2024-09-14T06:17:08.653 INFO:teuthology.orchestra.run.smithi039.stdout:3735 write 0x19ff22e thru 0x1a059e4 (0x67b7 bytes) 2024-09-14T06:17:08.658 INFO:teuthology.orchestra.run.smithi039.stdout:3737 write 0xb8224fa thru 0xb824eb9 (0x29c0 bytes) 2024-09-14T06:17:08.663 INFO:teuthology.orchestra.run.smithi039.stdout:3739 read 0x44c3c3d thru 0x44ce20b (0xa5cf bytes) 2024-09-14T06:17:08.663 INFO:teuthology.orchestra.run.smithi039.stdout:3740 writesame 0x4ff2c28 thru 0x4ffee44 data_size 0x1f(0xc21d bytes) 2024-09-14T06:17:08.667 INFO:teuthology.orchestra.run.smithi039.stdout:3741 punch from 0x4713da7 to 0x471f71d, (0xb976 bytes) 2024-09-14T06:17:08.667 INFO:teuthology.orchestra.run.smithi039.stdout:3742 read 0x836f3b3 thru 0x837e01f (0xec6d bytes) 2024-09-14T06:17:08.667 INFO:teuthology.orchestra.run.smithi039.stdout:3743 read 0x9a91921 thru 0x9a9d162 (0xb842 bytes) 2024-09-14T06:17:08.667 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:08.667 INFO:teuthology.orchestra.run.smithi039.stdout:3745 write 0x1f97ed4 thru 0x1fa4f4e (0xd07b bytes) 2024-09-14T06:17:08.677 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:08.674+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:08.677 INFO:teuthology.orchestra.run.smithi039.stdout:3747 trunc from 0xd3f610a to 0x96a492e 2024-09-14T06:17:08.690 INFO:teuthology.orchestra.run.smithi039.stdout:3748 punch from 0x5fdabd6 to 0x5fe0da5, (0x61cf bytes) 2024-09-14T06:17:08.700 INFO:teuthology.orchestra.run.smithi039.stdout:3749 writesame 0xbc1a782 thru 0xbc290fe data_size 0x1(0xe97d bytes) 2024-09-14T06:17:08.702 INFO:teuthology.orchestra.run.smithi039.stdout:3750 trunc from 0xbc290ff to 0xd0bf09d 2024-09-14T06:17:08.705 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:08.705 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:08.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:08.705 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:08.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.705 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3916 3.16 3:6ab8b1d8:::rbd_data.1152bd92e04f.0000000000000002:head [stat,truncate 1590169] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:17:08.705 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:17:08.705 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.124528157999997, 2024-09-14T06:17:08.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022958078999999999, 2024-09-14T06:17:08.706 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.706 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.706 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:08.706 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:08.706 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:08.706 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:08.706 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3916 2024-09-14T06:17:08.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.706 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:17:08.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:17:08.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579203+0000", 2024-09-14T06:17:08.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.748e-06 2024-09-14T06:17:08.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579208+0000", 2024-09-14T06:17:08.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:17:08.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579223+0000", 2024-09-14T06:17:08.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.508e-05 2024-09-14T06:17:08.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579232+0000", 2024-09-14T06:17:08.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9719999999999998e-06 2024-09-14T06:17:08.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579307+0000", 2024-09-14T06:17:08.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5028000000000001e-05 2024-09-14T06:17:08.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579412+0000", 2024-09-14T06:17:08.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010525400000000001 2024-09-14T06:17:08.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:17:08.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579535+0000", 2024-09-14T06:17:08.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122726 2024-09-14T06:17:08.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:08.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.581415+0000", 2024-09-14T06:17:08.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018800360000000001 2024-09-14T06:17:08.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:08.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602017+0000", 2024-09-14T06:17:08.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.020602425000000001 2024-09-14T06:17:08.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:08.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602096+0000", 2024-09-14T06:17:08.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8968999999999996e-05 2024-09-14T06:17:08.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602159+0000", 2024-09-14T06:17:08.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2824999999999996e-05 2024-09-14T06:17:08.711 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.711 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.712 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.712 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4873 3.e 3:76656e66:::rbd_data.1152bd92e04f.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=2b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:17:08.712 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:17:08.712 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 40.510081618999997, 2024-09-14T06:17:08.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048796120000000002, 2024-09-14T06:17:08.712 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.712 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.712 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:08.712 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:08.712 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:08.712 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:08.713 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4873 2024-09-14T06:17:08.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.713 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:17:08.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:17:08.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193651+0000", 2024-09-14T06:17:08.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:17:08.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193663+0000", 2024-09-14T06:17:08.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2047e-05 2024-09-14T06:17:08.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193692+0000", 2024-09-14T06:17:08.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9346e-05 2024-09-14T06:17:08.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193708+0000", 2024-09-14T06:17:08.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6123000000000001e-05 2024-09-14T06:17:08.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193798+0000", 2024-09-14T06:17:08.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9909999999999998e-05 2024-09-14T06:17:08.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193863+0000", 2024-09-14T06:17:08.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4583000000000002e-05 2024-09-14T06:17:08.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:17:08.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.194717+0000", 2024-09-14T06:17:08.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00085473299999999999 2024-09-14T06:17:08.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:08.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198400+0000", 2024-09-14T06:17:08.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036830890000000001 2024-09-14T06:17:08.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:08.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198461+0000", 2024-09-14T06:17:08.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0844000000000002e-05 2024-09-14T06:17:08.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:08.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198496+0000", 2024-09-14T06:17:08.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4820000000000002e-05 2024-09-14T06:17:08.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198527+0000", 2024-09-14T06:17:08.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1229999999999997e-05 2024-09-14T06:17:08.717 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.718 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.718 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.718 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:6925 3.5 3:a15e9af1:::rbd_data.1152f87cbac6.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:17:08.718 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:17:08.718 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.588950081, 2024-09-14T06:17:08.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0049096490000000003, 2024-09-14T06:17:08.718 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.718 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.718 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:08.718 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:08.718 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:08.719 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:08.719 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 6925 2024-09-14T06:17:08.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.719 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:17:08.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:17:08.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114780+0000", 2024-09-14T06:17:08.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.09e-07 2024-09-14T06:17:08.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114784+0000", 2024-09-14T06:17:08.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4669999999999998e-06 2024-09-14T06:17:08.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114799+0000", 2024-09-14T06:17:08.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4491000000000001e-05 2024-09-14T06:17:08.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114810+0000", 2024-09-14T06:17:08.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1243000000000001e-05 2024-09-14T06:17:08.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115515+0000", 2024-09-14T06:17:08.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00070521899999999998 2024-09-14T06:17:08.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115610+0000", 2024-09-14T06:17:08.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4260999999999998e-05 2024-09-14T06:17:08.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:17:08.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115742+0000", 2024-09-14T06:17:08.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000132414 2024-09-14T06:17:08.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:08.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.116650+0000", 2024-09-14T06:17:08.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00090814900000000002 2024-09-14T06:17:08.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:08.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119568+0000", 2024-09-14T06:17:08.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029178440000000002 2024-09-14T06:17:08.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:08.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119636+0000", 2024-09-14T06:17:08.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8365e-05 2024-09-14T06:17:08.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119689+0000", 2024-09-14T06:17:08.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2386999999999998e-05 2024-09-14T06:17:08.724 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.724 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.724 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.724 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:7151 3.e 3:7793b17d:::rbd_data.1152f87cbac6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:17:08.724 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:17:08.724 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.374132291000002, 2024-09-14T06:17:08.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.029556783, 2024-09-14T06:17:08.724 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.724 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.725 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:08.725 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:08.725 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:08.725 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:08.725 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 7151 2024-09-14T06:17:08.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.725 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:17:08.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:17:08.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329599+0000", 2024-09-14T06:17:08.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5270000000000001e-06 2024-09-14T06:17:08.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329610+0000", 2024-09-14T06:17:08.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0431000000000001e-05 2024-09-14T06:17:08.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329634+0000", 2024-09-14T06:17:08.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3617000000000001e-05 2024-09-14T06:17:08.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329644+0000", 2024-09-14T06:17:08.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.095e-05 2024-09-14T06:17:08.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329712+0000", 2024-09-14T06:17:08.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7067e-05 2024-09-14T06:17:08.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329827+0000", 2024-09-14T06:17:08.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011498 2024-09-14T06:17:08.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:17:08.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.334271+0000", 2024-09-14T06:17:08.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044445839999999997 2024-09-14T06:17:08.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:08.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.358967+0000", 2024-09-14T06:17:08.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024696356999999999 2024-09-14T06:17:08.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:08.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359027+0000", 2024-09-14T06:17:08.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0006000000000001e-05 2024-09-14T06:17:08.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:08.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359099+0000", 2024-09-14T06:17:08.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1052999999999999e-05 2024-09-14T06:17:08.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359154+0000", 2024-09-14T06:17:08.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5210999999999997e-05 2024-09-14T06:17:08.730 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.730 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.730 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.730 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:7151 3.e e32/29 ondisk, result = 0)", 2024-09-14T06:17:08.730 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:17:08.730 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.354187856999999, 2024-09-14T06:17:08.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0096251340000000005, 2024-09-14T06:17:08.730 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.730 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.731 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:08.731 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:17:08.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:17:08.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349543+0000", 2024-09-14T06:17:08.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:17:08.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349552+0000", 2024-09-14T06:17:08.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1039999999999996e-06 2024-09-14T06:17:08.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349579+0000", 2024-09-14T06:17:08.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6449999999999999e-05 2024-09-14T06:17:08.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349593+0000", 2024-09-14T06:17:08.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4099e-05 2024-09-14T06:17:08.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359003+0000", 2024-09-14T06:17:08.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0094096119999999995 2024-09-14T06:17:08.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359014+0000", 2024-09-14T06:17:08.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0871e-05 2024-09-14T06:17:08.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359167+0000", 2024-09-14T06:17:08.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015300700000000001 2024-09-14T06:17:08.734 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.734 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.734 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.734 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15613 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'121, mlcod=36'116)", 2024-09-14T06:17:08.734 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:17:08.735 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 32.410825002000003, 2024-09-14T06:17:08.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053619389999999996, 2024-09-14T06:17:08.735 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.735 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.735 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:08.735 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:17:08.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:17:08.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292905+0000", 2024-09-14T06:17:08.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6800000000000009e-07 2024-09-14T06:17:08.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292915+0000", 2024-09-14T06:17:08.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:17:08.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292927+0000", 2024-09-14T06:17:08.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-09-14T06:17:08.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292935+0000", 2024-09-14T06:17:08.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2459999999999997e-06 2024-09-14T06:17:08.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292992+0000", 2024-09-14T06:17:08.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6873999999999998e-05 2024-09-14T06:17:08.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293016+0000", 2024-09-14T06:17:08.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4037e-05 2024-09-14T06:17:08.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:08.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298217+0000", 2024-09-14T06:17:08.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052008710000000001 2024-09-14T06:17:08.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298266+0000", 2024-09-14T06:17:08.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9691000000000002e-05 2024-09-14T06:17:08.739 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.739 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.739 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.739 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15615 3.11 e36/29 3:8d07d636:::rbd_data.115223d7013d.0000000000000002:head v 36'122, mlcod=36'116)", 2024-09-14T06:17:08.739 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:17:08.739 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 32.409928643000001, 2024-09-14T06:17:08.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053069229999999998, 2024-09-14T06:17:08.739 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.739 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.739 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:08.739 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:17:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:17:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:17:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5799999999999998e-07 2024-09-14T06:17:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294003+0000", 2024-09-14T06:17:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020149600000000001 2024-09-14T06:17:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294016+0000", 2024-09-14T06:17:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3424999999999999e-05 2024-09-14T06:17:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294024+0000", 2024-09-14T06:17:08.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5240000000000003e-06 2024-09-14T06:17:08.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294673+0000", 2024-09-14T06:17:08.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064906700000000003 2024-09-14T06:17:08.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294698+0000", 2024-09-14T06:17:08.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5301999999999999e-05 2024-09-14T06:17:08.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:08.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299064+0000", 2024-09-14T06:17:08.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043652170000000002 2024-09-14T06:17:08.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299108+0000", 2024-09-14T06:17:08.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4033999999999999e-05 2024-09-14T06:17:08.743 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.743 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.743 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.744 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15619 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'124, mlcod=36'119)", 2024-09-14T06:17:08.744 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:08.744 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 32.405874695000001, 2024-09-14T06:17:08.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061545389999999997, 2024-09-14T06:17:08.744 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.744 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.744 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:08.744 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:08.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:08.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:08.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3399999999999998e-07 2024-09-14T06:17:08.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297861+0000", 2024-09-14T06:17:08.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7640000000000002e-06 2024-09-14T06:17:08.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297876+0000", 2024-09-14T06:17:08.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4881e-05 2024-09-14T06:17:08.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297884+0000", 2024-09-14T06:17:08.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.588e-06 2024-09-14T06:17:08.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298294+0000", 2024-09-14T06:17:08.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00041011200000000002 2024-09-14T06:17:08.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298317+0000", 2024-09-14T06:17:08.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3417999999999998e-05 2024-09-14T06:17:08.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:08.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303972+0000", 2024-09-14T06:17:08.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056548680000000004 2024-09-14T06:17:08.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304009+0000", 2024-09-14T06:17:08.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7073999999999998e-05 2024-09-14T06:17:08.748 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.748 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.748 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.748 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15621 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'125, mlcod=36'119)", 2024-09-14T06:17:08.748 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:08.748 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 32.404363678000003, 2024-09-14T06:17:08.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0069496169999999999, 2024-09-14T06:17:08.748 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.748 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.748 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:08.749 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:08.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:08.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:08.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2200000000000003e-07 2024-09-14T06:17:08.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299372+0000", 2024-09-14T06:17:08.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3850000000000001e-06 2024-09-14T06:17:08.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299387+0000", 2024-09-14T06:17:08.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4905e-05 2024-09-14T06:17:08.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299393+0000", 2024-09-14T06:17:08.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9090000000000003e-06 2024-09-14T06:17:08.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303591+0000", 2024-09-14T06:17:08.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041976640000000003 2024-09-14T06:17:08.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303619+0000", 2024-09-14T06:17:08.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7651e-05 2024-09-14T06:17:08.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:08.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306287+0000", 2024-09-14T06:17:08.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026686689999999998 2024-09-14T06:17:08.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306315+0000", 2024-09-14T06:17:08.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7712000000000002e-05 2024-09-14T06:17:08.752 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.752 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.752 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.753 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:16521 3.e e36/34 3:77c8e3e5:::rbd_data.1152fc09e7e6.0000000000000003:head v 36'83, mlcod=36'81)", 2024-09-14T06:17:08.753 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:17:08.753 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 31.777710377999998, 2024-09-14T06:17:08.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044948899999999996, 2024-09-14T06:17:08.753 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.753 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.753 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:08.753 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.753 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:17:08.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:17:08.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926020+0000", 2024-09-14T06:17:08.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.545e-06 2024-09-14T06:17:08.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927192+0000", 2024-09-14T06:17:08.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011711919999999999 2024-09-14T06:17:08.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927215+0000", 2024-09-14T06:17:08.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3677e-05 2024-09-14T06:17:08.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927226+0000", 2024-09-14T06:17:08.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0665e-05 2024-09-14T06:17:08.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927314+0000", 2024-09-14T06:17:08.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8314999999999997e-05 2024-09-14T06:17:08.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927344+0000", 2024-09-14T06:17:08.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9941e-05 2024-09-14T06:17:08.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:08.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.930411+0000", 2024-09-14T06:17:08.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030670649999999999 2024-09-14T06:17:08.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.930514+0000", 2024-09-14T06:17:08.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010249 2024-09-14T06:17:08.757 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.757 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.757 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.757 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:18539 3.1e e40/29 3:7e28f783:::rbd_data.1152227be94f.0000000000000000:head v 40'59, mlcod=40'57)", 2024-09-14T06:17:08.757 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:17:08.757 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.420688327000001, 2024-09-14T06:17:08.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0059097430000000003, 2024-09-14T06:17:08.757 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.757 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.757 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:08.757 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.758 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:17:08.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.758 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:17:08.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.758 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283043+0000", 2024-09-14T06:17:08.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4499999999999998e-06 2024-09-14T06:17:08.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285878+0000", 2024-09-14T06:17:08.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028348039999999998 2024-09-14T06:17:08.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285908+0000", 2024-09-14T06:17:08.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9756000000000001e-05 2024-09-14T06:17:08.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285922+0000", 2024-09-14T06:17:08.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.43e-05 2024-09-14T06:17:08.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.286030+0000", 2024-09-14T06:17:08.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000107954 2024-09-14T06:17:08.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.286058+0000", 2024-09-14T06:17:08.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7796e-05 2024-09-14T06:17:08.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:08.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.288888+0000", 2024-09-14T06:17:08.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028295799999999999 2024-09-14T06:17:08.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.288951+0000", 2024-09-14T06:17:08.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3102999999999995e-05 2024-09-14T06:17:08.761 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.761 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.761 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.761 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:24229 3.5 e47/34 3:a30fd30b:::rbd_data.1152db28e106.0000000000000001:head v 47'55, mlcod=45'53)", 2024-09-14T06:17:08.761 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:17:08.762 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.793718080000001, 2024-09-14T06:17:08.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0092924199999999992, 2024-09-14T06:17:08.762 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.762 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.762 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:08.762 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:17:08.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:17:08.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910013+0000", 2024-09-14T06:17:08.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.812e-06 2024-09-14T06:17:08.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915374+0000", 2024-09-14T06:17:08.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053609240000000004 2024-09-14T06:17:08.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915395+0000", 2024-09-14T06:17:08.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1118e-05 2024-09-14T06:17:08.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915406+0000", 2024-09-14T06:17:08.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1303e-05 2024-09-14T06:17:08.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915490+0000", 2024-09-14T06:17:08.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4017000000000006e-05 2024-09-14T06:17:08.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915515+0000", 2024-09-14T06:17:08.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4712e-05 2024-09-14T06:17:08.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:08.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919240+0000", 2024-09-14T06:17:08.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037247690000000002 2024-09-14T06:17:08.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919304+0000", 2024-09-14T06:17:08.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3764999999999997e-05 2024-09-14T06:17:08.766 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.766 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.766 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.766 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:4, version: 31'46, data_included: [0~98304,360448~278528,704512~131072,1048576~114688,1261568~12416,1409024~196608], data_size: 831616, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:4@31'46, size: 1966080, copy_subset: [(0, 1966080)], clone_subset: {}, snapset: 4=[]:{4: [4]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1966080, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:17:08.766 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.207851+0000", 2024-09-14T06:17:08.766 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.4958778630000005, 2024-09-14T06:17:08.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056781219999999999, 2024-09-14T06:17:08.766 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.766 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.766 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:08.766 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.207851+0000", 2024-09-14T06:17:08.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.207851+0000", 2024-09-14T06:17:08.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.207853+0000", 2024-09-14T06:17:08.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3960000000000001e-06 2024-09-14T06:17:08.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210067+0000", 2024-09-14T06:17:08.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022146290000000001 2024-09-14T06:17:08.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210118+0000", 2024-09-14T06:17:08.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0201999999999998e-05 2024-09-14T06:17:08.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210155+0000", 2024-09-14T06:17:08.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7574999999999999e-05 2024-09-14T06:17:08.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210424+0000", 2024-09-14T06:17:08.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00026907500000000002 2024-09-14T06:17:08.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210467+0000", 2024-09-14T06:17:08.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3061e-05 2024-09-14T06:17:08.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.213530+0000", 2024-09-14T06:17:08.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030621839999999999 2024-09-14T06:17:08.770 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.770 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.770 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.770 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:head, version: 31'47, data_included: [0~98304,360448~278528], data_size: 376832, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:head@31'47, size: 638976, copy_subset: [(0, 638976)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 638976, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:17:08.770 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:17:08.770 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.4934551349999996, 2024-09-14T06:17:08.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062487460000000003, 2024-09-14T06:17:08.770 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.771 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.771 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:08.771 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:17:08.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:17:08.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210275+0000", 2024-09-14T06:17:08.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:17:08.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210752+0000", 2024-09-14T06:17:08.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047772100000000002 2024-09-14T06:17:08.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210781+0000", 2024-09-14T06:17:08.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8391999999999999e-05 2024-09-14T06:17:08.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210805+0000", 2024-09-14T06:17:08.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4142999999999999e-05 2024-09-14T06:17:08.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.213641+0000", 2024-09-14T06:17:08.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002835827 2024-09-14T06:17:08.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.213663+0000", 2024-09-14T06:17:08.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.243e-05 2024-09-14T06:17:08.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.216523+0000", 2024-09-14T06:17:08.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028597729999999999 2024-09-14T06:17:08.774 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.774 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.774 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.774 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fcd050ea:::rbd_data.1152b921b97f.0000000000000001:e, version: 45'188, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:fcd050ea:::rbd_data.1152b921b97f.0000000000000001:e@45'188, size: 3250974, copy_subset: [(0, 3250974)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:17:08.774 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.238893+0000", 2024-09-14T06:17:08.774 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.4648361760000004, 2024-09-14T06:17:08.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044553459999999998, 2024-09-14T06:17:08.775 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.775 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.775 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:08.775 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.238893+0000", 2024-09-14T06:17:08.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.238893+0000", 2024-09-14T06:17:08.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.238918+0000", 2024-09-14T06:17:08.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4845e-05 2024-09-14T06:17:08.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.241095+0000", 2024-09-14T06:17:08.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0021769070000000001 2024-09-14T06:17:08.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.241143+0000", 2024-09-14T06:17:08.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7858000000000002e-05 2024-09-14T06:17:08.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.241175+0000", 2024-09-14T06:17:08.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2592999999999998e-05 2024-09-14T06:17:08.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.241352+0000", 2024-09-14T06:17:08.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0001772 2024-09-14T06:17:08.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.241375+0000", 2024-09-14T06:17:08.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2163000000000001e-05 2024-09-14T06:17:08.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.243348+0000", 2024-09-14T06:17:08.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0019737800000000001 2024-09-14T06:17:08.778 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.778 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.778 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.779 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fcd050ea:::rbd_data.1152b921b97f.0000000000000001:head, version: 45'189, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:fcd050ea:::rbd_data.1152b921b97f.0000000000000001:head@45'189, size: 1760991, copy_subset: [(0, 1760991)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:17:08.779 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.241225+0000", 2024-09-14T06:17:08.779 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.4625041220000004, 2024-09-14T06:17:08.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0042946119999999997, 2024-09-14T06:17:08.779 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.779 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.779 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:08.779 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.241225+0000", 2024-09-14T06:17:08.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.241225+0000", 2024-09-14T06:17:08.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.241226+0000", 2024-09-14T06:17:08.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.083e-06 2024-09-14T06:17:08.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.242910+0000", 2024-09-14T06:17:08.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0016836900000000001 2024-09-14T06:17:08.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.242953+0000", 2024-09-14T06:17:08.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3349000000000001e-05 2024-09-14T06:17:08.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.242980+0000", 2024-09-14T06:17:08.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6522000000000001e-05 2024-09-14T06:17:08.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.243531+0000", 2024-09-14T06:17:08.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00055117499999999999 2024-09-14T06:17:08.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.243562+0000", 2024-09-14T06:17:08.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1285999999999998e-05 2024-09-14T06:17:08.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.245520+0000", 2024-09-14T06:17:08.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0019575069999999998 2024-09-14T06:17:08.783 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.783 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.783 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.783 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fcd050ea:::rbd_data.1152b921b97f.0000000000000001:e, version: 45'188, data_included: [1048576~524288,1703936~262144], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:fcd050ea:::rbd_data.1152b921b97f.0000000000000001:e@45'188, size: 3250974, copy_subset: [(0, 3250974)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3250974, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-14T06:17:08.783 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.247262+0000", 2024-09-14T06:17:08.783 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.4564676179999996, 2024-09-14T06:17:08.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044393130000000003, 2024-09-14T06:17:08.783 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.783 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.783 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:08.784 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.247262+0000", 2024-09-14T06:17:08.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.247262+0000", 2024-09-14T06:17:08.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.247284+0000", 2024-09-14T06:17:08.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2792000000000001e-05 2024-09-14T06:17:08.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.249000+0000", 2024-09-14T06:17:08.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001715299 2024-09-14T06:17:08.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.249047+0000", 2024-09-14T06:17:08.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7425000000000001e-05 2024-09-14T06:17:08.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.249081+0000", 2024-09-14T06:17:08.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4125000000000003e-05 2024-09-14T06:17:08.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.249332+0000", 2024-09-14T06:17:08.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00025113199999999999 2024-09-14T06:17:08.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.249373+0000", 2024-09-14T06:17:08.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0711999999999999e-05 2024-09-14T06:17:08.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.787 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.251701+0000", 2024-09-14T06:17:08.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023278280000000001 2024-09-14T06:17:08.787 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.787 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.787 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.787 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fcd050ea:::rbd_data.1152b921b97f.0000000000000001:head, version: 45'189, data_included: [1048576~524288,1703936~57055], data_size: 581343, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:fcd050ea:::rbd_data.1152b921b97f.0000000000000001:head@45'189, size: 1760991, copy_subset: [(0, 1760991)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1760991, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-14T06:17:08.787 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.249132+0000", 2024-09-14T06:17:08.787 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.4545973910000001, 2024-09-14T06:17:08.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044146469999999998, 2024-09-14T06:17:08.788 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.788 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.788 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:08.788 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.249132+0000", 2024-09-14T06:17:08.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.249132+0000", 2024-09-14T06:17:08.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.249133+0000", 2024-09-14T06:17:08.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0130000000000001e-06 2024-09-14T06:17:08.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.250157+0000", 2024-09-14T06:17:08.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001024608 2024-09-14T06:17:08.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.250240+0000", 2024-09-14T06:17:08.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2484000000000005e-05 2024-09-14T06:17:08.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.250295+0000", 2024-09-14T06:17:08.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4737999999999997e-05 2024-09-14T06:17:08.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.251872+0000", 2024-09-14T06:17:08.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0015773269999999999 2024-09-14T06:17:08.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.791 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.791 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.251893+0000", 2024-09-14T06:17:08.791 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0919000000000001e-05 2024-09-14T06:17:08.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.791 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.791 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.253547+0000", 2024-09-14T06:17:08.791 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0016535580000000001 2024-09-14T06:17:08.791 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.791 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.791 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.791 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:30750 3.2 e55/29 3:452a53e9:::rbd_data.11529eb229bd.0000000000000003:head v 55'170, mlcod=55'168)", 2024-09-14T06:17:08.792 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.760736+0000", 2024-09-14T06:17:08.792 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 7.9429930259999999, 2024-09-14T06:17:08.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047328800000000001, 2024-09-14T06:17:08.792 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.792 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.792 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:08.792 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.760736+0000", 2024-09-14T06:17:08.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.760736+0000", 2024-09-14T06:17:08.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.760738+0000", 2024-09-14T06:17:08.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.917e-06 2024-09-14T06:17:08.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.763675+0000", 2024-09-14T06:17:08.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029371380000000002 2024-09-14T06:17:08.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.763705+0000", 2024-09-14T06:17:08.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9929000000000001e-05 2024-09-14T06:17:08.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.763725+0000", 2024-09-14T06:17:08.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9629000000000001e-05 2024-09-14T06:17:08.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.763893+0000", 2024-09-14T06:17:08.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016794100000000001 2024-09-14T06:17:08.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.763930+0000", 2024-09-14T06:17:08.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7595000000000002e-05 2024-09-14T06:17:08.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:08.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.765381+0000", 2024-09-14T06:17:08.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001450221 2024-09-14T06:17:08.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.796 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.765469+0000", 2024-09-14T06:17:08.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8510000000000005e-05 2024-09-14T06:17:08.796 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.796 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.796 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.796 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:32082 3.11 e58/29 3:8be13bcd:::rbd_object_map.115274ed0aa8:head v 58'244, mlcod=58'240)", 2024-09-14T06:17:08.796 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:04.005484+0000", 2024-09-14T06:17:08.796 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.6982452280000002, 2024-09-14T06:17:08.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050265900000000001, 2024-09-14T06:17:08.796 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:08.797 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:08.797 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:08.797 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:08.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:08.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.005484+0000", 2024-09-14T06:17:08.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:08.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.005484+0000", 2024-09-14T06:17:08.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:08.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:08.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.005485+0000", 2024-09-14T06:17:08.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:17:08.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:08.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.005490+0000", 2024-09-14T06:17:08.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7820000000000001e-06 2024-09-14T06:17:08.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:08.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.005498+0000", 2024-09-14T06:17:08.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2709999999999992e-06 2024-09-14T06:17:08.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:08.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.005505+0000", 2024-09-14T06:17:08.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9480000000000002e-06 2024-09-14T06:17:08.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:08.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.005536+0000", 2024-09-14T06:17:08.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.154e-05 2024-09-14T06:17:08.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:08.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.005548+0000", 2024-09-14T06:17:08.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1695999999999999e-05 2024-09-14T06:17:08.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:08.800 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.010470+0000", 2024-09-14T06:17:08.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0049218780000000002 2024-09-14T06:17:08.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:08.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:08.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:08.800 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.010511+0000", 2024-09-14T06:17:08.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0781999999999997e-05 2024-09-14T06:17:08.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.800 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.801 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:08.801 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:08.801 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:08.802 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:08.802 INFO:teuthology.orchestra.run.smithi039.stdout:3752 read 0x373ca7e thru 0x374288d (0x5e10 bytes) 2024-09-14T06:17:08.802 INFO:teuthology.orchestra.run.smithi039.stdout:3753 write 0x30592e0 thru 0x305ea10 (0x5731 bytes) 2024-09-14T06:17:08.802 INFO:teuthology.orchestra.run.smithi039.stdout:3754 write 0x4492cd7 thru 0x449860d (0x5937 bytes) 2024-09-14T06:17:08.803 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:17:08.805 INFO:teuthology.orchestra.run.smithi039.stdout:3756 write 0x60f4bfc thru 0x60fea5c (0x9e61 bytes) 2024-09-14T06:17:08.805 INFO:teuthology.orchestra.run.smithi039.stdout:3758 punch from 0xb1c8a19 to 0xb1caf04, (0x24eb bytes) 2024-09-14T06:17:08.805 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:08.805 INFO:teuthology.orchestra.run.smithi039.stdout:3761 punch from 0x3f9c54e to 0x3fa74cf, (0xaf81 bytes) 2024-09-14T06:17:08.805 INFO:teuthology.orchestra.run.smithi039.stdout:3762 read 0xae6307e thru 0xae719ad (0xe930 bytes) 2024-09-14T06:17:08.805 INFO:teuthology.orchestra.run.smithi039.stdout:3763 write 0x7a27aaa thru 0x7a2a84d (0x2da4 bytes) 2024-09-14T06:17:08.805 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:08.805 INFO:teuthology.orchestra.run.smithi039.stdout:3765 read 0x1f2386f thru 0x1f2595e (0x20f0 bytes) 2024-09-14T06:17:08.805 INFO:teuthology.orchestra.run.smithi039.stdout:3766 write 0xa79420b thru 0xa7a07ce (0xc5c4 bytes) 2024-09-14T06:17:08.805 INFO:teuthology.orchestra.run.smithi039.stdout:3768 read 0xb073eeb thru 0xb075c39 (0x1d4f bytes) 2024-09-14T06:17:08.806 INFO:teuthology.orchestra.run.smithi039.stdout:3769 read 0x8493f66 thru 0x849e33c (0xa3d7 bytes) 2024-09-14T06:17:08.806 INFO:teuthology.orchestra.run.smithi039.stdout:3770 read 0xb3745c6 thru 0xb378091 (0x3acc bytes) 2024-09-14T06:17:08.806 INFO:teuthology.orchestra.run.smithi039.stdout:3771 write 0x4c095f1 thru 0x4c0a564 (0xf74 bytes) 2024-09-14T06:17:08.806 INFO:teuthology.orchestra.run.smithi039.stdout:3772 write 0x5391acb thru 0x5393112 (0x1648 bytes) 2024-09-14T06:17:08.806 INFO:teuthology.orchestra.run.smithi039.stdout:3773 punch from 0x3064d9a to 0x3065749, (0x9af bytes) 2024-09-14T06:17:08.806 INFO:teuthology.orchestra.run.smithi039.stdout:3774 read 0x255bb8d thru 0x255f406 (0x387a bytes) 2024-09-14T06:17:08.806 INFO:teuthology.orchestra.run.smithi039.stdout:3775 write 0xe06e4de thru 0xe06fadf (0x1602 bytes) 2024-09-14T06:17:08.806 INFO:teuthology.orchestra.run.smithi039.stdout:3776 trunc from 0xe06fae0 to 0x1beb508 2024-09-14T06:17:08.806 INFO:teuthology.orchestra.run.smithi039.stdout:3777 trunc from 0x1beb508 to 0x4656962 2024-09-14T06:17:08.806 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:08.798+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:08.807 INFO:teuthology.orchestra.run.smithi039.stdout:3778 punch from 0xa7b76 to 0xa8f27, (0x13b1 bytes) 2024-09-14T06:17:08.807 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:08.807 INFO:teuthology.orchestra.run.smithi039.stdout:3780 punch from 0x1eba7ff to 0x1ebefba, (0x47bb bytes) 2024-09-14T06:17:08.808 INFO:teuthology.orchestra.run.smithi039.stdout:3781 trunc from 0x4656962 to 0x84e8a5d 2024-09-14T06:17:08.814 INFO:teuthology.orchestra.run.smithi039.stdout:3782 clone 20 order 20 su 131072 sc 14 2024-09-14T06:17:08.903 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:08.902+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:08.930 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:08.930 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:08.930 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:08.930 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:08.945 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:17:09.031 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:09.030+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:09.105 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:09.102+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:09.112 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:09.112 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:09.127 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:17:09.207 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:09.202+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:09.305 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11217 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 925696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:17:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.310049300000003, 2024-09-14T06:17:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049306313999999997, 2024-09-14T06:17:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11217 2024-09-14T06:17:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:17:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:17:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991344+0000", 2024-09-14T06:17:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.29e-07 2024-09-14T06:17:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991345+0000", 2024-09-14T06:17:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1480000000000001e-06 2024-09-14T06:17:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991349+0000", 2024-09-14T06:17:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9840000000000003e-06 2024-09-14T06:17:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991359+0000", 2024-09-14T06:17:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4050000000000006e-06 2024-09-14T06:17:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.030826+0000", 2024-09-14T06:17:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.039467214 2024-09-14T06:17:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.030894+0000", 2024-09-14T06:17:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8400999999999998e-05 2024-09-14T06:17:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040650+0000", 2024-09-14T06:17:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0097559329999999996 2024-09-14T06:17:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11262 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 81920~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:17:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.309863264000001, 2024-09-14T06:17:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049588879000000002, 2024-09-14T06:17:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11262 2024-09-14T06:17:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:17:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:17:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991530+0000", 2024-09-14T06:17:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9599999999999999e-07 2024-09-14T06:17:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991533+0000", 2024-09-14T06:17:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.362e-06 2024-09-14T06:17:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991539+0000", 2024-09-14T06:17:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:17:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991544+0000", 2024-09-14T06:17:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:17:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040755+0000", 2024-09-14T06:17:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049210938000000003 2024-09-14T06:17:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.040833+0000", 2024-09-14T06:17:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7812999999999995e-05 2024-09-14T06:17:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041119+0000", 2024-09-14T06:17:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0002855 2024-09-14T06:17:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11263 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.309840485999999, 2024-09-14T06:17:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049944574999999998, 2024-09-14T06:17:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11263 2024-09-14T06:17:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.51e-07 2024-09-14T06:17:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991560+0000", 2024-09-14T06:17:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6719999999999998e-06 2024-09-14T06:17:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991567+0000", 2024-09-14T06:17:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1470000000000004e-06 2024-09-14T06:17:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991572+0000", 2024-09-14T06:17:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.237e-06 2024-09-14T06:17:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041149+0000", 2024-09-14T06:17:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049576988000000002 2024-09-14T06:17:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041277+0000", 2024-09-14T06:17:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000127687 2024-09-14T06:17:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041497+0000", 2024-09-14T06:17:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022039299999999999 2024-09-14T06:17:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11264 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 327680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.309814144999997, 2024-09-14T06:17:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050248567000000001, 2024-09-14T06:17:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11264 2024-09-14T06:17:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.41e-07 2024-09-14T06:17:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991581+0000", 2024-09-14T06:17:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3740000000000001e-06 2024-09-14T06:17:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991585+0000", 2024-09-14T06:17:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9400000000000004e-06 2024-09-14T06:17:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991587+0000", 2024-09-14T06:17:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.694e-06 2024-09-14T06:17:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041527+0000", 2024-09-14T06:17:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049939349000000001 2024-09-14T06:17:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041598+0000", 2024-09-14T06:17:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1633999999999999e-05 2024-09-14T06:17:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041827+0000", 2024-09-14T06:17:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000229235 2024-09-14T06:17:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.309800805000002, 2024-09-14T06:17:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:17:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:17:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:17:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:17:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:17:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:17:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:17:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:17:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:17:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:17:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:17:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:17:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:17:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:17:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:17:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.309787733, 2024-09-14T06:17:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:17:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:17:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:17:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:17:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:17:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:17:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:17:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:17:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:17:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:17:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:17:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:17:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:17:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:17:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:17:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:17:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.309770301, 2024-09-14T06:17:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:17:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:17:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:17:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:17:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:17:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:17:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:17:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:17:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:17:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:17:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:17:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:17:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:17:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:17:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:17:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.309749195999999, 2024-09-14T06:17:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:17:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:17:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:17:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:17:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:17:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:17:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:17:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:17:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:17:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:17:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:17:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:17:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:17:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:17:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:17:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.309728088, 2024-09-14T06:17:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:17:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:17:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:17:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:17:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:17:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:17:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:17:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:17:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:17:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:17:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:17:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:17:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:17:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:17:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:17:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13590 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 491520~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:17:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.163753206000003, 2024-09-14T06:17:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042553466999999998, 2024-09-14T06:17:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13590 2024-09-14T06:17:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:17:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:17:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137640+0000", 2024-09-14T06:17:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.34e-07 2024-09-14T06:17:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137642+0000", 2024-09-14T06:17:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0150000000000002e-06 2024-09-14T06:17:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137649+0000", 2024-09-14T06:17:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4010000000000002e-06 2024-09-14T06:17:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137653+0000", 2024-09-14T06:17:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:17:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.179985+0000", 2024-09-14T06:17:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042331580000000001 2024-09-14T06:17:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180076+0000", 2024-09-14T06:17:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0796000000000003e-05 2024-09-14T06:17:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180193+0000", 2024-09-14T06:17:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000117471 2024-09-14T06:17:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13592 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 606208~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:17:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.163728638000002, 2024-09-14T06:17:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042709440000000001, 2024-09-14T06:17:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13592 2024-09-14T06:17:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:17:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137664+0000", 2024-09-14T06:17:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137665+0000", 2024-09-14T06:17:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4299999999999998e-07 2024-09-14T06:17:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137667+0000", 2024-09-14T06:17:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5780000000000001e-06 2024-09-14T06:17:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137675+0000", 2024-09-14T06:17:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0870000000000001e-06 2024-09-14T06:17:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137680+0000", 2024-09-14T06:17:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4999999999999999e-06 2024-09-14T06:17:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180223+0000", 2024-09-14T06:17:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042542808000000001 2024-09-14T06:17:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180294+0000", 2024-09-14T06:17:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1497999999999999e-05 2024-09-14T06:17:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180374+0000", 2024-09-14T06:17:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9525999999999997e-05 2024-09-14T06:17:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13594 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 729088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:17:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.163705073999999, 2024-09-14T06:17:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042830604000000001, 2024-09-14T06:17:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13594 2024-09-14T06:17:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:17:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:17:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137688+0000", 2024-09-14T06:17:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0699999999999998e-07 2024-09-14T06:17:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137690+0000", 2024-09-14T06:17:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.012e-06 2024-09-14T06:17:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137698+0000", 2024-09-14T06:17:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4379999999999998e-06 2024-09-14T06:17:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137703+0000", 2024-09-14T06:17:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1250000000000001e-06 2024-09-14T06:17:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180404+0000", 2024-09-14T06:17:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042701465000000001 2024-09-14T06:17:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180456+0000", 2024-09-14T06:17:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1227999999999998e-05 2024-09-14T06:17:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180519+0000", 2024-09-14T06:17:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-09-14T06:17:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13595 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 851968~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:17:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.163682569000002, 2024-09-14T06:17:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043012433000000003, 2024-09-14T06:17:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13595 2024-09-14T06:17:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:17:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137710+0000", 2024-09-14T06:17:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137711+0000", 2024-09-14T06:17:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:17:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137713+0000", 2024-09-14T06:17:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5170000000000002e-06 2024-09-14T06:17:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137720+0000", 2024-09-14T06:17:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-09-14T06:17:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137726+0000", 2024-09-14T06:17:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9580000000000004e-06 2024-09-14T06:17:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180551+0000", 2024-09-14T06:17:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042825611999999999 2024-09-14T06:17:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180623+0000", 2024-09-14T06:17:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1673000000000003e-05 2024-09-14T06:17:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180723+0000", 2024-09-14T06:17:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.9928999999999998e-05 2024-09-14T06:17:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13597 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 974848~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:17:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.163659684000002, 2024-09-14T06:17:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043180442999999999, 2024-09-14T06:17:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13597 2024-09-14T06:17:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:17:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137733+0000", 2024-09-14T06:17:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137734+0000", 2024-09-14T06:17:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4499999999999997e-07 2024-09-14T06:17:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137736+0000", 2024-09-14T06:17:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0470000000000001e-06 2024-09-14T06:17:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137742+0000", 2024-09-14T06:17:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2269999999999998e-06 2024-09-14T06:17:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.137747+0000", 2024-09-14T06:17:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6670000000000004e-06 2024-09-14T06:17:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180753+0000", 2024-09-14T06:17:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043006132000000002 2024-09-14T06:17:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180820+0000", 2024-09-14T06:17:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7516999999999997e-05 2024-09-14T06:17:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.180914+0000", 2024-09-14T06:17:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3407999999999997e-05 2024-09-14T06:17:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13684 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 999424~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:17:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.161427140999997, 2024-09-14T06:17:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042643702999999998, 2024-09-14T06:17:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13684 2024-09-14T06:17:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:17:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139966+0000", 2024-09-14T06:17:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.139967+0000", 2024-09-14T06:17:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1200000000000003e-07 2024-09-14T06:17:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141080+0000", 2024-09-14T06:17:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011133029999999999 2024-09-14T06:17:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141084+0000", 2024-09-14T06:17:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.544e-06 2024-09-14T06:17:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141088+0000", 2024-09-14T06:17:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0280000000000002e-06 2024-09-14T06:17:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182457+0000", 2024-09-14T06:17:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041368690999999999 2024-09-14T06:17:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182528+0000", 2024-09-14T06:17:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0881000000000001e-05 2024-09-14T06:17:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.182610+0000", 2024-09-14T06:17:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1644000000000006e-05 2024-09-14T06:17:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13721 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 532480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:17:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.160126407, 2024-09-14T06:17:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042577526999999997, 2024-09-14T06:17:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13721 2024-09-14T06:17:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:17:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:17:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141267+0000", 2024-09-14T06:17:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2999999999999999e-07 2024-09-14T06:17:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141268+0000", 2024-09-14T06:17:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0130000000000001e-06 2024-09-14T06:17:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141271+0000", 2024-09-14T06:17:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0970000000000002e-06 2024-09-14T06:17:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141274+0000", 2024-09-14T06:17:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5809999999999999e-06 2024-09-14T06:17:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183522+0000", 2024-09-14T06:17:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042248591000000002 2024-09-14T06:17:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183691+0000", 2024-09-14T06:17:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016885399999999999 2024-09-14T06:17:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183844+0000", 2024-09-14T06:17:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015316099999999999 2024-09-14T06:17:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13722 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 655360~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:17:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.160116105, 2024-09-14T06:17:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042741610999999999, 2024-09-14T06:17:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13722 2024-09-14T06:17:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:17:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:17:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141277+0000", 2024-09-14T06:17:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:17:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141278+0000", 2024-09-14T06:17:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.066e-06 2024-09-14T06:17:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141282+0000", 2024-09-14T06:17:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7100000000000001e-06 2024-09-14T06:17:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141285+0000", 2024-09-14T06:17:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.554e-06 2024-09-14T06:17:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183824+0000", 2024-09-14T06:17:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042539043999999998 2024-09-14T06:17:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.183906+0000", 2024-09-14T06:17:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2298000000000004e-05 2024-09-14T06:17:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184019+0000", 2024-09-14T06:17:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000112724 2024-09-14T06:17:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13723 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 778240~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:17:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.160105135999999, 2024-09-14T06:17:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042995901000000003, 2024-09-14T06:17:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13723 2024-09-14T06:17:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:17:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:17:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141288+0000", 2024-09-14T06:17:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9799999999999999e-07 2024-09-14T06:17:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141289+0000", 2024-09-14T06:17:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0589999999999999e-06 2024-09-14T06:17:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141292+0000", 2024-09-14T06:17:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.208e-06 2024-09-14T06:17:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141295+0000", 2024-09-14T06:17:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5160000000000001e-06 2024-09-14T06:17:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184069+0000", 2024-09-14T06:17:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042774176999999997 2024-09-14T06:17:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184169+0000", 2024-09-14T06:17:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000100073 2024-09-14T06:17:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184284+0000", 2024-09-14T06:17:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011457 2024-09-14T06:17:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13724 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 901120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:17:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.160094741999998, 2024-09-14T06:17:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043197179000000002, 2024-09-14T06:17:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13724 2024-09-14T06:17:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:17:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141298+0000", 2024-09-14T06:17:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141299+0000", 2024-09-14T06:17:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.03e-07 2024-09-14T06:17:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141300+0000", 2024-09-14T06:17:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0389999999999999e-06 2024-09-14T06:17:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141303+0000", 2024-09-14T06:17:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0479999999999999e-06 2024-09-14T06:17:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141305+0000", 2024-09-14T06:17:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5739999999999998e-06 2024-09-14T06:17:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184317+0000", 2024-09-14T06:17:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043011701999999999 2024-09-14T06:17:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184394+0000", 2024-09-14T06:17:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7324999999999996e-05 2024-09-14T06:17:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184496+0000", 2024-09-14T06:17:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010118799999999999 2024-09-14T06:17:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:13726 3.14 3:2c19d8a5:::rbd_data.1152f87cbac6.0000000000000002:6 [read 1024000~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:17:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.160084468999997, 2024-09-14T06:17:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043397185999999997, 2024-09-14T06:17:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 13726 2024-09-14T06:17:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:17:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:17:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141309+0000", 2024-09-14T06:17:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:17:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141310+0000", 2024-09-14T06:17:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:17:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141313+0000", 2024-09-14T06:17:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0309999999999999e-06 2024-09-14T06:17:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.141315+0000", 2024-09-14T06:17:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.52e-06 2024-09-14T06:17:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184527+0000", 2024-09-14T06:17:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.043211855 2024-09-14T06:17:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184606+0000", 2024-09-14T06:17:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8287000000000004e-05 2024-09-14T06:17:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.184706+0000", 2024-09-14T06:17:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010021700000000001 2024-09-14T06:17:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.400 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:09.400 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:09.400 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:09.401 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:17:09.402 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:09.398+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:09.502 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:09.498+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:09.533 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:09.533 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:09.533 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:09.533 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:09.547 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:17:09.576 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-14T06:17:09.602 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:09.598+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:09.691 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:09.691 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:09.691 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:09.691 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:09.691 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:09.703 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:09.702+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:09.713 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:17:09.804 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:09.802+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:09.906 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:09.901+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:09.942 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:09.942 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:09.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:09.942 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:09.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.942 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13229 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.942 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:09.942 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.841582786000004, 2024-09-14T06:17:09.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081798868999999996, 2024-09-14T06:17:09.942 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:09.942 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:09.943 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:09.943 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:09.943 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:09.943 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.943 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13229 2024-09-14T06:17:09.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.943 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:09.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.943 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:09.943 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:09.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.943 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:09.944 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:09.944 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.944 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:09.944 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098710+0000", 2024-09-14T06:17:09.944 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8199999999999999e-07 2024-09-14T06:17:09.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.944 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:09.944 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098714+0000", 2024-09-14T06:17:09.944 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:17:09.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:09.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098727+0000", 2024-09-14T06:17:09.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2520999999999999e-05 2024-09-14T06:17:09.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098749+0000", 2024-09-14T06:17:09.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2773999999999999e-05 2024-09-14T06:17:09.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:09.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160234+0000", 2024-09-14T06:17:09.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.061484430999999999 2024-09-14T06:17:09.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:09.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160281+0000", 2024-09-14T06:17:09.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7437e-05 2024-09-14T06:17:09.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:09.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160569+0000", 2024-09-14T06:17:09.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028788700000000002 2024-09-14T06:17:09.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:09.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163121+0000", 2024-09-14T06:17:09.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025514090000000001 2024-09-14T06:17:09.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:09.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180410+0000", 2024-09-14T06:17:09.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.017289098999999999 2024-09-14T06:17:09.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:09.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180472+0000", 2024-09-14T06:17:09.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2547999999999998e-05 2024-09-14T06:17:09.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:09.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180508+0000", 2024-09-14T06:17:09.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5534000000000002e-05 2024-09-14T06:17:09.947 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.947 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:09.948 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.948 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.948 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13230 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.948 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:09.948 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.841501584, 2024-09-14T06:17:09.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081851558000000005, 2024-09-14T06:17:09.948 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:09.948 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:09.948 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:09.948 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:09.948 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:09.948 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.949 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13230 2024-09-14T06:17:09.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.949 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:09.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.949 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:09.949 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:09.949 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.949 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:09.949 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:09.949 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:09.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098791+0000", 2024-09-14T06:17:09.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8800000000000001e-07 2024-09-14T06:17:09.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:09.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098795+0000", 2024-09-14T06:17:09.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.391e-06 2024-09-14T06:17:09.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:09.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098804+0000", 2024-09-14T06:17:09.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3440000000000001e-06 2024-09-14T06:17:09.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098810+0000", 2024-09-14T06:17:09.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2339999999999999e-06 2024-09-14T06:17:09.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:09.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160930+0000", 2024-09-14T06:17:09.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062120018999999999 2024-09-14T06:17:09.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:09.952 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161017+0000", 2024-09-14T06:17:09.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6868999999999998e-05 2024-09-14T06:17:09.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.952 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:09.952 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161300+0000", 2024-09-14T06:17:09.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028349499999999998 2024-09-14T06:17:09.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.952 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:09.952 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164612+0000", 2024-09-14T06:17:09.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033119680000000002 2024-09-14T06:17:09.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.953 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.953 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:09.953 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180559+0000", 2024-09-14T06:17:09.953 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015946612999999998 2024-09-14T06:17:09.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.953 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.953 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:09.953 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180605+0000", 2024-09-14T06:17:09.953 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5890000000000003e-05 2024-09-14T06:17:09.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.953 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.953 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:09.953 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180642+0000", 2024-09-14T06:17:09.953 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:17:09.954 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.954 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:09.954 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.954 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13231 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.954 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:09.954 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.841239170000001, 2024-09-14T06:17:09.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081750685000000003, 2024-09-14T06:17:09.954 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:09.954 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:09.954 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:09.954 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:09.955 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:09.955 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.955 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13231 2024-09-14T06:17:09.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.955 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:09.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:09.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:09.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:09.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:09.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:09.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099053+0000", 2024-09-14T06:17:09.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:17:09.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:09.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099059+0000", 2024-09-14T06:17:09.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1789999999999999e-06 2024-09-14T06:17:09.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:09.957 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099072+0000", 2024-09-14T06:17:09.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3179999999999999e-05 2024-09-14T06:17:09.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.957 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.957 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099080+0000", 2024-09-14T06:17:09.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9030000000000004e-06 2024-09-14T06:17:09.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.957 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:09.957 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161681+0000", 2024-09-14T06:17:09.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062601641999999999 2024-09-14T06:17:09.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:09.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161766+0000", 2024-09-14T06:17:09.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4909999999999998e-05 2024-09-14T06:17:09.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.958 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:09.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162109+0000", 2024-09-14T06:17:09.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00034264899999999999 2024-09-14T06:17:09.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.958 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:09.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164615+0000", 2024-09-14T06:17:09.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025061979999999998 2024-09-14T06:17:09.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:09.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180718+0000", 2024-09-14T06:17:09.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016103240000000001 2024-09-14T06:17:09.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:09.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180768+0000", 2024-09-14T06:17:09.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9781999999999999e-05 2024-09-14T06:17:09.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:09.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180803+0000", 2024-09-14T06:17:09.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5021999999999998e-05 2024-09-14T06:17:09.960 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.960 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:09.960 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.960 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13232 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.960 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:09.960 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.841197395999998, 2024-09-14T06:17:09.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081861981, 2024-09-14T06:17:09.960 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:09.960 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:09.960 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:09.960 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:09.961 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:09.961 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.961 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13232 2024-09-14T06:17:09.961 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.961 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:09.961 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.961 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:09.961 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:09.961 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.961 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.961 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.961 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:09.961 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:09.962 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.962 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.962 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:09.962 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099095+0000", 2024-09-14T06:17:09.962 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:17:09.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.962 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.962 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:09.962 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099098+0000", 2024-09-14T06:17:09.962 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5420000000000001e-06 2024-09-14T06:17:09.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.962 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:09.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099106+0000", 2024-09-14T06:17:09.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.824e-06 2024-09-14T06:17:09.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099112+0000", 2024-09-14T06:17:09.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9789999999999998e-06 2024-09-14T06:17:09.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:09.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162576+0000", 2024-09-14T06:17:09.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.063463723999999999 2024-09-14T06:17:09.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:09.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162627+0000", 2024-09-14T06:17:09.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1530000000000003e-05 2024-09-14T06:17:09.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:09.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162854+0000", 2024-09-14T06:17:09.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000226159 2024-09-14T06:17:09.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:09.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166090+0000", 2024-09-14T06:17:09.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032362319999999999 2024-09-14T06:17:09.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.965 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.965 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:09.965 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180877+0000", 2024-09-14T06:17:09.965 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014786726 2024-09-14T06:17:09.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.965 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.965 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:09.965 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180920+0000", 2024-09-14T06:17:09.965 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3427000000000003e-05 2024-09-14T06:17:09.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.965 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.965 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:09.965 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180956+0000", 2024-09-14T06:17:09.966 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6195999999999997e-05 2024-09-14T06:17:09.966 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.966 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:09.966 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.966 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.966 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.966 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13233 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.966 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:09.966 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.841172133000001, 2024-09-14T06:17:09.966 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081924860000000002, 2024-09-14T06:17:09.966 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:09.966 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:09.966 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:09.966 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:09.967 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:09.967 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.967 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13233 2024-09-14T06:17:09.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.967 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:09.967 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.967 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:09.967 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:09.967 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.967 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.967 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:09.967 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:09.967 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.968 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.968 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:09.968 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099120+0000", 2024-09-14T06:17:09.968 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:17:09.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.968 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.968 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:09.968 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099122+0000", 2024-09-14T06:17:09.968 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:17:09.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.968 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.968 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:09.968 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099126+0000", 2024-09-14T06:17:09.968 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2540000000000003e-06 2024-09-14T06:17:09.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.969 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.969 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099130+0000", 2024-09-14T06:17:09.969 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4060000000000001e-06 2024-09-14T06:17:09.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.969 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:09.969 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163169+0000", 2024-09-14T06:17:09.969 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064039415000000002 2024-09-14T06:17:09.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.969 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:09.970 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163260+0000", 2024-09-14T06:17:09.970 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1038000000000005e-05 2024-09-14T06:17:09.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.970 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.970 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:09.970 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163640+0000", 2024-09-14T06:17:09.970 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037954699999999999 2024-09-14T06:17:09.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.970 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.970 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:09.970 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166093+0000", 2024-09-14T06:17:09.970 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024533229999999999 2024-09-14T06:17:09.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.970 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.971 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:09.971 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180993+0000", 2024-09-14T06:17:09.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01489997 2024-09-14T06:17:09.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.971 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.971 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:09.971 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181020+0000", 2024-09-14T06:17:09.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6806999999999999e-05 2024-09-14T06:17:09.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.971 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.971 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:09.971 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181044+0000", 2024-09-14T06:17:09.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4405000000000001e-05 2024-09-14T06:17:09.971 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.972 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:09.972 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.972 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.972 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13234 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.972 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:09.972 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.841129928000001, 2024-09-14T06:17:09.972 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082015597999999995, 2024-09-14T06:17:09.972 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:09.972 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:09.972 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:09.972 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:09.973 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:09.973 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.973 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13234 2024-09-14T06:17:09.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.973 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:09.973 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:09.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:09.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.973 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:09.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:09.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.974 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.974 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:09.974 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:09.974 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0999999999999998e-07 2024-09-14T06:17:09.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.974 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.974 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:09.974 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099166+0000", 2024-09-14T06:17:09.974 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:17:09.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.974 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.974 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:09.974 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099171+0000", 2024-09-14T06:17:09.974 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6949999999999997e-06 2024-09-14T06:17:09.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.975 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.975 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.975 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099175+0000", 2024-09-14T06:17:09.975 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9029999999999997e-06 2024-09-14T06:17:09.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.975 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.975 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:09.975 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164063+0000", 2024-09-14T06:17:09.975 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064887608999999999 2024-09-14T06:17:09.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.975 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.975 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:09.975 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164111+0000", 2024-09-14T06:17:09.975 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7648999999999997e-05 2024-09-14T06:17:09.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.976 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:09.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164351+0000", 2024-09-14T06:17:09.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024072899999999999 2024-09-14T06:17:09.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.976 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:09.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167561+0000", 2024-09-14T06:17:09.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032094179999999999 2024-09-14T06:17:09.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.976 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:09.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181093+0000", 2024-09-14T06:17:09.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013532245 2024-09-14T06:17:09.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.977 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:09.977 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181139+0000", 2024-09-14T06:17:09.977 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6483999999999997e-05 2024-09-14T06:17:09.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.977 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:09.977 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181177+0000", 2024-09-14T06:17:09.977 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7861000000000002e-05 2024-09-14T06:17:09.977 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.977 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:09.977 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.978 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.978 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13235 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.978 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:09.978 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.841022654, 2024-09-14T06:17:09.978 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081988088000000001, 2024-09-14T06:17:09.978 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:09.978 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:09.978 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:09.978 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:09.978 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:09.978 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.978 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13235 2024-09-14T06:17:09.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.978 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:09.978 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:09.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:09.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:09.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:09.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:09.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099270+0000", 2024-09-14T06:17:09.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7400000000000003e-07 2024-09-14T06:17:09.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.980 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:09.980 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099276+0000", 2024-09-14T06:17:09.980 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:17:09.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.980 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.980 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:09.980 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099284+0000", 2024-09-14T06:17:09.980 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7979999999999995e-06 2024-09-14T06:17:09.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.980 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.980 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.980 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099289+0000", 2024-09-14T06:17:09.980 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:17:09.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.981 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.981 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:09.981 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164664+0000", 2024-09-14T06:17:09.981 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065374901999999999 2024-09-14T06:17:09.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.981 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.981 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:09.981 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164755+0000", 2024-09-14T06:17:09.981 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0790000000000003e-05 2024-09-14T06:17:09.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.981 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:09.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165143+0000", 2024-09-14T06:17:09.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000387904 2024-09-14T06:17:09.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.982 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:09.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167564+0000", 2024-09-14T06:17:09.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024208989999999998 2024-09-14T06:17:09.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.982 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:09.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181212+0000", 2024-09-14T06:17:09.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013648664 2024-09-14T06:17:09.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.983 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:09.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181237+0000", 2024-09-14T06:17:09.983 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5177999999999998e-05 2024-09-14T06:17:09.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.983 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:09.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181257+0000", 2024-09-14T06:17:09.983 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9660000000000002e-05 2024-09-14T06:17:09.983 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.983 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:09.983 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.984 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.984 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13236 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.984 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:09.984 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.840915834999997, 2024-09-14T06:17:09.984 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082010939000000005, 2024-09-14T06:17:09.984 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:09.984 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:09.984 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:09.984 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:09.984 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:09.984 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.984 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13236 2024-09-14T06:17:09.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.985 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:09.985 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.985 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:09.985 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:09.985 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.985 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.985 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:09.985 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:09.985 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.985 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.985 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:09.986 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:09.986 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.61e-07 2024-09-14T06:17:09.986 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.986 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.986 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:09.986 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099382+0000", 2024-09-14T06:17:09.986 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3589999999999999e-06 2024-09-14T06:17:09.986 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.986 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.986 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:09.986 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099388+0000", 2024-09-14T06:17:09.986 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9619999999999998e-06 2024-09-14T06:17:09.986 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.986 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.987 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.987 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099393+0000", 2024-09-14T06:17:09.987 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3410000000000002e-06 2024-09-14T06:17:09.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.987 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.987 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:09.987 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165553+0000", 2024-09-14T06:17:09.987 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066159604999999996 2024-09-14T06:17:09.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.987 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.987 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:09.987 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165603+0000", 2024-09-14T06:17:09.987 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0427999999999999e-05 2024-09-14T06:17:09.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.988 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.988 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:09.988 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165827+0000", 2024-09-14T06:17:09.988 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022358299999999999 2024-09-14T06:17:09.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.988 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.988 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:09.988 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177005+0000", 2024-09-14T06:17:09.988 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011178044 2024-09-14T06:17:09.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.988 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.988 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:09.989 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181302+0000", 2024-09-14T06:17:09.989 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042970669999999999 2024-09-14T06:17:09.989 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.989 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.989 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:09.989 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181349+0000", 2024-09-14T06:17:09.989 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7287999999999999e-05 2024-09-14T06:17:09.989 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.989 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.989 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:09.989 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181387+0000", 2024-09-14T06:17:09.989 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7601000000000001e-05 2024-09-14T06:17:09.989 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.990 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:09.990 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.990 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.990 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13237 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.990 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:09.990 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.840757687999997, 2024-09-14T06:17:09.990 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081927299999999995, 2024-09-14T06:17:09.990 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:09.990 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:09.990 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:09.990 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:09.990 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:09.991 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.991 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13237 2024-09-14T06:17:09.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.991 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:09.991 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.991 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:09.991 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:09.991 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.991 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.991 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:09.991 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:09.991 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.992 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.992 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:09.992 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099535+0000", 2024-09-14T06:17:09.992 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:17:09.992 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.992 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.992 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:09.992 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099538+0000", 2024-09-14T06:17:09.992 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:17:09.992 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.992 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.992 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:09.992 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099545+0000", 2024-09-14T06:17:09.992 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:17:09.993 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.993 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.993 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.993 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099551+0000", 2024-09-14T06:17:09.993 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1770000000000003e-06 2024-09-14T06:17:09.993 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.993 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.993 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:09.993 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166131+0000", 2024-09-14T06:17:09.993 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066580499000000001 2024-09-14T06:17:09.993 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.993 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.993 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:09.993 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166221+0000", 2024-09-14T06:17:09.994 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0000000000000006e-05 2024-09-14T06:17:09.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.994 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:09.994 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166622+0000", 2024-09-14T06:17:09.994 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040029799999999999 2024-09-14T06:17:09.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.994 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:09.994 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177008+0000", 2024-09-14T06:17:09.994 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010386772000000001 2024-09-14T06:17:09.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:09.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181420+0000", 2024-09-14T06:17:09.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044119190000000003 2024-09-14T06:17:09.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:09.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181443+0000", 2024-09-14T06:17:09.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2603999999999999e-05 2024-09-14T06:17:09.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:09.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181461+0000", 2024-09-14T06:17:09.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8405000000000001e-05 2024-09-14T06:17:09.996 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.996 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:09.996 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:09.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.996 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.996 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13238 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:09.996 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:09.996 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.840721379999998, 2024-09-14T06:17:09.996 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082014927000000001, 2024-09-14T06:17:09.996 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:09.996 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:09.996 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:09.996 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:09.997 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:09.997 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:09.997 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13238 2024-09-14T06:17:09.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.997 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:09.997 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.997 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:09.997 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:09.997 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.997 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.997 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:09.997 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:09.998 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:09.998 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.998 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.998 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:09.998 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099571+0000", 2024-09-14T06:17:09.998 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9699999999999996e-07 2024-09-14T06:17:09.998 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.998 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.998 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:09.998 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099577+0000", 2024-09-14T06:17:09.998 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2890000000000003e-06 2024-09-14T06:17:09.998 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.998 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.998 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:09.999 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099587+0000", 2024-09-14T06:17:09.999 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0002e-05 2024-09-14T06:17:09.999 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.999 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.999 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:09.999 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099593+0000", 2024-09-14T06:17:09.999 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4539999999999998e-06 2024-09-14T06:17:09.999 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:09.999 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:09.999 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:09.999 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167024+0000", 2024-09-14T06:17:09.999 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067431480000000002 2024-09-14T06:17:09.999 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.000 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.000 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167074+0000", 2024-09-14T06:17:10.000 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9996e-05 2024-09-14T06:17:10.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.000 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:10.000 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167288+0000", 2024-09-14T06:17:10.000 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021410900000000001 2024-09-14T06:17:10.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.000 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:10.000 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177997+0000", 2024-09-14T06:17:10.000 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010708756 2024-09-14T06:17:10.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:10.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181508+0000", 2024-09-14T06:17:10.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003511363 2024-09-14T06:17:10.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:10.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181552+0000", 2024-09-14T06:17:10.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3609999999999998e-05 2024-09-14T06:17:10.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181585+0000", 2024-09-14T06:17:10.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3271e-05 2024-09-14T06:17:10.002 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.002 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.002 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.002 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13239 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:10.002 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:10.002 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.840629561999997, 2024-09-14T06:17:10.002 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081998772999999997, 2024-09-14T06:17:10.002 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.002 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.002 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:10.002 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.002 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.002 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.003 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13239 2024-09-14T06:17:10.003 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.003 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.003 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.003 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.003 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:10.003 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.003 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.003 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.003 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.003 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:10.003 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.003 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.003 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.003 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.004 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099663+0000", 2024-09-14T06:17:10.004 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6100000000000001e-07 2024-09-14T06:17:10.004 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.004 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.004 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.004 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099666+0000", 2024-09-14T06:17:10.004 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.056e-06 2024-09-14T06:17:10.004 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.004 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.004 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.004 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099672+0000", 2024-09-14T06:17:10.004 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8120000000000002e-06 2024-09-14T06:17:10.004 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.004 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.004 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.005 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099676+0000", 2024-09-14T06:17:10.005 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:17:10.005 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.005 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.005 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.005 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167602+0000", 2024-09-14T06:17:10.005 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067925867000000001 2024-09-14T06:17:10.005 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.005 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.005 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.005 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167693+0000", 2024-09-14T06:17:10.005 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0736000000000007e-05 2024-09-14T06:17:10.005 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.005 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.006 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:10.006 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168065+0000", 2024-09-14T06:17:10.006 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037245399999999998 2024-09-14T06:17:10.006 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.006 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.006 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:10.006 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178003+0000", 2024-09-14T06:17:10.006 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0099372360000000003 2024-09-14T06:17:10.006 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.006 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.006 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:10.006 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181621+0000", 2024-09-14T06:17:10.006 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036178489999999998 2024-09-14T06:17:10.006 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.006 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.007 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:10.007 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181643+0000", 2024-09-14T06:17:10.007 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2155e-05 2024-09-14T06:17:10.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.007 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.007 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.007 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181661+0000", 2024-09-14T06:17:10.007 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8147000000000001e-05 2024-09-14T06:17:10.007 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.007 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.007 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.007 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.007 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13240 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:10.008 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:10.008 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.840510469000002, 2024-09-14T06:17:10.008 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081942607000000001, 2024-09-14T06:17:10.008 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.008 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.008 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:10.008 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.008 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.008 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.008 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13240 2024-09-14T06:17:10.008 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.008 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.008 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.008 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.009 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:10.009 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.009 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.009 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.009 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:10.009 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.009 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.009 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.009 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099782+0000", 2024-09-14T06:17:10.009 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3900000000000004e-07 2024-09-14T06:17:10.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.009 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.009 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.010 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099787+0000", 2024-09-14T06:17:10.010 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6189999999999999e-06 2024-09-14T06:17:10.010 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.010 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.010 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.010 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099793+0000", 2024-09-14T06:17:10.010 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.998e-06 2024-09-14T06:17:10.010 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.010 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.010 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.010 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099797+0000", 2024-09-14T06:17:10.010 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8449999999999996e-06 2024-09-14T06:17:10.010 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.010 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.010 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.011 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168494+0000", 2024-09-14T06:17:10.011 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.068696500999999993 2024-09-14T06:17:10.011 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.011 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.011 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.011 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168590+0000", 2024-09-14T06:17:10.011 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6733999999999999e-05 2024-09-14T06:17:10.011 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.011 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.011 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:10.011 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168973+0000", 2024-09-14T06:17:10.011 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038229399999999998 2024-09-14T06:17:10.011 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.011 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.011 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:10.012 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178013+0000", 2024-09-14T06:17:10.012 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0090397199999999994 2024-09-14T06:17:10.012 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.012 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.012 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:10.012 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181683+0000", 2024-09-14T06:17:10.012 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036705819999999999 2024-09-14T06:17:10.012 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.012 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.012 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:10.012 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181705+0000", 2024-09-14T06:17:10.012 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1551000000000001e-05 2024-09-14T06:17:10.012 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.013 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.013 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.013 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181724+0000", 2024-09-14T06:17:10.013 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9123999999999999e-05 2024-09-14T06:17:10.013 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.013 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.013 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.013 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.013 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.013 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13243 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:10.013 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:10.013 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.839976739000001, 2024-09-14T06:17:10.013 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081767815999999993, 2024-09-14T06:17:10.013 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.014 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.014 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:10.014 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.014 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.014 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.014 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13243 2024-09-14T06:17:10.014 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.014 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.014 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.014 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.014 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:10.014 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.014 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.014 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.015 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.015 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:10.015 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.015 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.015 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.015 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.015 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:10.015 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1e-07 2024-09-14T06:17:10.015 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.015 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.015 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.015 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100317+0000", 2024-09-14T06:17:10.015 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.092e-06 2024-09-14T06:17:10.015 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.015 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.016 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.016 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100321+0000", 2024-09-14T06:17:10.016 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7809999999999999e-06 2024-09-14T06:17:10.016 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.016 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.016 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.016 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100324+0000", 2024-09-14T06:17:10.016 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.117e-06 2024-09-14T06:17:10.016 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.016 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.016 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.016 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171113+0000", 2024-09-14T06:17:10.016 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.070788816000000004 2024-09-14T06:17:10.017 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.017 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.017 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.017 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171202+0000", 2024-09-14T06:17:10.017 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8843000000000003e-05 2024-09-14T06:17:10.017 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.017 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.017 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:10.017 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171584+0000", 2024-09-14T06:17:10.017 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038231099999999998 2024-09-14T06:17:10.017 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.017 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.017 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:10.017 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178101+0000", 2024-09-14T06:17:10.017 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0065168329999999997 2024-09-14T06:17:10.018 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.018 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.018 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:10.018 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182041+0000", 2024-09-14T06:17:10.018 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039401810000000001 2024-09-14T06:17:10.018 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.018 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.018 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:10.018 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182065+0000", 2024-09-14T06:17:10.018 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3623000000000001e-05 2024-09-14T06:17:10.018 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.018 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.018 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.018 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182083+0000", 2024-09-14T06:17:10.018 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7909000000000001e-05 2024-09-14T06:17:10.019 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.019 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.019 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.019 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.019 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.019 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13244 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:10.019 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:10.019 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.839961918999997, 2024-09-14T06:17:10.019 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081837800000000002, 2024-09-14T06:17:10.019 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.019 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.019 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:10.019 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.019 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.020 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.020 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13244 2024-09-14T06:17:10.020 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.020 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.020 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.020 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.020 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:10.020 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.020 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.020 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.020 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.020 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:10.020 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.021 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.021 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.021 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.021 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:10.021 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:17:10.021 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.021 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.021 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.021 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100337+0000", 2024-09-14T06:17:10.021 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.939e-06 2024-09-14T06:17:10.021 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.021 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.021 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.021 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100341+0000", 2024-09-14T06:17:10.021 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2320000000000003e-06 2024-09-14T06:17:10.022 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.022 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.022 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.022 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100344+0000", 2024-09-14T06:17:10.022 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0759999999999999e-06 2024-09-14T06:17:10.022 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.022 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.022 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.022 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171996+0000", 2024-09-14T06:17:10.022 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.071651919999999994 2024-09-14T06:17:10.022 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.022 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.022 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.022 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172079+0000", 2024-09-14T06:17:10.022 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2579999999999999e-05 2024-09-14T06:17:10.023 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.023 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.023 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:10.023 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172457+0000", 2024-09-14T06:17:10.023 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037785100000000001 2024-09-14T06:17:10.023 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.023 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.023 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:10.023 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178104+0000", 2024-09-14T06:17:10.023 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056471730000000001 2024-09-14T06:17:10.023 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.023 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.023 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:10.023 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182123+0000", 2024-09-14T06:17:10.024 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040190019999999998 2024-09-14T06:17:10.024 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.024 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.024 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:10.024 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182147+0000", 2024-09-14T06:17:10.024 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4241000000000001e-05 2024-09-14T06:17:10.024 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.024 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.024 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.024 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182168+0000", 2024-09-14T06:17:10.024 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0492999999999998e-05 2024-09-14T06:17:10.024 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.024 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.025 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.025 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.025 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.025 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13245 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:10.025 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:10.025 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.839942411999999, 2024-09-14T06:17:10.025 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081893899000000006, 2024-09-14T06:17:10.025 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.025 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.025 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:10.025 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.025 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.025 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.025 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13245 2024-09-14T06:17:10.026 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.026 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.026 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.026 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.026 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:10.026 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.026 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.026 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.026 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.026 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:10.026 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.026 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.026 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.026 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.027 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100350+0000", 2024-09-14T06:17:10.027 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5499999999999999e-07 2024-09-14T06:17:10.027 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.027 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.027 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.027 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100353+0000", 2024-09-14T06:17:10.027 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:17:10.027 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.027 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.027 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.027 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100357+0000", 2024-09-14T06:17:10.027 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8500000000000004e-06 2024-09-14T06:17:10.027 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.027 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.027 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.028 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100360+0000", 2024-09-14T06:17:10.028 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8700000000000001e-06 2024-09-14T06:17:10.028 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.028 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.028 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.028 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172862+0000", 2024-09-14T06:17:10.028 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.072501992000000001 2024-09-14T06:17:10.028 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.028 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.028 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.028 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172950+0000", 2024-09-14T06:17:10.028 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8988999999999998e-05 2024-09-14T06:17:10.028 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.028 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.029 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:10.029 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173324+0000", 2024-09-14T06:17:10.029 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037366199999999997 2024-09-14T06:17:10.029 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.029 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.029 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:10.029 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178107+0000", 2024-09-14T06:17:10.029 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047830520000000003 2024-09-14T06:17:10.029 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.029 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.029 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:10.029 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182202+0000", 2024-09-14T06:17:10.029 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040949280000000003 2024-09-14T06:17:10.030 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.030 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.030 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:10.030 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182226+0000", 2024-09-14T06:17:10.030 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3535000000000001e-05 2024-09-14T06:17:10.030 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.030 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.030 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.030 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182243+0000", 2024-09-14T06:17:10.030 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7445e-05 2024-09-14T06:17:10.030 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.030 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.030 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.031 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.031 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.031 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13246 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:10.031 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:10.031 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.839928090000001, 2024-09-14T06:17:10.031 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081962276000000001, 2024-09-14T06:17:10.031 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.031 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.031 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:10.031 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.031 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.031 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.031 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13246 2024-09-14T06:17:10.032 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.032 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.032 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.032 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.032 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:10.032 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.032 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.032 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.032 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.032 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:10.032 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.032 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.032 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.032 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.032 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:10.033 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0800000000000001e-07 2024-09-14T06:17:10.033 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.033 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.033 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.033 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100367+0000", 2024-09-14T06:17:10.033 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8439999999999998e-06 2024-09-14T06:17:10.033 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.033 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.033 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.033 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100372+0000", 2024-09-14T06:17:10.033 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4009999999999996e-06 2024-09-14T06:17:10.033 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.033 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.033 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.034 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100378+0000", 2024-09-14T06:17:10.034 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7180000000000001e-06 2024-09-14T06:17:10.034 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.034 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.034 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.034 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173780+0000", 2024-09-14T06:17:10.034 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.073401991 2024-09-14T06:17:10.034 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.034 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.034 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.034 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173879+0000", 2024-09-14T06:17:10.034 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9449000000000003e-05 2024-09-14T06:17:10.034 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.034 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.034 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:10.035 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174240+0000", 2024-09-14T06:17:10.035 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036118100000000002 2024-09-14T06:17:10.035 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.035 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.035 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:10.035 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178109+0000", 2024-09-14T06:17:10.035 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038690619999999999 2024-09-14T06:17:10.035 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.035 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.035 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:10.035 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182281+0000", 2024-09-14T06:17:10.035 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041715900000000002 2024-09-14T06:17:10.035 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.035 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.036 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:10.036 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182306+0000", 2024-09-14T06:17:10.036 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4477e-05 2024-09-14T06:17:10.036 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.036 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.036 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.036 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182326+0000", 2024-09-14T06:17:10.036 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0254999999999998e-05 2024-09-14T06:17:10.036 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.036 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.036 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.036 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.036 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.037 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13247 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:10.037 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:10.037 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.839830913, 2024-09-14T06:17:10.037 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966593000000004, 2024-09-14T06:17:10.037 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.037 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.037 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:10.037 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.037 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.037 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.037 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13247 2024-09-14T06:17:10.037 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.037 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.037 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.038 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.038 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:10.038 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.038 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.038 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.038 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.038 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:10.038 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.038 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.038 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.038 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.038 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100462+0000", 2024-09-14T06:17:10.038 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9800000000000002e-07 2024-09-14T06:17:10.039 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.039 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.039 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.039 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100466+0000", 2024-09-14T06:17:10.039 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4039999999999997e-06 2024-09-14T06:17:10.039 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.039 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.039 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.039 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100475+0000", 2024-09-14T06:17:10.039 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3680000000000008e-06 2024-09-14T06:17:10.039 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.039 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.039 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.039 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100482+0000", 2024-09-14T06:17:10.040 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8630000000000002e-06 2024-09-14T06:17:10.040 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.040 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.040 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.040 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174669+0000", 2024-09-14T06:17:10.040 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074186849999999999 2024-09-14T06:17:10.040 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.040 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.040 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.040 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174758+0000", 2024-09-14T06:17:10.040 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9021e-05 2024-09-14T06:17:10.040 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.040 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.041 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:10.041 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175137+0000", 2024-09-14T06:17:10.041 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378511 2024-09-14T06:17:10.041 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.041 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.041 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:10.041 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178158+0000", 2024-09-14T06:17:10.041 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003020956 2024-09-14T06:17:10.041 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.041 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.041 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:10.041 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182386+0000", 2024-09-14T06:17:10.041 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042278840000000003 2024-09-14T06:17:10.042 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.042 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.042 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:10.042 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182410+0000", 2024-09-14T06:17:10.042 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4026e-05 2024-09-14T06:17:10.042 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.042 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.042 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.042 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182427+0000", 2024-09-14T06:17:10.042 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7711999999999999e-05 2024-09-14T06:17:10.042 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.042 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.042 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.042 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.043 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.043 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13248 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:10.043 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:10.043 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.839748831999998, 2024-09-14T06:17:10.043 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966527999999997, 2024-09-14T06:17:10.043 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.043 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.043 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:10.043 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.043 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.043 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.043 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13248 2024-09-14T06:17:10.043 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.043 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.044 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.044 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.044 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:10.044 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.044 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.044 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.044 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.044 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:10.044 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.044 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.044 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.044 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.044 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:10.044 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:17:10.045 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.045 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.045 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.045 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100548+0000", 2024-09-14T06:17:10.045 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:17:10.045 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.045 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.045 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.045 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100557+0000", 2024-09-14T06:17:10.045 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:17:10.045 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.045 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.045 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.045 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100563+0000", 2024-09-14T06:17:10.045 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2519999999999996e-06 2024-09-14T06:17:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175572+0000", 2024-09-14T06:17:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075008459 2024-09-14T06:17:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175656+0000", 2024-09-14T06:17:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3906999999999995e-05 2024-09-14T06:17:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176010+0000", 2024-09-14T06:17:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00035447499999999998 2024-09-14T06:17:10.047 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.047 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.047 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:10.047 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178304+0000", 2024-09-14T06:17:10.047 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0022941379999999998 2024-09-14T06:17:10.047 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.047 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.047 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:10.047 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182467+0000", 2024-09-14T06:17:10.047 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041632020000000004 2024-09-14T06:17:10.047 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.047 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.047 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:10.047 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182491+0000", 2024-09-14T06:17:10.047 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3733000000000001e-05 2024-09-14T06:17:10.048 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.048 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.048 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.048 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182509+0000", 2024-09-14T06:17:10.048 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8113000000000001e-05 2024-09-14T06:17:10.048 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.048 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.048 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.048 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.048 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.048 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13249 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:10.048 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:10.048 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.839713897999999, 2024-09-14T06:17:10.049 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082037225000000005, 2024-09-14T06:17:10.049 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.049 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.049 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:10.049 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.049 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.049 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.049 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13249 2024-09-14T06:17:10.049 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.049 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.049 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.049 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.049 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:10.050 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.050 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.050 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.050 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.050 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:10.050 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.050 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.050 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.050 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.050 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:10.050 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4600000000000005e-07 2024-09-14T06:17:10.050 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.050 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.050 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.051 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100584+0000", 2024-09-14T06:17:10.051 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.947e-06 2024-09-14T06:17:10.051 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.051 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.051 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.051 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100593+0000", 2024-09-14T06:17:10.051 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9760000000000001e-06 2024-09-14T06:17:10.051 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.051 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.051 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.051 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100597+0000", 2024-09-14T06:17:10.051 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9689999999999996e-06 2024-09-14T06:17:10.051 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.051 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.051 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.052 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176461+0000", 2024-09-14T06:17:10.052 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075864132000000001 2024-09-14T06:17:10.052 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.052 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.052 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.052 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176516+0000", 2024-09-14T06:17:10.052 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4416999999999998e-05 2024-09-14T06:17:10.052 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.052 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.052 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:10.052 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176734+0000", 2024-09-14T06:17:10.052 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021859399999999999 2024-09-14T06:17:10.052 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.052 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.052 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:10.053 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178308+0000", 2024-09-14T06:17:10.053 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015733660000000001 2024-09-14T06:17:10.053 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.053 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.053 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:10.053 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182573+0000", 2024-09-14T06:17:10.053 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042649200000000002 2024-09-14T06:17:10.053 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.053 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.053 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:10.053 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182597+0000", 2024-09-14T06:17:10.053 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4355e-05 2024-09-14T06:17:10.053 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.054 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.054 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.054 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182615+0000", 2024-09-14T06:17:10.054 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8003e-05 2024-09-14T06:17:10.054 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.054 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.054 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.054 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.054 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.054 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13250 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:10.054 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:10.054 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.839549292999997, 2024-09-14T06:17:10.054 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081983890000000004, 2024-09-14T06:17:10.054 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.054 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.055 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:10.055 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.055 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.055 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.055 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13250 2024-09-14T06:17:10.055 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.055 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.055 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.055 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.055 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:10.055 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.055 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.055 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.055 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.056 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:10.056 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.056 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.056 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.056 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.056 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100743+0000", 2024-09-14T06:17:10.056 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:17:10.056 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.056 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.056 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.056 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100753+0000", 2024-09-14T06:17:10.056 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9979999999999998e-06 2024-09-14T06:17:10.056 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.056 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.057 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.057 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100772+0000", 2024-09-14T06:17:10.057 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8664e-05 2024-09-14T06:17:10.057 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.057 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.057 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.057 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100779+0000", 2024-09-14T06:17:10.057 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.046e-06 2024-09-14T06:17:10.057 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.057 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.057 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.057 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177049+0000", 2024-09-14T06:17:10.057 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076270819000000004 2024-09-14T06:17:10.058 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.058 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.058 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.058 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177140+0000", 2024-09-14T06:17:10.058 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0746000000000002e-05 2024-09-14T06:17:10.058 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.058 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.058 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:10.058 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177519+0000", 2024-09-14T06:17:10.058 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378656 2024-09-14T06:17:10.058 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.058 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.058 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:10.059 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178349+0000", 2024-09-14T06:17:10.059 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00083061599999999995 2024-09-14T06:17:10.059 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.059 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.059 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:10.059 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182679+0000", 2024-09-14T06:17:10.059 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043298379999999999 2024-09-14T06:17:10.059 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.059 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.059 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:10.059 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182705+0000", 2024-09-14T06:17:10.059 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5602999999999999e-05 2024-09-14T06:17:10.059 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.059 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.060 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.060 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182726+0000", 2024-09-14T06:17:10.060 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1297e-05 2024-09-14T06:17:10.060 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.060 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.060 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.060 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.060 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.060 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:10.062 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:17:10.062 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:10.058+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:10.162 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:10.157+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:10.249 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:10.249 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:10.249 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:10.249 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:10.263 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:10.261+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:10.264 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:17:10.290 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:17:10.321 INFO:teuthology.orchestra.run.smithi039.stdout:3785 punch from 0x338a76f to 0x33940a1, (0x9932 bytes) 2024-09-14T06:17:10.327 INFO:teuthology.orchestra.run.smithi039.stdout:3786 punch from 0x8234260 to 0x82431b5, (0xef55 bytes) 2024-09-14T06:17:10.327 INFO:teuthology.orchestra.run.smithi039.stdout:3787 write 0x22f6421 thru 0x22fcd20 (0x6900 bytes) 2024-09-14T06:17:10.332 INFO:teuthology.orchestra.run.smithi039.stdout:3788 punch from 0x304b0e4 to 0x304b6b8, (0x5d4 bytes) 2024-09-14T06:17:10.332 INFO:teuthology.orchestra.run.smithi039.stdout:3789 write 0x24bd585 thru 0x24cc6b7 (0xf133 bytes) 2024-09-14T06:17:10.337 INFO:teuthology.orchestra.run.smithi039.stdout:3791 write 0x32f80e8 thru 0x3307c18 (0xfb31 bytes) 2024-09-14T06:17:10.349 INFO:teuthology.orchestra.run.smithi039.stdout:3793 write 0x8b82371 thru 0x8b8f74c (0xd3dc bytes) 2024-09-14T06:17:10.354 INFO:teuthology.orchestra.run.smithi039.stdout:3794 punch from 0x3a9f634 to 0x3aa5401, (0x5dcd bytes) 2024-09-14T06:17:10.354 INFO:teuthology.orchestra.run.smithi039.stdout:3795 punch from 0x7fc8ca6 to 0x7fce2ea, (0x5644 bytes) 2024-09-14T06:17:10.360 INFO:teuthology.orchestra.run.smithi039.stdout:3796 trunc from 0x8b8f74d to 0xb7638fc 2024-09-14T06:17:10.363 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:10.362+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:10.366 INFO:teuthology.orchestra.run.smithi039.stdout:3799 write 0x2004d54 thru 0x200b70b (0x69b8 bytes) 2024-09-14T06:17:10.370 INFO:teuthology.orchestra.run.smithi039.stdout:3800 write 0x972acc1 thru 0x972fdaa (0x50ea bytes) 2024-09-14T06:17:10.374 INFO:teuthology.orchestra.run.smithi039.stdout:3801 write 0x9288ae thru 0x92980b (0xf5e bytes) 2024-09-14T06:17:10.379 INFO:teuthology.orchestra.run.smithi039.stdout:3802 read 0x15b2a8c thru 0x15be858 (0xbdcd bytes) 2024-09-14T06:17:10.379 INFO:teuthology.orchestra.run.smithi039.stdout:3803 punch from 0x6016f09 to 0x601d7b8, (0x68af bytes) 2024-09-14T06:17:10.390 INFO:teuthology.orchestra.run.smithi039.stdout:3804 writesame 0xb836a2b thru 0xb837ada data_size 0x18(0x10b0 bytes) 2024-09-14T06:17:10.390 INFO:teuthology.orchestra.run.smithi039.stdout:3806 read 0x7ea0296 thru 0x7eab953 (0xb6be bytes) 2024-09-14T06:17:10.390 INFO:teuthology.orchestra.run.smithi039.stdout:3809 writesame 0x88e53f6 thru 0x88f4ab9 data_size 0x22(0xf6c4 bytes) 2024-09-14T06:17:10.396 INFO:teuthology.orchestra.run.smithi039.stdout:3812 write 0xb1f1de6 thru 0xb1f7210 (0x542b bytes) 2024-09-14T06:17:10.413 INFO:teuthology.orchestra.run.smithi039.stdout:3813 trunc from 0xb837adb to 0x5dd6bcf 2024-09-14T06:17:10.419 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:10.419 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:10.419 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:10.419 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:10.419 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:10.435 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:17:10.450 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:10.450 INFO:teuthology.orchestra.run.smithi039.stdout:3815 punch from 0x4484d89 to 0x4489beb, (0x4e62 bytes) 2024-09-14T06:17:10.450 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:10.450 INFO:teuthology.orchestra.run.smithi039.stdout:3817 read 0x4c28082 thru 0x4c2dfe7 (0x5f66 bytes) 2024-09-14T06:17:10.451 INFO:teuthology.orchestra.run.smithi039.stdout:3819 punch from 0x2c06b25 to 0x2c1234c, (0xb827 bytes) 2024-09-14T06:17:10.457 INFO:teuthology.orchestra.run.smithi039.stdout:3820 trunc from 0x5dd6bcf to 0x9ea4478 2024-09-14T06:17:10.464 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:10.462+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:10.473 INFO:teuthology.orchestra.run.smithi039.stdout:3821 write 0xc70db05 thru 0xc71a500 (0xc9fc bytes) 2024-09-14T06:17:10.478 INFO:teuthology.orchestra.run.smithi039.stdout:3822 read 0x563223a thru 0x563e05c (0xbe23 bytes) 2024-09-14T06:17:10.479 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:10.479 INFO:teuthology.orchestra.run.smithi039.stdout:3825 read 0xa730b68 thru 0xa74070b (0xfba4 bytes) 2024-09-14T06:17:10.479 INFO:teuthology.orchestra.run.smithi039.stdout:3826 read 0x1e74293 thru 0x1e7c4a5 (0x8213 bytes) 2024-09-14T06:17:10.479 INFO:teuthology.orchestra.run.smithi039.stdout:3827 read 0x2a8cc17 thru 0x2a93cde (0x70c8 bytes) 2024-09-14T06:17:10.479 INFO:teuthology.orchestra.run.smithi039.stdout:3828 write 0x24da592 thru 0x24dac57 (0x6c6 bytes) 2024-09-14T06:17:10.480 INFO:teuthology.orchestra.run.smithi039.stdout:3829 write 0x985d587 thru 0x986c32c (0xeda6 bytes) 2024-09-14T06:17:10.487 INFO:teuthology.orchestra.run.smithi039.stdout:3830 read 0x6341c34 thru 0x63494c5 (0x7892 bytes) 2024-09-14T06:17:10.487 INFO:teuthology.orchestra.run.smithi039.stdout:3831 write 0x8735e49 thru 0x873cf38 (0x70f0 bytes) 2024-09-14T06:17:10.490 INFO:teuthology.orchestra.run.smithi039.stdout:3832 read 0x2f7b47f thru 0x2f7c6ea (0x126c bytes) 2024-09-14T06:17:10.491 INFO:teuthology.orchestra.run.smithi039.stdout:3834 read 0x36327e8 thru 0x36370b3 (0x48cc bytes) 2024-09-14T06:17:10.491 INFO:teuthology.orchestra.run.smithi039.stdout:3837 read 0x77ddfab thru 0x77eb922 (0xd978 bytes) 2024-09-14T06:17:10.491 INFO:teuthology.orchestra.run.smithi039.stdout:3838 read 0xaa01b83 thru 0xaa049f8 (0x2e76 bytes) 2024-09-14T06:17:10.499 INFO:teuthology.orchestra.run.smithi039.stdout:3840 trunc from 0xc71a501 to 0x6c2b2ad 2024-09-14T06:17:10.511 INFO:teuthology.orchestra.run.smithi039.stdout:3842 write 0xc2eb05 thru 0xc36048 (0x7544 bytes) 2024-09-14T06:17:10.529 INFO:teuthology.orchestra.run.smithi039.stdout:3843 write 0xb0c144e thru 0xb0cf76b (0xe31e bytes) 2024-09-14T06:17:10.533 INFO:teuthology.orchestra.run.smithi039.stdout:3845 read 0x930dc21 thru 0x930ecae (0x108e bytes) 2024-09-14T06:17:10.533 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:10.533 INFO:teuthology.orchestra.run.smithi039.stdout:3848 read 0x6562df6 thru 0x6569c79 (0x6e84 bytes) 2024-09-14T06:17:10.534 INFO:teuthology.orchestra.run.smithi039.stdout:3849 writesame 0x2f073da thru 0x2f14fd3 data_size 0x25(0xdbfa bytes) 2024-09-14T06:17:10.541 INFO:teuthology.orchestra.run.smithi039.stdout:3850 trunc from 0xb0cf76c to 0x92e3177 2024-09-14T06:17:10.555 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:30547 3.1f 3:f84dfb53:::rbd_data.11529eb229bd.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write 16226533~50642] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T06:17:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 11.363674704999999, 2024-09-14T06:17:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0079770020000001, 2024-09-14T06:17:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 30547 2024-09-14T06:17:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189811+0000", 2024-09-14T06:17:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2060000000000001e-06 2024-09-14T06:17:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189871+0000", 2024-09-14T06:17:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9862999999999999e-05 2024-09-14T06:17:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189899+0000", 2024-09-14T06:17:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7849000000000001e-05 2024-09-14T06:17:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189915+0000", 2024-09-14T06:17:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:17:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190024+0000", 2024-09-14T06:17:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000109202 2024-09-14T06:17:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:17:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190041+0000", 2024-09-14T06:17:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7665000000000001e-05 2024-09-14T06:17:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188512+0000", 2024-09-14T06:17:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.99847100700000002 2024-09-14T06:17:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for missing object", 2024-09-14T06:17:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188646+0000", 2024-09-14T06:17:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000133986 2024-09-14T06:17:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193097+0000", 2024-09-14T06:17:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044507059999999996 2024-09-14T06:17:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193183+0000", 2024-09-14T06:17:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5496999999999998e-05 2024-09-14T06:17:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:17:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193307+0000", 2024-09-14T06:17:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000124611 2024-09-14T06:17:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.194318+0000", 2024-09-14T06:17:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0010106 2024-09-14T06:17:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197670+0000", 2024-09-14T06:17:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033523170000000001 2024-09-14T06:17:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197712+0000", 2024-09-14T06:17:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1681000000000003e-05 2024-09-14T06:17:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197786+0000", 2024-09-14T06:17:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4148999999999998e-05 2024-09-14T06:17:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32022 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 6.6852944939999999, 2024-09-14T06:17:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095598104000000003, 2024-09-14T06:17:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32022 2024-09-14T06:17:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868190+0000", 2024-09-14T06:17:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6000000000000003e-07 2024-09-14T06:17:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868243+0000", 2024-09-14T06:17:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3443999999999999e-05 2024-09-14T06:17:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868260+0000", 2024-09-14T06:17:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6799e-05 2024-09-14T06:17:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868271+0000", 2024-09-14T06:17:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0523000000000001e-05 2024-09-14T06:17:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963502+0000", 2024-09-14T06:17:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095231466000000001 2024-09-14T06:17:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963636+0000", 2024-09-14T06:17:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013388699999999999 2024-09-14T06:17:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963787+0000", 2024-09-14T06:17:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000151225 2024-09-14T06:17:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32023 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5816320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 6.6851993680000001, 2024-09-14T06:17:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095879753999999998, 2024-09-14T06:17:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32023 2024-09-14T06:17:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868285+0000", 2024-09-14T06:17:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6299999999999993e-07 2024-09-14T06:17:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868288+0000", 2024-09-14T06:17:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.18e-06 2024-09-14T06:17:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868298+0000", 2024-09-14T06:17:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0029e-05 2024-09-14T06:17:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868307+0000", 2024-09-14T06:17:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7229999999999993e-06 2024-09-14T06:17:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963871+0000", 2024-09-14T06:17:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095563876000000006 2024-09-14T06:17:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964015+0000", 2024-09-14T06:17:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143573 2024-09-14T06:17:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964164+0000", 2024-09-14T06:17:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014940999999999999 2024-09-14T06:17:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32024 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 6.6851662239999996, 2024-09-14T06:17:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096230231999999999, 2024-09-14T06:17:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32024 2024-09-14T06:17:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868318+0000", 2024-09-14T06:17:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1699999999999997e-07 2024-09-14T06:17:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868321+0000", 2024-09-14T06:17:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6769999999999999e-06 2024-09-14T06:17:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868334+0000", 2024-09-14T06:17:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2928999999999999e-05 2024-09-14T06:17:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868341+0000", 2024-09-14T06:17:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9800000000000001e-06 2024-09-14T06:17:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964258+0000", 2024-09-14T06:17:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095917099000000006 2024-09-14T06:17:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964392+0000", 2024-09-14T06:17:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013396999999999999 2024-09-14T06:17:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964548+0000", 2024-09-14T06:17:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015585999999999999 2024-09-14T06:17:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32025 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 6.6851335680000004, 2024-09-14T06:17:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096576591000000003, 2024-09-14T06:17:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32025 2024-09-14T06:17:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868351+0000", 2024-09-14T06:17:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1600000000000001e-07 2024-09-14T06:17:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868353+0000", 2024-09-14T06:17:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.627e-06 2024-09-14T06:17:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868364+0000", 2024-09-14T06:17:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0893000000000001e-05 2024-09-14T06:17:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868381+0000", 2024-09-14T06:17:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6769000000000002e-05 2024-09-14T06:17:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964632+0000", 2024-09-14T06:17:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096250589999999997 2024-09-14T06:17:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964772+0000", 2024-09-14T06:17:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014045299999999999 2024-09-14T06:17:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964927+0000", 2024-09-14T06:17:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015454299999999999 2024-09-14T06:17:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32026 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 6.6850926170000005, 2024-09-14T06:17:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096901622000000007, 2024-09-14T06:17:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32026 2024-09-14T06:17:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868392+0000", 2024-09-14T06:17:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0699999999999996e-07 2024-09-14T06:17:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868395+0000", 2024-09-14T06:17:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9859999999999999e-06 2024-09-14T06:17:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868405+0000", 2024-09-14T06:17:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9229999999999997e-06 2024-09-14T06:17:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868412+0000", 2024-09-14T06:17:10.586 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9670000000000004e-06 2024-09-14T06:17:10.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.586 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.586 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.586 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965010+0000", 2024-09-14T06:17:10.586 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096598562999999998 2024-09-14T06:17:10.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.586 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.586 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.586 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965143+0000", 2024-09-14T06:17:10.586 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013247700000000001 2024-09-14T06:17:10.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.586 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.587 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.587 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965293+0000", 2024-09-14T06:17:10.587 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000149899 2024-09-14T06:17:10.587 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.587 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.587 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.587 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.587 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.587 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32027 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:10.587 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:10.587 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 6.6850242800000004, 2024-09-14T06:17:10.587 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097211306999999997, 2024-09-14T06:17:10.587 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.587 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32027 2024-09-14T06:17:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868461+0000", 2024-09-14T06:17:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.296e-06 2024-09-14T06:17:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868465+0000", 2024-09-14T06:17:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9570000000000004e-06 2024-09-14T06:17:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868476+0000", 2024-09-14T06:17:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1365999999999999e-05 2024-09-14T06:17:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868483+0000", 2024-09-14T06:17:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2529999999999998e-06 2024-09-14T06:17:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965375+0000", 2024-09-14T06:17:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096891973000000006 2024-09-14T06:17:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965518+0000", 2024-09-14T06:17:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143017 2024-09-14T06:17:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965671+0000", 2024-09-14T06:17:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015244499999999999 2024-09-14T06:17:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32028 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 6.6849894519999999, 2024-09-14T06:17:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097841480999999994, 2024-09-14T06:17:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32028 2024-09-14T06:17:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868495+0000", 2024-09-14T06:17:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6400000000000001e-07 2024-09-14T06:17:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868498+0000", 2024-09-14T06:17:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7760000000000002e-06 2024-09-14T06:17:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868507+0000", 2024-09-14T06:17:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1530000000000003e-06 2024-09-14T06:17:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868516+0000", 2024-09-14T06:17:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8929999999999994e-06 2024-09-14T06:17:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965753+0000", 2024-09-14T06:17:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097236970000000006 2024-09-14T06:17:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965890+0000", 2024-09-14T06:17:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013677599999999999 2024-09-14T06:17:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966336+0000", 2024-09-14T06:17:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00044614900000000001 2024-09-14T06:17:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32029 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 6.6849583770000001, 2024-09-14T06:17:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098577779000000004, 2024-09-14T06:17:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32029 2024-09-14T06:17:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868526+0000", 2024-09-14T06:17:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9300000000000006e-07 2024-09-14T06:17:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868529+0000", 2024-09-14T06:17:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5969999999999999e-06 2024-09-14T06:17:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868538+0000", 2024-09-14T06:17:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1700000000000003e-06 2024-09-14T06:17:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868560+0000", 2024-09-14T06:17:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1596000000000001e-05 2024-09-14T06:17:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966423+0000", 2024-09-14T06:17:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097863396000000005 2024-09-14T06:17:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966578+0000", 2024-09-14T06:17:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000154792 2024-09-14T06:17:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967103+0000", 2024-09-14T06:17:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00052523499999999998 2024-09-14T06:17:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32030 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 6.6849134589999997, 2024-09-14T06:17:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099324030999999993, 2024-09-14T06:17:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32030 2024-09-14T06:17:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868571+0000", 2024-09-14T06:17:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9699999999999995e-07 2024-09-14T06:17:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868574+0000", 2024-09-14T06:17:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9529999999999999e-06 2024-09-14T06:17:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868584+0000", 2024-09-14T06:17:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7860000000000008e-06 2024-09-14T06:17:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868591+0000", 2024-09-14T06:17:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0500000000000003e-06 2024-09-14T06:17:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967191+0000", 2024-09-14T06:17:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098600035000000003 2024-09-14T06:17:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967333+0000", 2024-09-14T06:17:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014190599999999999 2024-09-14T06:17:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967894+0000", 2024-09-14T06:17:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000561504 2024-09-14T06:17:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32031 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 6.6848838969999997, 2024-09-14T06:17:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.10010279, 2024-09-14T06:17:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32031 2024-09-14T06:17:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:17:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868603+0000", 2024-09-14T06:17:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.599e-06 2024-09-14T06:17:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868612+0000", 2024-09-14T06:17:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.986e-06 2024-09-14T06:17:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868619+0000", 2024-09-14T06:17:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8900000000000001e-06 2024-09-14T06:17:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967983+0000", 2024-09-14T06:17:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099363612000000004 2024-09-14T06:17:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968140+0000", 2024-09-14T06:17:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015788600000000001 2024-09-14T06:17:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968703+0000", 2024-09-14T06:17:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00056208300000000001 2024-09-14T06:17:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32032 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6258688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 6.684855422, 2024-09-14T06:17:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100488088, 2024-09-14T06:17:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32032 2024-09-14T06:17:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868629+0000", 2024-09-14T06:17:10.614 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:17:10.614 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.614 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.614 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.614 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868631+0000", 2024-09-14T06:17:10.614 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4099999999999998e-06 2024-09-14T06:17:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868640+0000", 2024-09-14T06:17:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0939999999999997e-06 2024-09-14T06:17:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868647+0000", 2024-09-14T06:17:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9460000000000001e-06 2024-09-14T06:17:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968797+0000", 2024-09-14T06:17:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100149867 2024-09-14T06:17:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968937+0000", 2024-09-14T06:17:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013981399999999999 2024-09-14T06:17:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.618 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.618 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969116+0000", 2024-09-14T06:17:10.618 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017927399999999999 2024-09-14T06:17:10.618 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.618 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.618 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.618 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.619 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.619 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32044 3.1e 3:7e6619d4:::rbd_data.115274ed0aa8.0000000000000068:head [set-alloc-hint object_size 1048576 write_size 1048576,write 835584~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:10.619 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:10.619 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 6.6771663390000002, 2024-09-14T06:17:10.619 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096039367, 2024-09-14T06:17:10.619 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.619 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.619 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:10.619 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.619 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.619 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.620 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32044 2024-09-14T06:17:10.620 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.620 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.620 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.620 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.620 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:10.620 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.620 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.620 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.620 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.620 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:10.620 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.620 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.620 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.621 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.621 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876320+0000", 2024-09-14T06:17:10.621 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8270000000000002e-06 2024-09-14T06:17:10.621 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.621 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.621 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.621 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876397+0000", 2024-09-14T06:17:10.621 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6985999999999998e-05 2024-09-14T06:17:10.621 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.621 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.621 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.621 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876431+0000", 2024-09-14T06:17:10.621 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3719e-05 2024-09-14T06:17:10.622 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.622 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.622 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.622 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876452+0000", 2024-09-14T06:17:10.622 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0891999999999999e-05 2024-09-14T06:17:10.622 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.622 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.622 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.622 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969186+0000", 2024-09-14T06:17:10.622 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.092734681999999999 2024-09-14T06:17:10.622 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.622 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.622 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.622 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969550+0000", 2024-09-14T06:17:10.623 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036402199999999998 2024-09-14T06:17:10.623 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.623 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.623 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:17:10.623 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969987+0000", 2024-09-14T06:17:10.623 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000436759 2024-09-14T06:17:10.624 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.624 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.624 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:10.624 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971569+0000", 2024-09-14T06:17:10.624 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001581821 2024-09-14T06:17:10.624 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.625 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.625 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:10.625 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971947+0000", 2024-09-14T06:17:10.625 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378446 2024-09-14T06:17:10.625 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.625 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.625 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:10.626 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972146+0000", 2024-09-14T06:17:10.626 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019819 2024-09-14T06:17:10.626 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.626 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.626 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.626 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972357+0000", 2024-09-14T06:17:10.626 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021102299999999999 2024-09-14T06:17:10.626 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.626 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.626 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.626 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.626 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.626 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40252 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 827392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:10.627 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:10.627 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.83653023299999996, 2024-09-14T06:17:10.627 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095633474999999996, 2024-09-14T06:17:10.627 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.627 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.627 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.627 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.627 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.627 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.627 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40252 2024-09-14T06:17:10.627 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.627 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.627 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.627 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.627 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:10.628 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.628 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.628 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.628 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.628 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:10.628 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.628 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.628 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.628 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.628 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717376+0000", 2024-09-14T06:17:10.628 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000422216 2024-09-14T06:17:10.628 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.628 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.629 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.629 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717386+0000", 2024-09-14T06:17:10.629 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0396e-05 2024-09-14T06:17:10.629 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.629 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.629 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.629 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717395+0000", 2024-09-14T06:17:10.629 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7819999999999999e-06 2024-09-14T06:17:10.629 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.629 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.629 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.629 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717400+0000", 2024-09-14T06:17:10.630 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1599999999999997e-06 2024-09-14T06:17:10.630 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.630 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.630 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.630 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.811922+0000", 2024-09-14T06:17:10.630 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.094522018999999999 2024-09-14T06:17:10.630 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.630 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.630 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.630 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812007+0000", 2024-09-14T06:17:10.631 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5328999999999996e-05 2024-09-14T06:17:10.631 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.631 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.631 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.631 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812587+0000", 2024-09-14T06:17:10.631 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000579573 2024-09-14T06:17:10.631 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.632 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.632 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.632 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.632 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.632 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40254 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:10.632 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:10.632 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.83607206000000001, 2024-09-14T06:17:10.632 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095562731999999997, 2024-09-14T06:17:10.632 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.633 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.633 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.633 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.633 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.633 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.633 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40254 2024-09-14T06:17:10.633 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.633 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.633 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.633 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.633 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:10.633 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.634 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.634 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.634 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.634 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:10.634 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.634 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.634 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.634 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.634 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:10.634 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1699999999999999e-07 2024-09-14T06:17:10.634 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.634 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.634 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.635 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717414+0000", 2024-09-14T06:17:10.635 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:17:10.635 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.635 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.635 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.635 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717419+0000", 2024-09-14T06:17:10.635 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2839999999999997e-06 2024-09-14T06:17:10.635 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.635 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.635 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.635 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717424+0000", 2024-09-14T06:17:10.635 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1549999999999998e-06 2024-09-14T06:17:10.636 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.636 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.636 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.636 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812619+0000", 2024-09-14T06:17:10.636 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095194940000000006 2024-09-14T06:17:10.636 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.636 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.636 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.636 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812717+0000", 2024-09-14T06:17:10.636 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7647999999999997e-05 2024-09-14T06:17:10.636 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.636 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.636 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.637 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812974+0000", 2024-09-14T06:17:10.637 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025752099999999998 2024-09-14T06:17:10.637 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.637 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.637 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.637 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.637 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.637 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40256 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 843776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:10.637 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:10.637 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.83604945600000002, 2024-09-14T06:17:10.637 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095932066999999996, 2024-09-14T06:17:10.637 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.637 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.638 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.638 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.638 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.638 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.638 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40256 2024-09-14T06:17:10.638 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.638 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.638 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.638 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.638 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:10.638 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.638 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.638 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.638 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.639 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:10.639 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.639 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.639 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.639 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.639 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717628+0000", 2024-09-14T06:17:10.639 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019339499999999999 2024-09-14T06:17:10.639 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.639 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.639 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.639 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717632+0000", 2024-09-14T06:17:10.639 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2230000000000001e-06 2024-09-14T06:17:10.639 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.639 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.640 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.640 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717641+0000", 2024-09-14T06:17:10.640 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6309999999999998e-06 2024-09-14T06:17:10.640 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.640 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.640 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.640 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717656+0000", 2024-09-14T06:17:10.640 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4724e-05 2024-09-14T06:17:10.640 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.640 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.640 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.640 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813006+0000", 2024-09-14T06:17:10.641 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095349801999999997 2024-09-14T06:17:10.641 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.641 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.641 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.641 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813107+0000", 2024-09-14T06:17:10.641 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000100973 2024-09-14T06:17:10.641 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.641 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.641 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.641 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813366+0000", 2024-09-14T06:17:10.641 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025931899999999999 2024-09-14T06:17:10.641 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.641 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.641 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.642 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.642 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.642 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40258 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 851968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:10.642 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:10.642 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.83581335599999995, 2024-09-14T06:17:10.642 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096062543, 2024-09-14T06:17:10.642 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.642 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.642 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.642 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.642 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.642 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.642 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40258 2024-09-14T06:17:10.642 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.643 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.643 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.643 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.643 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:10.643 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.643 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.643 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.643 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.643 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:10.643 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.643 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.643 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.643 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.643 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717671+0000", 2024-09-14T06:17:10.643 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8299999999999997e-07 2024-09-14T06:17:10.644 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.644 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.644 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.644 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717674+0000", 2024-09-14T06:17:10.644 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7080000000000002e-06 2024-09-14T06:17:10.644 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.644 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.644 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.644 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717681+0000", 2024-09-14T06:17:10.644 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7889999999999995e-06 2024-09-14T06:17:10.644 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.644 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.644 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.644 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717687+0000", 2024-09-14T06:17:10.645 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7529999999999998e-06 2024-09-14T06:17:10.645 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.645 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.645 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.645 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813408+0000", 2024-09-14T06:17:10.645 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095720451999999998 2024-09-14T06:17:10.645 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.645 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.645 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.645 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813499+0000", 2024-09-14T06:17:10.645 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0933000000000005e-05 2024-09-14T06:17:10.645 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.645 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.645 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.646 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813733+0000", 2024-09-14T06:17:10.646 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00023432499999999999 2024-09-14T06:17:10.646 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.646 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.646 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.646 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.646 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.646 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40260 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 860160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:10.646 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:10.646 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.83578750700000004, 2024-09-14T06:17:10.646 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096337381999999999, 2024-09-14T06:17:10.646 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.646 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.646 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.646 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.646 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.647 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.647 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40260 2024-09-14T06:17:10.647 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.647 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.647 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.647 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.647 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:10.647 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.647 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.647 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.647 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.647 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:10.647 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.647 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.647 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.648 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.648 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718008+0000", 2024-09-14T06:17:10.648 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00031192699999999998 2024-09-14T06:17:10.648 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.648 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.648 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.648 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718013+0000", 2024-09-14T06:17:10.648 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1830000000000002e-06 2024-09-14T06:17:10.648 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.648 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.648 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.648 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718036+0000", 2024-09-14T06:17:10.648 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2721000000000002e-05 2024-09-14T06:17:10.648 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.648 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.649 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.649 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718053+0000", 2024-09-14T06:17:10.649 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6736e-05 2024-09-14T06:17:10.649 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.649 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.649 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.649 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813767+0000", 2024-09-14T06:17:10.649 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095714658999999994 2024-09-14T06:17:10.649 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.649 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.649 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.649 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813816+0000", 2024-09-14T06:17:10.649 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8848999999999999e-05 2024-09-14T06:17:10.649 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.650 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.650 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.650 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814034+0000", 2024-09-14T06:17:10.650 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000217307 2024-09-14T06:17:10.650 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.650 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.650 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.650 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.650 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.650 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40262 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 868352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:10.650 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:10.650 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.835387824, 2024-09-14T06:17:10.650 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096354165000000006, 2024-09-14T06:17:10.650 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.651 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.651 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.651 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.651 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.651 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.651 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40262 2024-09-14T06:17:10.651 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.651 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.651 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.651 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.651 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:10.651 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.651 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.651 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.651 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.652 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:10.652 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.652 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.652 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.652 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.652 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718281+0000", 2024-09-14T06:17:10.652 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000185069 2024-09-14T06:17:10.652 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.652 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.652 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.652 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718286+0000", 2024-09-14T06:17:10.653 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.118e-06 2024-09-14T06:17:10.653 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.653 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.653 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.653 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718295+0000", 2024-09-14T06:17:10.653 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0429999999999996e-06 2024-09-14T06:17:10.653 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.653 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.653 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.653 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718302+0000", 2024-09-14T06:17:10.653 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2790000000000001e-06 2024-09-14T06:17:10.653 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.653 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.653 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.654 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814076+0000", 2024-09-14T06:17:10.654 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095774076999999999 2024-09-14T06:17:10.654 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.654 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.654 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.654 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814191+0000", 2024-09-14T06:17:10.654 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00011500700000000001 2024-09-14T06:17:10.654 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.654 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.654 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.654 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814450+0000", 2024-09-14T06:17:10.655 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025857199999999999 2024-09-14T06:17:10.655 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.655 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.655 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.655 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.655 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.655 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40264 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 876544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:10.655 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:10.655 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.83503075599999999, 2024-09-14T06:17:10.655 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096372478999999997, 2024-09-14T06:17:10.655 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:10.655 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:10.656 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:10.656 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:10.656 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:10.656 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:10.656 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40264 2024-09-14T06:17:10.656 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.656 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:10.656 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.656 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:10.656 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:10.656 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.656 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.656 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.657 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:10.657 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:10.657 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:10.657 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.657 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.657 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:10.657 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718793+0000", 2024-09-14T06:17:10.657 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00033991099999999998 2024-09-14T06:17:10.657 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.657 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.657 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:10.657 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718805+0000", 2024-09-14T06:17:10.657 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2514000000000001e-05 2024-09-14T06:17:10.658 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.658 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.658 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:10.658 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718815+0000", 2024-09-14T06:17:10.658 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9590000000000007e-06 2024-09-14T06:17:10.658 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.658 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.658 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:10.658 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718822+0000", 2024-09-14T06:17:10.658 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2030000000000003e-06 2024-09-14T06:17:10.659 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.659 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.659 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:10.659 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814490+0000", 2024-09-14T06:17:10.659 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095667346 2024-09-14T06:17:10.659 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.659 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.659 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:10.659 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814577+0000", 2024-09-14T06:17:10.659 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6920000000000001e-05 2024-09-14T06:17:10.659 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:10.659 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:10.659 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:10.659 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814825+0000", 2024-09-14T06:17:10.660 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000248626 2024-09-14T06:17:10.660 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.660 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.660 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.660 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:10.660 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:10.660 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:10.661 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:17:10.663 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:17:10.663 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:10.657+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:10.665 INFO:teuthology.orchestra.run.smithi039.stdout:3852 writesame 0xb79e719 thru 0xb7a30d1 data_size 0x1b(0x49b9 bytes) 2024-09-14T06:17:10.665 INFO:teuthology.orchestra.run.smithi039.stdout:3854 write 0xeac491d thru 0xead0c30 (0xc314 bytes) 2024-09-14T06:17:10.665 INFO:teuthology.orchestra.run.smithi039.stdout:3855 punch from 0x173a915 to 0x174380b, (0x8ef6 bytes) 2024-09-14T06:17:10.665 INFO:teuthology.orchestra.run.smithi039.stdout:3857 writesame 0x8d2a3b0 thru 0x8d3538d data_size 0x1(0xafde bytes) 2024-09-14T06:17:10.665 INFO:teuthology.orchestra.run.smithi039.stdout:3859 read 0x9c13ace thru 0x9c1b42e (0x7961 bytes) 2024-09-14T06:17:10.665 INFO:teuthology.orchestra.run.smithi039.stdout:3860 read 0xb796dd thru 0xb88931 (0xf255 bytes) 2024-09-14T06:17:10.665 INFO:teuthology.orchestra.run.smithi039.stdout:3861 write 0xc92b769 thru 0xc9313ac (0x5c44 bytes) 2024-09-14T06:17:10.665 INFO:teuthology.orchestra.run.smithi039.stdout:3865 punch from 0x2a4a49f to 0x2a59bed, (0xf74e bytes) 2024-09-14T06:17:10.665 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:10.665 INFO:teuthology.orchestra.run.smithi039.stdout:3868 read 0x964351 thru 0x969bc3 (0x5873 bytes) 2024-09-14T06:17:10.666 INFO:teuthology.orchestra.run.smithi039.stdout:3869 write 0x4131a16 thru 0x4136168 (0x4753 bytes) 2024-09-14T06:17:10.666 INFO:teuthology.orchestra.run.smithi039.stdout:3871 punch from 0x223798d to 0x2242b06, (0xb179 bytes) 2024-09-14T06:17:10.666 INFO:teuthology.orchestra.run.smithi039.stdout:3872 write 0x2a912f9 thru 0x2a9b8ce (0xa5d6 bytes) 2024-09-14T06:17:10.666 INFO:teuthology.orchestra.run.smithi039.stdout:3873 punch from 0xb7c6ca7 to 0xb7c7775, (0xace bytes) 2024-09-14T06:17:10.666 INFO:teuthology.orchestra.run.smithi039.stdout:3874 read 0xc3f0352 thru 0xc3fd800 (0xd4af bytes) 2024-09-14T06:17:10.666 INFO:teuthology.orchestra.run.smithi039.stdout:3877 read 0x2e24488 thru 0x2e29273 (0x4dec bytes) 2024-09-14T06:17:10.666 INFO:teuthology.orchestra.run.smithi039.stdout:3878 writesame 0x8f7c3ed thru 0x8f84514 data_size 0x4(0x8128 bytes) 2024-09-14T06:17:10.666 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:10.667 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:10.667 INFO:teuthology.orchestra.run.smithi039.stdout:3881 trunc from 0xead0c31 to 0x8796e49 2024-09-14T06:17:10.667 INFO:teuthology.orchestra.run.smithi039.stdout:3882 punch from 0x592a5f0 to 0x59333cf, (0x8ddf bytes) 2024-09-14T06:17:10.667 INFO:teuthology.orchestra.run.smithi039.stdout:3885 read 0x9d0b5b thru 0x9da25f (0x9705 bytes) 2024-09-14T06:17:10.667 INFO:teuthology.orchestra.run.smithi039.stdout:3886 read 0x1de320c thru 0x1df2004 (0xedf9 bytes) 2024-09-14T06:17:10.667 INFO:teuthology.orchestra.run.smithi039.stdout:3887 clone 21 order 23 su 262144 sc 12 2024-09-14T06:17:10.762 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:10.758+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:10.815 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:10.815 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:10.825 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:10.825 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:10.825 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:10.825 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:10.832 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:10.832 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:10.837 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:17:10.848 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:10.849 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:10.863 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:10.858+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:10.866 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:10.867 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:10.884 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:10.884 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:10.903 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:10.904 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:10.921 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:10.922 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:10.940 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:10.941 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:10.959 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:10.959 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:10.964 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:10.961+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:10.972 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:17:11.065 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:11.062+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:11.130 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:11.130 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:11.144 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:17:11.165 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:11.162+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:11.257 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:11.257 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:11.266 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:11.261+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:11.270 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:17:11.273 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:17:11.273 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:17:11.273 INFO:tasks.thrashosds.thrasher:Reweighting osd 3 to 0.1128750285712298 2024-09-14T06:17:11.274 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 0.1128750285712298 2024-09-14T06:17:11.319 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-09-14T06:17:11.366 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:11.365+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:11.393 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone20 from 0x8796e49 (overlap 0x5dd6bcf) to 0x962461 2024-09-14T06:17:11.461 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:11.461 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:11.467 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:11.465+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:11.478 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:17:11.537 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-14T06:17:11.568 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:11.565+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:11.644 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:11.644 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:11.659 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:17:11.703 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:11.702+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:11.770 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:11.765+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:11.784 INFO:teuthology.orchestra.run.smithi039.stdout:3888 writesame 0x835b6fc thru 0x835ec10 data_size 0x1(0x3515 bytes) 2024-09-14T06:17:11.785 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:11.785 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:11.801 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:17:11.803 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:11.803 INFO:teuthology.orchestra.run.smithi039.stdout:3892 read 0x9b82af thru 0x9c6f0a (0xec5c bytes) 2024-09-14T06:17:11.806 INFO:teuthology.orchestra.run.smithi039.stdout:3893 read 0x82c5d7b thru 0x82d5ade (0xfd64 bytes) 2024-09-14T06:17:11.806 INFO:teuthology.orchestra.run.smithi039.stdout:3894 read 0x8208ad2 thru 0x820dcf5 (0x5224 bytes) 2024-09-14T06:17:11.814 INFO:teuthology.orchestra.run.smithi039.stdout:3895 trunc from 0x8796e49 to 0xd61a89c 2024-09-14T06:17:11.886 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:11.870+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:11.917 INFO:teuthology.orchestra.run.smithi039.stdout:3896 trunc from 0xd61a89c to 0x3741ffe 2024-09-14T06:17:11.928 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:11.928 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:11.957 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:17:12.000 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:12.000 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:12.000 INFO:teuthology.orchestra.run.smithi039.stdout:3899 read 0x2488c8c thru 0x248ef39 (0x62ae bytes) 2024-09-14T06:17:12.000 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:11.998+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:12.002 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:12.002 INFO:teuthology.orchestra.run.smithi039.stdout:3901 punch from 0xf2143e to 0xf22653, (0x1215 bytes) 2024-09-14T06:17:12.002 INFO:teuthology.orchestra.run.smithi039.stdout:3902 clone 22 order 19 su 131072 sc 11 2024-09-14T06:17:12.073 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:12.070+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:12.123 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:12.123 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:12.138 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:17:12.173 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:12.169+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:12.274 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:12.270+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:12.308 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.3 to 0.112875 (1ce5) 2024-09-14T06:17:12.311 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:12.311 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:12.335 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:17:12.375 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:12.373+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:12.476 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:12.474+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:12.508 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:12.508 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:12.523 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:17:12.611 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:12.610+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:12.678 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:12.677+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:12.696 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:12.696 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:12.710 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:17:12.795 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:12.794+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:12.880 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:12.877+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:12.881 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:12.881 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:12.896 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:17:12.958 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:17:12.983 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:12.982+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:13.067 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:13.067 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:13.083 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:17:13.083 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:13.078+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:13.133 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:13.133 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:13.150 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:13.150 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:13.167 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:13.167 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:13.183 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:13.181+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:13.184 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:13.184 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:13.204 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:13.205 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:13.225 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:13.226 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:13.246 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:13.246 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:13.246 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:13.246 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:13.246 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.246 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3916 3.16 3:6ab8b1d8:::rbd_data.1152bd92e04f.0000000000000002:head [stat,truncate 1590169] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:17:13.246 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:17:13.246 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 45.665077893000003, 2024-09-14T06:17:13.246 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022958078999999999, 2024-09-14T06:17:13.246 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.246 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.246 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.247 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.247 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.247 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.247 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3916 2024-09-14T06:17:13.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.247 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.247 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.247 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.247 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:17:13.247 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.247 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.247 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.248 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:17:13.248 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.248 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.248 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579203+0000", 2024-09-14T06:17:13.248 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.748e-06 2024-09-14T06:17:13.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.248 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.248 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579208+0000", 2024-09-14T06:17:13.248 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:17:13.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.249 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.249 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579223+0000", 2024-09-14T06:17:13.249 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.508e-05 2024-09-14T06:17:13.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.249 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.249 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.249 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579232+0000", 2024-09-14T06:17:13.249 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9719999999999998e-06 2024-09-14T06:17:13.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.249 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.249 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.249 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579307+0000", 2024-09-14T06:17:13.249 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5028000000000001e-05 2024-09-14T06:17:13.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.250 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.250 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.250 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579412+0000", 2024-09-14T06:17:13.250 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010525400000000001 2024-09-14T06:17:13.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.250 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.250 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:17:13.250 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579535+0000", 2024-09-14T06:17:13.250 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122726 2024-09-14T06:17:13.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.250 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.250 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.250 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.581415+0000", 2024-09-14T06:17:13.250 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018800360000000001 2024-09-14T06:17:13.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.251 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.251 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.251 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602017+0000", 2024-09-14T06:17:13.251 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.020602425000000001 2024-09-14T06:17:13.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.251 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.251 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.251 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602096+0000", 2024-09-14T06:17:13.251 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8968999999999996e-05 2024-09-14T06:17:13.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.251 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.251 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.251 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602159+0000", 2024-09-14T06:17:13.251 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2824999999999996e-05 2024-09-14T06:17:13.251 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.252 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.252 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.252 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.252 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:4873 3.e 3:76656e66:::rbd_data.1152bd92e04f.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=2b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:17:13.252 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:17:13.252 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 45.050631353999997, 2024-09-14T06:17:13.252 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048796120000000002, 2024-09-14T06:17:13.252 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.252 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.252 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.252 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.252 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.252 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.253 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4873 2024-09-14T06:17:13.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.253 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.253 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.253 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:17:13.253 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.253 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.253 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193648+0000", 2024-09-14T06:17:13.253 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.254 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.254 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193651+0000", 2024-09-14T06:17:13.254 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:17:13.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.254 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.254 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.254 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193663+0000", 2024-09-14T06:17:13.254 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2047e-05 2024-09-14T06:17:13.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.254 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.254 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.254 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193692+0000", 2024-09-14T06:17:13.254 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9346e-05 2024-09-14T06:17:13.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.255 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.255 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193708+0000", 2024-09-14T06:17:13.255 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6123000000000001e-05 2024-09-14T06:17:13.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.255 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.255 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193798+0000", 2024-09-14T06:17:13.255 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9909999999999998e-05 2024-09-14T06:17:13.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.255 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.255 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.193863+0000", 2024-09-14T06:17:13.255 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4583000000000002e-05 2024-09-14T06:17:13.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.256 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.256 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:17:13.256 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.194717+0000", 2024-09-14T06:17:13.256 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00085473299999999999 2024-09-14T06:17:13.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.256 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.256 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.256 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198400+0000", 2024-09-14T06:17:13.256 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036830890000000001 2024-09-14T06:17:13.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.256 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.256 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.257 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198461+0000", 2024-09-14T06:17:13.257 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0844000000000002e-05 2024-09-14T06:17:13.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.257 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.257 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.257 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198496+0000", 2024-09-14T06:17:13.257 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4820000000000002e-05 2024-09-14T06:17:13.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.257 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.257 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.257 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:28.198527+0000", 2024-09-14T06:17:13.257 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1229999999999997e-05 2024-09-14T06:17:13.257 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.257 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.258 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.258 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.258 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:6925 3.5 3:a15e9af1:::rbd_data.1152f87cbac6.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:17:13.258 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:17:13.258 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.129499815999999, 2024-09-14T06:17:13.258 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0049096490000000003, 2024-09-14T06:17:13.258 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.258 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.258 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.258 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.258 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.258 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.258 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 6925 2024-09-14T06:17:13.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.259 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.259 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.259 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.259 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:17:13.259 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.259 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.259 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.259 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114779+0000", 2024-09-14T06:17:13.259 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.259 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.260 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.260 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114780+0000", 2024-09-14T06:17:13.260 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.09e-07 2024-09-14T06:17:13.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.260 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.260 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.260 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114784+0000", 2024-09-14T06:17:13.260 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4669999999999998e-06 2024-09-14T06:17:13.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.260 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.260 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.260 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114799+0000", 2024-09-14T06:17:13.260 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4491000000000001e-05 2024-09-14T06:17:13.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.261 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.261 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.114810+0000", 2024-09-14T06:17:13.261 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1243000000000001e-05 2024-09-14T06:17:13.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.261 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.261 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115515+0000", 2024-09-14T06:17:13.261 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00070521899999999998 2024-09-14T06:17:13.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.261 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.262 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115610+0000", 2024-09-14T06:17:13.262 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.4260999999999998e-05 2024-09-14T06:17:13.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.262 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.262 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 6", 2024-09-14T06:17:13.262 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.115742+0000", 2024-09-14T06:17:13.262 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000132414 2024-09-14T06:17:13.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.262 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.262 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.262 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.116650+0000", 2024-09-14T06:17:13.262 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00090814900000000002 2024-09-14T06:17:13.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.262 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.263 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.263 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119568+0000", 2024-09-14T06:17:13.263 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029178440000000002 2024-09-14T06:17:13.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.263 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.263 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.263 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119636+0000", 2024-09-14T06:17:13.263 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8365e-05 2024-09-14T06:17:13.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.263 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.263 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.263 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.119689+0000", 2024-09-14T06:17:13.263 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2386999999999998e-05 2024-09-14T06:17:13.263 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.264 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.264 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.264 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.264 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:7151 3.e 3:7793b17d:::rbd_data.1152f87cbac6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:17:13.264 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:17:13.264 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.914682026000001, 2024-09-14T06:17:13.264 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.029556783, 2024-09-14T06:17:13.264 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.264 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.264 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.264 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.264 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.265 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.265 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 7151 2024-09-14T06:17:13.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.265 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.265 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.265 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.265 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:17:13.265 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.265 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.265 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.265 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:17:13.265 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.266 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.266 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329599+0000", 2024-09-14T06:17:13.266 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5270000000000001e-06 2024-09-14T06:17:13.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.266 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.266 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329610+0000", 2024-09-14T06:17:13.266 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0431000000000001e-05 2024-09-14T06:17:13.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.266 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.266 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329634+0000", 2024-09-14T06:17:13.267 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3617000000000001e-05 2024-09-14T06:17:13.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.267 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.267 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.267 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329644+0000", 2024-09-14T06:17:13.267 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.095e-05 2024-09-14T06:17:13.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.267 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.267 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.267 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329712+0000", 2024-09-14T06:17:13.267 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7067e-05 2024-09-14T06:17:13.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.267 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.267 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.267 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329827+0000", 2024-09-14T06:17:13.268 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011498 2024-09-14T06:17:13.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.268 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.268 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:17:13.268 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.334271+0000", 2024-09-14T06:17:13.268 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044445839999999997 2024-09-14T06:17:13.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.268 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.268 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.268 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.358967+0000", 2024-09-14T06:17:13.268 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024696356999999999 2024-09-14T06:17:13.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.268 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.269 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.269 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359027+0000", 2024-09-14T06:17:13.269 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0006000000000001e-05 2024-09-14T06:17:13.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.269 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.269 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.269 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359099+0000", 2024-09-14T06:17:13.269 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1052999999999999e-05 2024-09-14T06:17:13.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.269 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.269 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.269 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359154+0000", 2024-09-14T06:17:13.269 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5210999999999997e-05 2024-09-14T06:17:13.269 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:7151 3.e e32/29 ondisk, result = 0)", 2024-09-14T06:17:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:17:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.894737591999998, 2024-09-14T06:17:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0096251340000000005, 2024-09-14T06:17:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:17:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:17:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349543+0000", 2024-09-14T06:17:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:17:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349552+0000", 2024-09-14T06:17:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1039999999999996e-06 2024-09-14T06:17:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349579+0000", 2024-09-14T06:17:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6449999999999999e-05 2024-09-14T06:17:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349593+0000", 2024-09-14T06:17:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4099e-05 2024-09-14T06:17:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359003+0000", 2024-09-14T06:17:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0094096119999999995 2024-09-14T06:17:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359014+0000", 2024-09-14T06:17:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0871e-05 2024-09-14T06:17:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359167+0000", 2024-09-14T06:17:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015300700000000001 2024-09-14T06:17:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15613 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'121, mlcod=36'116)", 2024-09-14T06:17:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:17:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.951374737000002, 2024-09-14T06:17:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053619389999999996, 2024-09-14T06:17:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:17:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292904+0000", 2024-09-14T06:17:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292905+0000", 2024-09-14T06:17:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6800000000000009e-07 2024-09-14T06:17:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292915+0000", 2024-09-14T06:17:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:17:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292927+0000", 2024-09-14T06:17:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-09-14T06:17:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292935+0000", 2024-09-14T06:17:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2459999999999997e-06 2024-09-14T06:17:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.292992+0000", 2024-09-14T06:17:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6873999999999998e-05 2024-09-14T06:17:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293016+0000", 2024-09-14T06:17:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4037e-05 2024-09-14T06:17:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298217+0000", 2024-09-14T06:17:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052008710000000001 2024-09-14T06:17:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298266+0000", 2024-09-14T06:17:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9691000000000002e-05 2024-09-14T06:17:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15615 3.11 e36/29 3:8d07d636:::rbd_data.115223d7013d.0000000000000002:head v 36'122, mlcod=36'116)", 2024-09-14T06:17:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:17:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.950478378, 2024-09-14T06:17:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053069229999999998, 2024-09-14T06:17:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:17:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:17:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.293801+0000", 2024-09-14T06:17:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5799999999999998e-07 2024-09-14T06:17:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294003+0000", 2024-09-14T06:17:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020149600000000001 2024-09-14T06:17:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294016+0000", 2024-09-14T06:17:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3424999999999999e-05 2024-09-14T06:17:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294024+0000", 2024-09-14T06:17:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5240000000000003e-06 2024-09-14T06:17:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294673+0000", 2024-09-14T06:17:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064906700000000003 2024-09-14T06:17:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.294698+0000", 2024-09-14T06:17:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5301999999999999e-05 2024-09-14T06:17:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299064+0000", 2024-09-14T06:17:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043652170000000002 2024-09-14T06:17:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.283 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299108+0000", 2024-09-14T06:17:13.283 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4033999999999999e-05 2024-09-14T06:17:13.283 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.283 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.283 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.283 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.283 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15619 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'124, mlcod=36'119)", 2024-09-14T06:17:13.283 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:13.283 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.94642443, 2024-09-14T06:17:13.283 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061545389999999997, 2024-09-14T06:17:13.283 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.283 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.283 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.284 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.284 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.284 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.284 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:13.284 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.284 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.284 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.284 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:13.284 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.284 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.284 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.284 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:13.284 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3399999999999998e-07 2024-09-14T06:17:13.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.285 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.285 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.285 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297861+0000", 2024-09-14T06:17:13.285 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7640000000000002e-06 2024-09-14T06:17:13.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.285 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.285 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.285 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297876+0000", 2024-09-14T06:17:13.285 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4881e-05 2024-09-14T06:17:13.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.285 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.285 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.285 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297884+0000", 2024-09-14T06:17:13.286 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.588e-06 2024-09-14T06:17:13.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.286 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.286 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298294+0000", 2024-09-14T06:17:13.286 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00041011200000000002 2024-09-14T06:17:13.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.286 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.286 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298317+0000", 2024-09-14T06:17:13.286 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3417999999999998e-05 2024-09-14T06:17:13.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.286 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.286 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303972+0000", 2024-09-14T06:17:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056548680000000004 2024-09-14T06:17:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304009+0000", 2024-09-14T06:17:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7073999999999998e-05 2024-09-14T06:17:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15621 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'125, mlcod=36'119)", 2024-09-14T06:17:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.944913413000002, 2024-09-14T06:17:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0069496169999999999, 2024-09-14T06:17:13.288 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.288 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.288 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.288 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.288 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.288 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.288 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:13.288 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.288 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2200000000000003e-07 2024-09-14T06:17:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299372+0000", 2024-09-14T06:17:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3850000000000001e-06 2024-09-14T06:17:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299387+0000", 2024-09-14T06:17:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4905e-05 2024-09-14T06:17:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299393+0000", 2024-09-14T06:17:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9090000000000003e-06 2024-09-14T06:17:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303591+0000", 2024-09-14T06:17:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041976640000000003 2024-09-14T06:17:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303619+0000", 2024-09-14T06:17:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7651e-05 2024-09-14T06:17:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306287+0000", 2024-09-14T06:17:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026686689999999998 2024-09-14T06:17:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306315+0000", 2024-09-14T06:17:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7712000000000002e-05 2024-09-14T06:17:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:16521 3.e e36/34 3:77c8e3e5:::rbd_data.1152fc09e7e6.0000000000000003:head v 36'83, mlcod=36'81)", 2024-09-14T06:17:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:17:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.318260113000001, 2024-09-14T06:17:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044948899999999996, 2024-09-14T06:17:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:17:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926019+0000", 2024-09-14T06:17:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.926020+0000", 2024-09-14T06:17:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.545e-06 2024-09-14T06:17:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927192+0000", 2024-09-14T06:17:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0011711919999999999 2024-09-14T06:17:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927215+0000", 2024-09-14T06:17:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3677e-05 2024-09-14T06:17:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927226+0000", 2024-09-14T06:17:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0665e-05 2024-09-14T06:17:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927314+0000", 2024-09-14T06:17:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8314999999999997e-05 2024-09-14T06:17:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.927344+0000", 2024-09-14T06:17:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9941e-05 2024-09-14T06:17:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.930411+0000", 2024-09-14T06:17:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030670649999999999 2024-09-14T06:17:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.930514+0000", 2024-09-14T06:17:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010249 2024-09-14T06:17:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:18539 3.1e e40/29 3:7e28f783:::rbd_data.1152227be94f.0000000000000000:head v 40'59, mlcod=40'57)", 2024-09-14T06:17:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:17:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 30.961238062, 2024-09-14T06:17:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0059097430000000003, 2024-09-14T06:17:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:17:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:17:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283043+0000", 2024-09-14T06:17:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4499999999999998e-06 2024-09-14T06:17:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285878+0000", 2024-09-14T06:17:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028348039999999998 2024-09-14T06:17:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285908+0000", 2024-09-14T06:17:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9756000000000001e-05 2024-09-14T06:17:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285922+0000", 2024-09-14T06:17:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.43e-05 2024-09-14T06:17:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.286030+0000", 2024-09-14T06:17:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000107954 2024-09-14T06:17:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.286058+0000", 2024-09-14T06:17:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7796e-05 2024-09-14T06:17:13.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.300 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.300 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.300 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.288888+0000", 2024-09-14T06:17:13.300 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028295799999999999 2024-09-14T06:17:13.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.300 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.300 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.300 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.288951+0000", 2024-09-14T06:17:13.300 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3102999999999995e-05 2024-09-14T06:17:13.300 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.300 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.300 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:24229 3.5 e47/34 3:a30fd30b:::rbd_data.1152db28e106.0000000000000001:head v 47'55, mlcod=45'53)", 2024-09-14T06:17:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:17:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 21.334267815, 2024-09-14T06:17:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0092924199999999992, 2024-09-14T06:17:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:17:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:17:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910013+0000", 2024-09-14T06:17:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.812e-06 2024-09-14T06:17:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.303 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915374+0000", 2024-09-14T06:17:13.303 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053609240000000004 2024-09-14T06:17:13.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.303 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.303 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915395+0000", 2024-09-14T06:17:13.303 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1118e-05 2024-09-14T06:17:13.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.303 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.303 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915406+0000", 2024-09-14T06:17:13.303 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1303e-05 2024-09-14T06:17:13.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915490+0000", 2024-09-14T06:17:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4017000000000006e-05 2024-09-14T06:17:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915515+0000", 2024-09-14T06:17:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4712e-05 2024-09-14T06:17:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919240+0000", 2024-09-14T06:17:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037247690000000002 2024-09-14T06:17:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919304+0000", 2024-09-14T06:17:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3764999999999997e-05 2024-09-14T06:17:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:4, version: 31'46, data_included: [0~98304,360448~278528,704512~131072,1048576~114688,1261568~12416,1409024~196608], data_size: 831616, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:4@31'46, size: 1966080, copy_subset: [(0, 1966080)], clone_subset: {}, snapset: 4=[]:{4: [4]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1966080, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:17:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.207851+0000", 2024-09-14T06:17:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.036427598, 2024-09-14T06:17:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056781219999999999, 2024-09-14T06:17:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.207851+0000", 2024-09-14T06:17:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.207851+0000", 2024-09-14T06:17:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.207853+0000", 2024-09-14T06:17:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3960000000000001e-06 2024-09-14T06:17:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210067+0000", 2024-09-14T06:17:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022146290000000001 2024-09-14T06:17:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210118+0000", 2024-09-14T06:17:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0201999999999998e-05 2024-09-14T06:17:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210155+0000", 2024-09-14T06:17:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7574999999999999e-05 2024-09-14T06:17:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210424+0000", 2024-09-14T06:17:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00026907500000000002 2024-09-14T06:17:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.309 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210467+0000", 2024-09-14T06:17:13.309 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3061e-05 2024-09-14T06:17:13.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.309 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.309 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.309 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.213530+0000", 2024-09-14T06:17:13.309 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0030621839999999999 2024-09-14T06:17:13.309 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.309 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.309 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.309 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.309 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:head, version: 31'47, data_included: [0~98304,360448~278528], data_size: 376832, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:head@31'47, size: 638976, copy_subset: [(0, 638976)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 638976, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:17:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:17:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.03400487, 2024-09-14T06:17:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062487460000000003, 2024-09-14T06:17:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:17:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:17:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210275+0000", 2024-09-14T06:17:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:17:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210752+0000", 2024-09-14T06:17:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047772100000000002 2024-09-14T06:17:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210781+0000", 2024-09-14T06:17:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8391999999999999e-05 2024-09-14T06:17:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210805+0000", 2024-09-14T06:17:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4142999999999999e-05 2024-09-14T06:17:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.213641+0000", 2024-09-14T06:17:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002835827 2024-09-14T06:17:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.213663+0000", 2024-09-14T06:17:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.243e-05 2024-09-14T06:17:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.216523+0000", 2024-09-14T06:17:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028597729999999999 2024-09-14T06:17:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fcd050ea:::rbd_data.1152b921b97f.0000000000000001:e, version: 45'188, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:fcd050ea:::rbd_data.1152b921b97f.0000000000000001:e@45'188, size: 3250974, copy_subset: [(0, 3250974)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:17:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.238893+0000", 2024-09-14T06:17:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.005385910999999, 2024-09-14T06:17:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044553459999999998, 2024-09-14T06:17:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.238893+0000", 2024-09-14T06:17:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.238893+0000", 2024-09-14T06:17:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.238918+0000", 2024-09-14T06:17:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4845e-05 2024-09-14T06:17:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.241095+0000", 2024-09-14T06:17:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0021769070000000001 2024-09-14T06:17:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.241143+0000", 2024-09-14T06:17:13.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7858000000000002e-05 2024-09-14T06:17:13.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.316 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.316 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.241175+0000", 2024-09-14T06:17:13.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2592999999999998e-05 2024-09-14T06:17:13.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.316 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.316 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.241352+0000", 2024-09-14T06:17:13.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0001772 2024-09-14T06:17:13.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.316 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.316 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.241375+0000", 2024-09-14T06:17:13.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2163000000000001e-05 2024-09-14T06:17:13.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.243348+0000", 2024-09-14T06:17:13.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0019737800000000001 2024-09-14T06:17:13.317 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.317 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.317 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.317 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fcd050ea:::rbd_data.1152b921b97f.0000000000000001:head, version: 45'189, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:fcd050ea:::rbd_data.1152b921b97f.0000000000000001:head@45'189, size: 1760991, copy_subset: [(0, 1760991)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:17:13.317 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.241225+0000", 2024-09-14T06:17:13.317 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.003053856999999, 2024-09-14T06:17:13.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0042946119999999997, 2024-09-14T06:17:13.318 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.318 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.318 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:13.318 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.241225+0000", 2024-09-14T06:17:13.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.241225+0000", 2024-09-14T06:17:13.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.319 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.319 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.241226+0000", 2024-09-14T06:17:13.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.083e-06 2024-09-14T06:17:13.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.319 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.319 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.242910+0000", 2024-09-14T06:17:13.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0016836900000000001 2024-09-14T06:17:13.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.319 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.319 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.242953+0000", 2024-09-14T06:17:13.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3349000000000001e-05 2024-09-14T06:17:13.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.242980+0000", 2024-09-14T06:17:13.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6522000000000001e-05 2024-09-14T06:17:13.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.243531+0000", 2024-09-14T06:17:13.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00055117499999999999 2024-09-14T06:17:13.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.243562+0000", 2024-09-14T06:17:13.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1285999999999998e-05 2024-09-14T06:17:13.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.321 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.321 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.245520+0000", 2024-09-14T06:17:13.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0019575069999999998 2024-09-14T06:17:13.321 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.321 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.321 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.321 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fcd050ea:::rbd_data.1152b921b97f.0000000000000001:e, version: 45'188, data_included: [1048576~524288,1703936~262144], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:fcd050ea:::rbd_data.1152b921b97f.0000000000000001:e@45'188, size: 3250974, copy_subset: [(0, 3250974)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3250974, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-14T06:17:13.321 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.247262+0000", 2024-09-14T06:17:13.321 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.997017353, 2024-09-14T06:17:13.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044393130000000003, 2024-09-14T06:17:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.247262+0000", 2024-09-14T06:17:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.247262+0000", 2024-09-14T06:17:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.247284+0000", 2024-09-14T06:17:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2792000000000001e-05 2024-09-14T06:17:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.249000+0000", 2024-09-14T06:17:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001715299 2024-09-14T06:17:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.249047+0000", 2024-09-14T06:17:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7425000000000001e-05 2024-09-14T06:17:13.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.324 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.324 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.249081+0000", 2024-09-14T06:17:13.324 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4125000000000003e-05 2024-09-14T06:17:13.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.324 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.324 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.249332+0000", 2024-09-14T06:17:13.324 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00025113199999999999 2024-09-14T06:17:13.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.325 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.325 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.249373+0000", 2024-09-14T06:17:13.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0711999999999999e-05 2024-09-14T06:17:13.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.325 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.325 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.251701+0000", 2024-09-14T06:17:13.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023278280000000001 2024-09-14T06:17:13.325 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.325 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.325 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.325 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fcd050ea:::rbd_data.1152b921b97f.0000000000000001:head, version: 45'189, data_included: [1048576~524288,1703936~57055], data_size: 581343, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:fcd050ea:::rbd_data.1152b921b97f.0000000000000001:head@45'189, size: 1760991, copy_subset: [(0, 1760991)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1760991, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-14T06:17:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.249132+0000", 2024-09-14T06:17:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.995147125999999, 2024-09-14T06:17:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044146469999999998, 2024-09-14T06:17:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.249132+0000", 2024-09-14T06:17:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.327 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.249132+0000", 2024-09-14T06:17:13.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.327 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.327 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.249133+0000", 2024-09-14T06:17:13.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0130000000000001e-06 2024-09-14T06:17:13.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.327 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.327 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.250157+0000", 2024-09-14T06:17:13.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001024608 2024-09-14T06:17:13.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.250240+0000", 2024-09-14T06:17:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2484000000000005e-05 2024-09-14T06:17:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.250295+0000", 2024-09-14T06:17:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4737999999999997e-05 2024-09-14T06:17:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.251872+0000", 2024-09-14T06:17:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0015773269999999999 2024-09-14T06:17:13.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.329 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.329 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.251893+0000", 2024-09-14T06:17:13.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0919000000000001e-05 2024-09-14T06:17:13.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.329 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.329 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.253547+0000", 2024-09-14T06:17:13.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0016535580000000001 2024-09-14T06:17:13.329 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.329 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.329 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.330 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:30750 3.2 e55/29 3:452a53e9:::rbd_data.11529eb229bd.0000000000000003:head v 55'170, mlcod=55'168)", 2024-09-14T06:17:13.330 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.760736+0000", 2024-09-14T06:17:13.330 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.483542761000001, 2024-09-14T06:17:13.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047328800000000001, 2024-09-14T06:17:13.330 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.330 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.330 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.330 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.760736+0000", 2024-09-14T06:17:13.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.760736+0000", 2024-09-14T06:17:13.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.760738+0000", 2024-09-14T06:17:13.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.917e-06 2024-09-14T06:17:13.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.763675+0000", 2024-09-14T06:17:13.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029371380000000002 2024-09-14T06:17:13.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.763705+0000", 2024-09-14T06:17:13.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9929000000000001e-05 2024-09-14T06:17:13.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.763725+0000", 2024-09-14T06:17:13.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9629000000000001e-05 2024-09-14T06:17:13.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.763893+0000", 2024-09-14T06:17:13.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016794100000000001 2024-09-14T06:17:13.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.333 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.763930+0000", 2024-09-14T06:17:13.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7595000000000002e-05 2024-09-14T06:17:13.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.333 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.765381+0000", 2024-09-14T06:17:13.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001450221 2024-09-14T06:17:13.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.765469+0000", 2024-09-14T06:17:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8510000000000005e-05 2024-09-14T06:17:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:32082 3.11 e58/29 3:8be13bcd:::rbd_object_map.115274ed0aa8:head v 58'244, mlcod=58'240)", 2024-09-14T06:17:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:04.005484+0000", 2024-09-14T06:17:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.2387949630000001, 2024-09-14T06:17:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050265900000000001, 2024-09-14T06:17:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.005484+0000", 2024-09-14T06:17:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.005484+0000", 2024-09-14T06:17:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.005485+0000", 2024-09-14T06:17:13.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:17:13.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.005490+0000", 2024-09-14T06:17:13.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7820000000000001e-06 2024-09-14T06:17:13.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.005498+0000", 2024-09-14T06:17:13.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2709999999999992e-06 2024-09-14T06:17:13.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.005505+0000", 2024-09-14T06:17:13.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9480000000000002e-06 2024-09-14T06:17:13.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.337 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.337 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.005536+0000", 2024-09-14T06:17:13.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.154e-05 2024-09-14T06:17:13.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.337 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.337 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.005548+0000", 2024-09-14T06:17:13.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1695999999999999e-05 2024-09-14T06:17:13.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.337 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.010470+0000", 2024-09-14T06:17:13.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0049218780000000002 2024-09-14T06:17:13.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:04.010511+0000", 2024-09-14T06:17:13.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0781999999999997e-05 2024-09-14T06:17:13.338 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.338 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.338 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.338 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.338 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.338 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:13.339 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:13.339 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:13.340 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:17:13.341 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:13.341 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:13.342 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:13.337+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:13.352 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:17:13.440 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:13.437+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:13.519 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:13.519 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:13.519 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:13.519 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:13.536 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:17:13.542 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:13.537+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:13.643 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:13.641+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:13.652 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:13.652 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:13.652 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:13.652 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:13.652 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:13.667 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:17:13.744 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:13.741+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:13.802 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:13.802 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:13.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:13.802 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:13.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.802 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11263 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:13.802 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:13.802 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.809085609, 2024-09-14T06:17:13.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049944574999999998, 2024-09-14T06:17:13.803 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.803 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.803 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:13.803 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.803 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.803 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.803 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11263 2024-09-14T06:17:13.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.803 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:13.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.804 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.804 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:13.804 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.804 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.804 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:13.804 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.51e-07 2024-09-14T06:17:13.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.804 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.804 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991560+0000", 2024-09-14T06:17:13.805 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6719999999999998e-06 2024-09-14T06:17:13.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.805 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.805 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991567+0000", 2024-09-14T06:17:13.805 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1470000000000004e-06 2024-09-14T06:17:13.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.805 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.805 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991572+0000", 2024-09-14T06:17:13.805 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.237e-06 2024-09-14T06:17:13.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.806 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.806 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041149+0000", 2024-09-14T06:17:13.806 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049576988000000002 2024-09-14T06:17:13.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.806 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.806 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.806 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041277+0000", 2024-09-14T06:17:13.806 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000127687 2024-09-14T06:17:13.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.806 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.806 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.806 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041497+0000", 2024-09-14T06:17:13.806 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022039299999999999 2024-09-14T06:17:13.807 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.807 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.807 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.807 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11264 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 327680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:13.807 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:13.807 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.809059267999999, 2024-09-14T06:17:13.807 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050248567000000001, 2024-09-14T06:17:13.807 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.807 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.807 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:13.807 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.807 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.808 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.808 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11264 2024-09-14T06:17:13.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.808 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.808 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.808 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.808 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:13.808 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.808 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.808 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.808 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:13.808 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.809 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.809 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.809 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:13.809 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.41e-07 2024-09-14T06:17:13.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.809 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.809 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.809 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991581+0000", 2024-09-14T06:17:13.809 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3740000000000001e-06 2024-09-14T06:17:13.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.809 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.809 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.809 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991585+0000", 2024-09-14T06:17:13.810 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9400000000000004e-06 2024-09-14T06:17:13.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.810 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.810 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.810 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991587+0000", 2024-09-14T06:17:13.810 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.694e-06 2024-09-14T06:17:13.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.810 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.810 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.810 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041527+0000", 2024-09-14T06:17:13.810 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049939349000000001 2024-09-14T06:17:13.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.810 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.810 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.810 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041598+0000", 2024-09-14T06:17:13.811 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1633999999999999e-05 2024-09-14T06:17:13.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.811 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.811 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041827+0000", 2024-09-14T06:17:13.811 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000229235 2024-09-14T06:17:13.811 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.811 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.811 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.811 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:13.811 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:13.812 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.809045928000003, 2024-09-14T06:17:13.812 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:17:13.812 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.812 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.812 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:13.812 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.812 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.812 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.812 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:17:13.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.812 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.812 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.812 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.812 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:13.813 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.813 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.813 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.813 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:13.813 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.813 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.813 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.813 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:13.813 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:17:13.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.813 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.813 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.814 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:17:13.814 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:17:13.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.814 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.814 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:17:13.814 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:17:13.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.814 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.814 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:17:13.814 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:17:13.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.815 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.815 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:17:13.815 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:17:13.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.815 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.815 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:17:13.815 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:17:13.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.815 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.815 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:17:13.816 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:17:13.816 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.816 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.816 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.816 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.816 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:13.816 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:13.816 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.809032856000002, 2024-09-14T06:17:13.816 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:17:13.816 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.816 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.817 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:13.817 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.817 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.817 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.817 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:17:13.817 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.817 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.817 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.817 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.817 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:13.817 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.817 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.817 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.817 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.818 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:13.818 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.818 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.818 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.818 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:17:13.818 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:17:13.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.818 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.818 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.818 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:17:13.818 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:17:13.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.818 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.819 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.819 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:17:13.819 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:17:13.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.819 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.819 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.819 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:17:13.819 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:17:13.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.819 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.819 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.819 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:17:13.819 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:17:13.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.820 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.820 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.820 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:17:13.820 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:17:13.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.820 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.820 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.820 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:17:13.820 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:17:13.820 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.820 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.820 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.821 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.821 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:13.821 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:13.821 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.809015424000002, 2024-09-14T06:17:13.821 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:17:13.821 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.821 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.821 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:13.821 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.821 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.821 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.821 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:17:13.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.821 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.822 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.822 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:13.822 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.822 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.822 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:13.822 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.822 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.822 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:13.822 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:17:13.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.823 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.823 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:17:13.823 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:17:13.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.823 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.823 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:17:13.823 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:17:13.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.823 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.823 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:17:13.823 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:17:13.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.824 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.824 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:17:13.824 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:17:13.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.824 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.824 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:17:13.824 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:17:13.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.824 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.824 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:17:13.825 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:17:13.825 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.825 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.825 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.825 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:13.825 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:13.825 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.808994319, 2024-09-14T06:17:13.825 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:17:13.825 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.825 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.825 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:13.825 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.826 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.826 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.826 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:17:13.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.826 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.826 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.826 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.826 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:13.826 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.826 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.826 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.826 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:13.826 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.827 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.827 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:13.827 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:17:13.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.827 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.827 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:17:13.827 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:17:13.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.827 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.827 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:17:13.827 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:17:13.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.828 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.828 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.828 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:17:13.828 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:17:13.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.828 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.828 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.828 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:17:13.828 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:17:13.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.828 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.828 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.828 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:17:13.828 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:17:13.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.829 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.829 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:17:13.829 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:17:13.829 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.829 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.829 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.829 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:13.829 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:13.829 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.808973211000001, 2024-09-14T06:17:13.829 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:17:13.830 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.830 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.830 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:13.830 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.830 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.830 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.830 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:17:13.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.830 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.830 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.830 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:13.830 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.831 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.831 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.831 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:13.831 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.831 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.831 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.831 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:13.831 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:17:13.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.831 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.831 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.831 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:17:13.831 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:17:13.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.832 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.832 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.832 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:17:13.832 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:17:13.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.832 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.832 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.832 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:17:13.832 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:17:13.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.832 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.832 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.833 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:17:13.833 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:17:13.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.833 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.833 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:17:13.833 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:17:13.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.833 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.833 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:17:13.833 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:17:13.833 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.833 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.834 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.834 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48234 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:13.834 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:13.834 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.7677019189999998, 2024-09-14T06:17:13.834 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051730353999999999, 2024-09-14T06:17:13.834 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.834 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.834 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.834 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.834 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.834 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.834 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48234 2024-09-14T06:17:13.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.835 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.835 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.835 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:13.835 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.835 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.835 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:13.835 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.835 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.835 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032937+0000", 2024-09-14T06:17:13.835 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:17:13.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.836 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.836 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.836 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032940+0000", 2024-09-14T06:17:13.836 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0580000000000002e-06 2024-09-14T06:17:13.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.836 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.836 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.836 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032946+0000", 2024-09-14T06:17:13.836 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2960000000000004e-06 2024-09-14T06:17:13.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.836 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.836 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.836 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032951+0000", 2024-09-14T06:17:13.837 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.938e-06 2024-09-14T06:17:13.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.837 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.837 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.837 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074419+0000", 2024-09-14T06:17:13.837 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041467979000000002 2024-09-14T06:17:13.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.837 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.837 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.837 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074490+0000", 2024-09-14T06:17:13.837 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1324999999999999e-05 2024-09-14T06:17:13.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.837 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.838 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:13.838 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074859+0000", 2024-09-14T06:17:13.838 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036914899999999999 2024-09-14T06:17:13.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.838 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.838 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.838 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078464+0000", 2024-09-14T06:17:13.838 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036043170000000001 2024-09-14T06:17:13.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.838 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.838 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.838 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084547+0000", 2024-09-14T06:17:13.838 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006083097 2024-09-14T06:17:13.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.839 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.839 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.839 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084614+0000", 2024-09-14T06:17:13.839 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7509999999999996e-05 2024-09-14T06:17:13.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.839 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.839 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.839 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084667+0000", 2024-09-14T06:17:13.839 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:17:13.839 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.839 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.839 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.839 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.839 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48237 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:13.840 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:13.840 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.7676221750000001, 2024-09-14T06:17:13.840 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051772830999999998, 2024-09-14T06:17:13.840 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.840 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.840 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.840 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.840 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.840 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.840 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48237 2024-09-14T06:17:13.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.840 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.840 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.840 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.841 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:13.841 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.841 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.841 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.841 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:13.841 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.841 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.841 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.841 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:13.841 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0499999999999999e-07 2024-09-14T06:17:13.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.841 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.842 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.842 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033018+0000", 2024-09-14T06:17:13.842 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0949999999999998e-06 2024-09-14T06:17:13.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.842 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.842 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.842 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033023+0000", 2024-09-14T06:17:13.842 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1620000000000001e-06 2024-09-14T06:17:13.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.842 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.842 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.842 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033026+0000", 2024-09-14T06:17:13.842 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0120000000000001e-06 2024-09-14T06:17:13.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.842 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.843 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.843 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075349+0000", 2024-09-14T06:17:13.843 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042323373999999997 2024-09-14T06:17:13.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.843 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.843 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.843 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075394+0000", 2024-09-14T06:17:13.843 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4861000000000003e-05 2024-09-14T06:17:13.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.843 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.843 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:13.843 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075578+0000", 2024-09-14T06:17:13.843 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00018414899999999999 2024-09-14T06:17:13.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.844 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.844 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.844 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078894+0000", 2024-09-14T06:17:13.844 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033158480000000001 2024-09-14T06:17:13.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.844 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.844 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.844 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084677+0000", 2024-09-14T06:17:13.844 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0057832120000000002 2024-09-14T06:17:13.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.844 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.844 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.844 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084752+0000", 2024-09-14T06:17:13.844 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5123000000000006e-05 2024-09-14T06:17:13.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.845 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.845 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.845 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084789+0000", 2024-09-14T06:17:13.845 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6690000000000003e-05 2024-09-14T06:17:13.845 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.845 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.845 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.845 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.845 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48243 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:13.845 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:13.845 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.7675410500000002, 2024-09-14T06:17:13.845 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051780221000000001, 2024-09-14T06:17:13.846 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.846 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.846 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.846 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.846 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.846 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.846 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48243 2024-09-14T06:17:13.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.846 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.846 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.846 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.846 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:13.846 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.847 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.847 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.847 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:13.847 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.847 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.847 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.847 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:13.847 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:17:13.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.847 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.847 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.847 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033100+0000", 2024-09-14T06:17:13.848 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:17:13.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.848 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.848 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.848 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033108+0000", 2024-09-14T06:17:13.848 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1760000000000004e-06 2024-09-14T06:17:13.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.848 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.848 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.848 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033111+0000", 2024-09-14T06:17:13.848 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0790000000000001e-06 2024-09-14T06:17:13.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.848 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.849 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.849 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075903+0000", 2024-09-14T06:17:13.849 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042791942999999999 2024-09-14T06:17:13.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.849 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.849 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.849 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075987+0000", 2024-09-14T06:17:13.849 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4467000000000003e-05 2024-09-14T06:17:13.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.849 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.849 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:13.849 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.076259+0000", 2024-09-14T06:17:13.849 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027201900000000003 2024-09-14T06:17:13.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.850 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.850 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.850 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078947+0000", 2024-09-14T06:17:13.850 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026875879999999999 2024-09-14T06:17:13.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.850 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.850 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.850 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084797+0000", 2024-09-14T06:17:13.850 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0058500380000000001 2024-09-14T06:17:13.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.850 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.850 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.850 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084843+0000", 2024-09-14T06:17:13.851 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6298999999999998e-05 2024-09-14T06:17:13.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.851 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.851 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.851 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084877+0000", 2024-09-14T06:17:13.851 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4378999999999997e-05 2024-09-14T06:17:13.851 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.851 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.851 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.851 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.851 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48287 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:13.851 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:13.852 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.76750125, 2024-09-14T06:17:13.852 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051924706000000001, 2024-09-14T06:17:13.852 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.852 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.852 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.852 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.852 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.852 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.852 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48287 2024-09-14T06:17:13.852 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.852 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.852 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.852 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.853 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:13.853 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.853 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.853 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.853 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:13.853 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.853 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.853 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.853 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033138+0000", 2024-09-14T06:17:13.853 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6599999999999996e-07 2024-09-14T06:17:13.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.854 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.854 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.854 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033141+0000", 2024-09-14T06:17:13.854 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0879999999999999e-06 2024-09-14T06:17:13.854 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.854 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.854 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.854 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033147+0000", 2024-09-14T06:17:13.854 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2249999999999997e-06 2024-09-14T06:17:13.854 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.854 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.854 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.854 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033153+0000", 2024-09-14T06:17:13.855 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.066e-06 2024-09-14T06:17:13.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.855 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.855 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.855 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077790+0000", 2024-09-14T06:17:13.855 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.044637255000000001 2024-09-14T06:17:13.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.855 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.855 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.855 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077845+0000", 2024-09-14T06:17:13.855 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4410999999999998e-05 2024-09-14T06:17:13.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.855 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.855 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:13.856 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078062+0000", 2024-09-14T06:17:13.856 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021755599999999999 2024-09-14T06:17:13.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.856 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.856 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.856 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.080991+0000", 2024-09-14T06:17:13.856 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029282969999999998 2024-09-14T06:17:13.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.856 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.856 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.856 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084885+0000", 2024-09-14T06:17:13.856 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003894495 2024-09-14T06:17:13.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.856 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.856 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.856 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085020+0000", 2024-09-14T06:17:13.857 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013500799999999999 2024-09-14T06:17:13.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.857 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.857 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.857 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085062+0000", 2024-09-14T06:17:13.857 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1638999999999999e-05 2024-09-14T06:17:13.857 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.857 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.857 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.857 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.857 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48288 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:13.857 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:13.857 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.7674775509999998, 2024-09-14T06:17:13.858 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051994018000000003, 2024-09-14T06:17:13.858 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.858 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.858 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.858 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.858 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.858 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.858 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48288 2024-09-14T06:17:13.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.858 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.858 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.858 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.858 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:13.858 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.859 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.859 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.859 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:13.859 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.859 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.859 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.859 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:13.859 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.89e-07 2024-09-14T06:17:13.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.859 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.859 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.859 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033164+0000", 2024-09-14T06:17:13.859 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.779e-06 2024-09-14T06:17:13.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.860 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.860 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.860 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033171+0000", 2024-09-14T06:17:13.860 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6279999999999999e-06 2024-09-14T06:17:13.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.860 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.860 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.860 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033176+0000", 2024-09-14T06:17:13.860 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1610000000000002e-06 2024-09-14T06:17:13.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.860 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.860 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.860 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078991+0000", 2024-09-14T06:17:13.861 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045815348999999998 2024-09-14T06:17:13.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.861 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.861 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.861 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079086+0000", 2024-09-14T06:17:13.861 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5282e-05 2024-09-14T06:17:13.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.861 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.861 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:13.861 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079397+0000", 2024-09-14T06:17:13.861 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031055999999999998 2024-09-14T06:17:13.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.861 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.861 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.862 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084432+0000", 2024-09-14T06:17:13.862 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050355599999999997 2024-09-14T06:17:13.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.862 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.862 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.862 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085073+0000", 2024-09-14T06:17:13.862 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064015199999999995 2024-09-14T06:17:13.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.862 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.862 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.862 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085120+0000", 2024-09-14T06:17:13.862 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7107000000000001e-05 2024-09-14T06:17:13.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.862 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.862 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.863 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085155+0000", 2024-09-14T06:17:13.863 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4950999999999998e-05 2024-09-14T06:17:13.863 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.863 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.863 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.863 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.863 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.863 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48416 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:13.863 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:13.863 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.7652371389999999, 2024-09-14T06:17:13.863 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053690144000000002, 2024-09-14T06:17:13.863 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.863 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.863 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.863 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.864 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.864 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.864 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48416 2024-09-14T06:17:13.864 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.864 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.864 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.864 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.864 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:13.864 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.864 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.864 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.864 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.864 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:13.864 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.865 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.865 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.865 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035402+0000", 2024-09-14T06:17:13.865 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7999999999999995e-07 2024-09-14T06:17:13.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.865 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.865 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.865 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035405+0000", 2024-09-14T06:17:13.865 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3960000000000002e-06 2024-09-14T06:17:13.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.865 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.865 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.865 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035413+0000", 2024-09-14T06:17:13.866 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2209999999999997e-06 2024-09-14T06:17:13.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.866 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.866 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.866 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035420+0000", 2024-09-14T06:17:13.866 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1539999999999996e-06 2024-09-14T06:17:13.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.866 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.866 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.866 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081021+0000", 2024-09-14T06:17:13.866 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045600389999999998 2024-09-14T06:17:13.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.866 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.867 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.867 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081091+0000", 2024-09-14T06:17:13.867 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0325000000000002e-05 2024-09-14T06:17:13.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.867 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.867 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:13.867 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081434+0000", 2024-09-14T06:17:13.867 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000342519 2024-09-14T06:17:13.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.867 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.867 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.867 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085254+0000", 2024-09-14T06:17:13.867 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038201609999999999 2024-09-14T06:17:13.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.868 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.868 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.868 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088977+0000", 2024-09-14T06:17:13.868 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003722805 2024-09-14T06:17:13.868 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.868 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.868 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.868 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089048+0000", 2024-09-14T06:17:13.868 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1266999999999994e-05 2024-09-14T06:17:13.868 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.868 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.868 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.868 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089091+0000", 2024-09-14T06:17:13.868 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3325999999999998e-05 2024-09-14T06:17:13.869 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.869 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.869 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.869 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.869 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.869 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48417 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:13.869 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:13.869 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.7652084700000001, 2024-09-14T06:17:13.869 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053805032000000003, 2024-09-14T06:17:13.869 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.869 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.869 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.869 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.869 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.870 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.870 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48417 2024-09-14T06:17:13.870 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.870 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.870 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.870 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.870 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:13.870 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.870 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.870 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.870 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.870 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:13.870 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.870 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.871 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.871 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.871 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:13.871 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9500000000000003e-07 2024-09-14T06:17:13.871 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.871 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.871 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.871 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035433+0000", 2024-09-14T06:17:13.871 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.796e-06 2024-09-14T06:17:13.871 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.871 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.871 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.871 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035442+0000", 2024-09-14T06:17:13.871 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7120000000000006e-06 2024-09-14T06:17:13.872 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.872 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.872 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.872 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035448+0000", 2024-09-14T06:17:13.872 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.015e-06 2024-09-14T06:17:13.872 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.872 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.872 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.872 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081855+0000", 2024-09-14T06:17:13.872 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.046407634000000003 2024-09-14T06:17:13.872 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.872 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.872 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.872 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081938+0000", 2024-09-14T06:17:13.873 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2687000000000003e-05 2024-09-14T06:17:13.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.873 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.873 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:13.873 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082313+0000", 2024-09-14T06:17:13.873 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000374872 2024-09-14T06:17:13.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.873 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.873 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.873 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087616+0000", 2024-09-14T06:17:13.873 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053032670000000004 2024-09-14T06:17:13.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.873 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.873 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.874 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089140+0000", 2024-09-14T06:17:13.874 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0015234350000000001 2024-09-14T06:17:13.874 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.874 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.874 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.874 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089184+0000", 2024-09-14T06:17:13.874 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4804e-05 2024-09-14T06:17:13.874 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.874 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.874 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.874 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089235+0000", 2024-09-14T06:17:13.874 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0315000000000002e-05 2024-09-14T06:17:13.874 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.874 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.875 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.875 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.875 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.875 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48418 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:13.875 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:13.875 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.7651643310000003, 2024-09-14T06:17:13.875 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053936425000000003, 2024-09-14T06:17:13.875 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.875 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.875 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.875 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.875 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.875 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.875 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48418 2024-09-14T06:17:13.875 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.876 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.876 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.876 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.876 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:13.876 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.876 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.876 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.876 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.876 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:13.876 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.876 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.876 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.876 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.877 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035475+0000", 2024-09-14T06:17:13.877 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7000000000000003e-07 2024-09-14T06:17:13.877 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.877 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.877 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.877 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035479+0000", 2024-09-14T06:17:13.877 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2350000000000001e-06 2024-09-14T06:17:13.877 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.877 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.877 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.877 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035488+0000", 2024-09-14T06:17:13.877 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8820000000000007e-06 2024-09-14T06:17:13.877 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.877 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.878 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.878 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035494+0000", 2024-09-14T06:17:13.878 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3060000000000003e-06 2024-09-14T06:17:13.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.878 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.878 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.878 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082826+0000", 2024-09-14T06:17:13.878 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.047332008000000002 2024-09-14T06:17:13.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.878 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.878 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.878 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082883+0000", 2024-09-14T06:17:13.878 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7124999999999999e-05 2024-09-14T06:17:13.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.879 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.879 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:13.879 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.083385+0000", 2024-09-14T06:17:13.879 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050134099999999998 2024-09-14T06:17:13.879 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.879 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.879 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.879 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088888+0000", 2024-09-14T06:17:13.879 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005503127 2024-09-14T06:17:13.879 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.879 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.879 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.879 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089294+0000", 2024-09-14T06:17:13.879 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040669700000000003 2024-09-14T06:17:13.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.880 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.880 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.880 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089348+0000", 2024-09-14T06:17:13.880 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3729e-05 2024-09-14T06:17:13.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.880 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.880 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.880 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089410+0000", 2024-09-14T06:17:13.880 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2105000000000003e-05 2024-09-14T06:17:13.880 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.880 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.880 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.881 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.881 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.881 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48419 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:13.881 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:13.881 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.765086701, 2024-09-14T06:17:13.881 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054007545999999997, 2024-09-14T06:17:13.881 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.881 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.881 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.881 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.881 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.881 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.881 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48419 2024-09-14T06:17:13.881 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.881 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.882 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.882 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.882 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:13.882 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.882 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.882 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.882 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:13.882 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.882 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.882 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.882 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035552+0000", 2024-09-14T06:17:13.882 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4099999999999998e-07 2024-09-14T06:17:13.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.883 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.883 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.883 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035557+0000", 2024-09-14T06:17:13.883 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1819999999999997e-06 2024-09-14T06:17:13.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.883 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.883 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.883 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035566+0000", 2024-09-14T06:17:13.883 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1120000000000002e-06 2024-09-14T06:17:13.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.883 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.883 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.884 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035572+0000", 2024-09-14T06:17:13.884 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0430000000000004e-06 2024-09-14T06:17:13.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.884 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.884 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.884 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085304+0000", 2024-09-14T06:17:13.884 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049731707 2024-09-14T06:17:13.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.884 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.884 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.884 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085371+0000", 2024-09-14T06:17:13.884 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6771000000000002e-05 2024-09-14T06:17:13.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.885 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.885 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:13.885 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085601+0000", 2024-09-14T06:17:13.885 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022968000000000001 2024-09-14T06:17:13.885 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.885 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.885 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.885 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088893+0000", 2024-09-14T06:17:13.885 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032927780000000001 2024-09-14T06:17:13.885 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.885 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.885 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.885 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089467+0000", 2024-09-14T06:17:13.885 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057330399999999996 2024-09-14T06:17:13.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.886 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.886 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.886 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089520+0000", 2024-09-14T06:17:13.886 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3133000000000001e-05 2024-09-14T06:17:13.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.886 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.886 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.886 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089559+0000", 2024-09-14T06:17:13.886 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9195e-05 2024-09-14T06:17:13.886 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.886 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.886 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.887 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.887 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48420 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:13.887 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:13.887 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.764997471, 2024-09-14T06:17:13.887 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056401932000000002, 2024-09-14T06:17:13.887 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.887 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.887 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.887 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.887 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.887 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.887 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48420 2024-09-14T06:17:13.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.888 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.888 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.888 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.888 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:13.888 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.888 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.888 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.888 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.888 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:13.888 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.888 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.888 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.888 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.888 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035642+0000", 2024-09-14T06:17:13.889 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1519999999999999e-06 2024-09-14T06:17:13.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.889 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.889 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.889 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035647+0000", 2024-09-14T06:17:13.889 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2299999999999999e-06 2024-09-14T06:17:13.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.889 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.889 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.889 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035660+0000", 2024-09-14T06:17:13.889 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3166999999999999e-05 2024-09-14T06:17:13.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.889 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.889 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.890 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035666+0000", 2024-09-14T06:17:13.890 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1720000000000004e-06 2024-09-14T06:17:13.890 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.890 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.890 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.890 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086104+0000", 2024-09-14T06:17:13.890 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050437956999999999 2024-09-14T06:17:13.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.891 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.891 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.891 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086164+0000", 2024-09-14T06:17:13.891 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9592999999999997e-05 2024-09-14T06:17:13.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.891 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.891 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:13.891 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086402+0000", 2024-09-14T06:17:13.891 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000238314 2024-09-14T06:17:13.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.891 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.891 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.892 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089103+0000", 2024-09-14T06:17:13.892 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027011600000000002 2024-09-14T06:17:13.892 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.892 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.892 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.892 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091962+0000", 2024-09-14T06:17:13.892 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028581259999999999 2024-09-14T06:17:13.892 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.892 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.892 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.892 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092005+0000", 2024-09-14T06:17:13.892 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3111000000000001e-05 2024-09-14T06:17:13.892 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.892 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.893 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.893 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092043+0000", 2024-09-14T06:17:13.893 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7950000000000001e-05 2024-09-14T06:17:13.893 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.893 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.893 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.893 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.893 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.893 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48424 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:13.893 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:13.893 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.7647335260000001, 2024-09-14T06:17:13.893 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056273173000000003, 2024-09-14T06:17:13.893 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.893 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.894 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.894 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.894 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.894 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.894 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48424 2024-09-14T06:17:13.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.894 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.894 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.894 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.894 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:13.894 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.894 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.895 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.895 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:13.895 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.895 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.895 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.895 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:13.895 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2900000000000003e-07 2024-09-14T06:17:13.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.895 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.895 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.895 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035909+0000", 2024-09-14T06:17:13.895 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6490000000000001e-06 2024-09-14T06:17:13.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.896 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.896 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.896 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035918+0000", 2024-09-14T06:17:13.896 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5089999999999999e-06 2024-09-14T06:17:13.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.896 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.896 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.896 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035924+0000", 2024-09-14T06:17:13.896 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7089999999999999e-06 2024-09-14T06:17:13.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.896 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.896 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.897 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087662+0000", 2024-09-14T06:17:13.897 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737621999999997 2024-09-14T06:17:13.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.897 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.897 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.897 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087744+0000", 2024-09-14T06:17:13.897 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2174000000000003e-05 2024-09-14T06:17:13.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.897 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.897 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:13.897 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088106+0000", 2024-09-14T06:17:13.897 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000361947 2024-09-14T06:17:13.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.897 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.898 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.898 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092053+0000", 2024-09-14T06:17:13.898 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0039473080000000001 2024-09-14T06:17:13.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.898 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.898 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.898 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092087+0000", 2024-09-14T06:17:13.898 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4153000000000003e-05 2024-09-14T06:17:13.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.898 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.898 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.898 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092138+0000", 2024-09-14T06:17:13.898 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0804999999999999e-05 2024-09-14T06:17:13.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.899 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.899 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.899 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092178+0000", 2024-09-14T06:17:13.899 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9668e-05 2024-09-14T06:17:13.899 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.899 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.899 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.899 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.899 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48508 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:13.899 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:13.899 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.7596584449999999, 2024-09-14T06:17:13.900 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054906017000000001, 2024-09-14T06:17:13.900 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.900 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.900 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.900 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.900 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.900 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.900 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48508 2024-09-14T06:17:13.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.900 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.900 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.900 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.900 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:13.901 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.901 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.901 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.901 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:13.901 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.901 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.901 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.901 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040981+0000", 2024-09-14T06:17:13.901 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7700000000000004e-07 2024-09-14T06:17:13.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.901 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.902 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.902 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040984+0000", 2024-09-14T06:17:13.902 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1480000000000002e-06 2024-09-14T06:17:13.902 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.902 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.902 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.902 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040991+0000", 2024-09-14T06:17:13.902 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4580000000000004e-06 2024-09-14T06:17:13.902 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.902 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.902 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.902 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040997+0000", 2024-09-14T06:17:13.902 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7799999999999997e-06 2024-09-14T06:17:13.903 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.903 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.903 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.903 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089601+0000", 2024-09-14T06:17:13.903 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048604264000000001 2024-09-14T06:17:13.903 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.903 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.903 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.903 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089708+0000", 2024-09-14T06:17:13.903 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010681200000000001 2024-09-14T06:17:13.903 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.903 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.903 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:13.903 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090384+0000", 2024-09-14T06:17:13.904 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00067557499999999998 2024-09-14T06:17:13.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.904 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.904 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.904 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093879+0000", 2024-09-14T06:17:13.904 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034954679999999998 2024-09-14T06:17:13.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.904 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.904 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.904 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095764+0000", 2024-09-14T06:17:13.904 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018848070000000001 2024-09-14T06:17:13.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.904 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.904 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.905 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095837+0000", 2024-09-14T06:17:13.905 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3433e-05 2024-09-14T06:17:13.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.905 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.905 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.905 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095886+0000", 2024-09-14T06:17:13.905 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8495000000000002e-05 2024-09-14T06:17:13.905 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.905 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.905 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.905 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.905 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48537 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:13.905 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:13.906 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.7582946750000001, 2024-09-14T06:17:13.906 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053681833999999998, 2024-09-14T06:17:13.906 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:13.906 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:13.906 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:13.906 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:13.906 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:13.906 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:13.906 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48537 2024-09-14T06:17:13.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.906 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:13.906 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.906 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:13.906 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:13.906 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.907 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.907 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:13.907 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:13.907 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:13.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.907 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.907 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:13.907 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:13.907 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2299999999999999e-07 2024-09-14T06:17:13.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.907 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.907 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:13.907 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042350+0000", 2024-09-14T06:17:13.907 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5690000000000004e-06 2024-09-14T06:17:13.908 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.908 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.908 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:13.908 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042361+0000", 2024-09-14T06:17:13.908 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1337e-05 2024-09-14T06:17:13.908 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.908 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.908 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:13.908 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042369+0000", 2024-09-14T06:17:13.908 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8359999999999992e-06 2024-09-14T06:17:13.908 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.908 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.908 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:13.908 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090914+0000", 2024-09-14T06:17:13.908 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048544945999999999 2024-09-14T06:17:13.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.909 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.909 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:13.909 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090995+0000", 2024-09-14T06:17:13.909 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0548000000000002e-05 2024-09-14T06:17:13.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.909 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.909 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:13.909 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091508+0000", 2024-09-14T06:17:13.909 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000513723 2024-09-14T06:17:13.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.909 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.909 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:13.909 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093900+0000", 2024-09-14T06:17:13.909 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023920429999999999 2024-09-14T06:17:13.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.910 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.910 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:13.910 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095944+0000", 2024-09-14T06:17:13.910 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020437379999999998 2024-09-14T06:17:13.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.910 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.910 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:13.910 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095987+0000", 2024-09-14T06:17:13.910 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2521999999999997e-05 2024-09-14T06:17:13.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:13.910 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:13.910 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:13.910 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.096025+0000", 2024-09-14T06:17:13.910 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8649000000000002e-05 2024-09-14T06:17:13.910 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.911 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.911 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.911 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:13.911 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:13.911 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:13.912 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:17:13.913 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:13.909+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:14.014 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:14.010+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:14.041 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:14.041 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:14.041 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:14.042 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:14.055 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:17:14.115 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:14.114+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:14.179 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:14.179 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:14.179 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:14.179 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:14.179 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:14.193 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:17:14.216 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:14.214+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:14.317 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:14.314+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:14.320 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:14.320 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:14.320 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:14.320 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:14.320 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.320 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13229 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.320 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:14.320 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.218319674, 2024-09-14T06:17:14.320 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081798868999999996, 2024-09-14T06:17:14.320 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13229 2024-09-14T06:17:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:14.321 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098710+0000", 2024-09-14T06:17:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8199999999999999e-07 2024-09-14T06:17:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.322 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098714+0000", 2024-09-14T06:17:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:17:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098727+0000", 2024-09-14T06:17:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2520999999999999e-05 2024-09-14T06:17:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098749+0000", 2024-09-14T06:17:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2773999999999999e-05 2024-09-14T06:17:14.323 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160234+0000", 2024-09-14T06:17:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.061484430999999999 2024-09-14T06:17:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160281+0000", 2024-09-14T06:17:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7437e-05 2024-09-14T06:17:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.324 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160569+0000", 2024-09-14T06:17:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028788700000000002 2024-09-14T06:17:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163121+0000", 2024-09-14T06:17:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025514090000000001 2024-09-14T06:17:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180410+0000", 2024-09-14T06:17:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.017289098999999999 2024-09-14T06:17:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.325 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180472+0000", 2024-09-14T06:17:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2547999999999998e-05 2024-09-14T06:17:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180508+0000", 2024-09-14T06:17:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5534000000000002e-05 2024-09-14T06:17:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.326 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13230 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.218238472000003, 2024-09-14T06:17:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081851558000000005, 2024-09-14T06:17:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.327 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13230 2024-09-14T06:17:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.328 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098791+0000", 2024-09-14T06:17:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8800000000000001e-07 2024-09-14T06:17:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098795+0000", 2024-09-14T06:17:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.391e-06 2024-09-14T06:17:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.329 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098804+0000", 2024-09-14T06:17:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3440000000000001e-06 2024-09-14T06:17:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098810+0000", 2024-09-14T06:17:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2339999999999999e-06 2024-09-14T06:17:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160930+0000", 2024-09-14T06:17:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062120018999999999 2024-09-14T06:17:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.330 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161017+0000", 2024-09-14T06:17:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6868999999999998e-05 2024-09-14T06:17:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161300+0000", 2024-09-14T06:17:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028349499999999998 2024-09-14T06:17:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164612+0000", 2024-09-14T06:17:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033119680000000002 2024-09-14T06:17:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.331 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180559+0000", 2024-09-14T06:17:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015946612999999998 2024-09-14T06:17:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180605+0000", 2024-09-14T06:17:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5890000000000003e-05 2024-09-14T06:17:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180642+0000", 2024-09-14T06:17:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:17:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.332 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13231 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.217976057999998, 2024-09-14T06:17:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081750685000000003, 2024-09-14T06:17:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.333 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13231 2024-09-14T06:17:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099053+0000", 2024-09-14T06:17:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:17:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099059+0000", 2024-09-14T06:17:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1789999999999999e-06 2024-09-14T06:17:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.335 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099072+0000", 2024-09-14T06:17:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3179999999999999e-05 2024-09-14T06:17:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099080+0000", 2024-09-14T06:17:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9030000000000004e-06 2024-09-14T06:17:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161681+0000", 2024-09-14T06:17:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062601641999999999 2024-09-14T06:17:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.336 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161766+0000", 2024-09-14T06:17:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4909999999999998e-05 2024-09-14T06:17:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162109+0000", 2024-09-14T06:17:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00034264899999999999 2024-09-14T06:17:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164615+0000", 2024-09-14T06:17:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025061979999999998 2024-09-14T06:17:14.337 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180718+0000", 2024-09-14T06:17:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016103240000000001 2024-09-14T06:17:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180768+0000", 2024-09-14T06:17:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9781999999999999e-05 2024-09-14T06:17:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.338 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180803+0000", 2024-09-14T06:17:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5021999999999998e-05 2024-09-14T06:17:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13232 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.217934284000002, 2024-09-14T06:17:14.339 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081861981, 2024-09-14T06:17:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13232 2024-09-14T06:17:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.340 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.341 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:14.341 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.341 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.341 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.341 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.341 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:14.341 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.341 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.341 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.341 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099095+0000", 2024-09-14T06:17:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:17:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099098+0000", 2024-09-14T06:17:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5420000000000001e-06 2024-09-14T06:17:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099106+0000", 2024-09-14T06:17:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.824e-06 2024-09-14T06:17:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099112+0000", 2024-09-14T06:17:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9789999999999998e-06 2024-09-14T06:17:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162576+0000", 2024-09-14T06:17:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.063463723999999999 2024-09-14T06:17:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.343 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162627+0000", 2024-09-14T06:17:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1530000000000003e-05 2024-09-14T06:17:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162854+0000", 2024-09-14T06:17:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000226159 2024-09-14T06:17:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166090+0000", 2024-09-14T06:17:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032362319999999999 2024-09-14T06:17:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.344 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180877+0000", 2024-09-14T06:17:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014786726 2024-09-14T06:17:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180920+0000", 2024-09-14T06:17:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3427000000000003e-05 2024-09-14T06:17:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180956+0000", 2024-09-14T06:17:14.345 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6195999999999997e-05 2024-09-14T06:17:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13233 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.217909020999997, 2024-09-14T06:17:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081924860000000002, 2024-09-14T06:17:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.346 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13233 2024-09-14T06:17:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.347 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099120+0000", 2024-09-14T06:17:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:17:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099122+0000", 2024-09-14T06:17:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:17:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099126+0000", 2024-09-14T06:17:14.348 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2540000000000003e-06 2024-09-14T06:17:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099130+0000", 2024-09-14T06:17:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4060000000000001e-06 2024-09-14T06:17:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163169+0000", 2024-09-14T06:17:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064039415000000002 2024-09-14T06:17:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.349 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163260+0000", 2024-09-14T06:17:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1038000000000005e-05 2024-09-14T06:17:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163640+0000", 2024-09-14T06:17:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037954699999999999 2024-09-14T06:17:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166093+0000", 2024-09-14T06:17:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024533229999999999 2024-09-14T06:17:14.350 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180993+0000", 2024-09-14T06:17:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01489997 2024-09-14T06:17:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181020+0000", 2024-09-14T06:17:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6806999999999999e-05 2024-09-14T06:17:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181044+0000", 2024-09-14T06:17:14.351 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4405000000000001e-05 2024-09-14T06:17:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13234 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.217866815999997, 2024-09-14T06:17:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082015597999999995, 2024-09-14T06:17:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.352 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13234 2024-09-14T06:17:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:14.353 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0999999999999998e-07 2024-09-14T06:17:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099166+0000", 2024-09-14T06:17:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:17:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.354 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099171+0000", 2024-09-14T06:17:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6949999999999997e-06 2024-09-14T06:17:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099175+0000", 2024-09-14T06:17:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9029999999999997e-06 2024-09-14T06:17:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.355 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164063+0000", 2024-09-14T06:17:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064887608999999999 2024-09-14T06:17:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164111+0000", 2024-09-14T06:17:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7648999999999997e-05 2024-09-14T06:17:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.356 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164351+0000", 2024-09-14T06:17:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024072899999999999 2024-09-14T06:17:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167561+0000", 2024-09-14T06:17:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032094179999999999 2024-09-14T06:17:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181093+0000", 2024-09-14T06:17:14.357 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013532245 2024-09-14T06:17:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181139+0000", 2024-09-14T06:17:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6483999999999997e-05 2024-09-14T06:17:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181177+0000", 2024-09-14T06:17:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7861000000000002e-05 2024-09-14T06:17:14.358 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13235 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.217759542000003, 2024-09-14T06:17:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081988088000000001, 2024-09-14T06:17:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13235 2024-09-14T06:17:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.361 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099270+0000", 2024-09-14T06:17:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7400000000000003e-07 2024-09-14T06:17:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099276+0000", 2024-09-14T06:17:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:17:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099284+0000", 2024-09-14T06:17:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7979999999999995e-06 2024-09-14T06:17:14.362 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099289+0000", 2024-09-14T06:17:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:17:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164664+0000", 2024-09-14T06:17:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065374901999999999 2024-09-14T06:17:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164755+0000", 2024-09-14T06:17:14.363 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0790000000000003e-05 2024-09-14T06:17:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165143+0000", 2024-09-14T06:17:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000387904 2024-09-14T06:17:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167564+0000", 2024-09-14T06:17:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024208989999999998 2024-09-14T06:17:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.364 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181212+0000", 2024-09-14T06:17:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013648664 2024-09-14T06:17:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181237+0000", 2024-09-14T06:17:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5177999999999998e-05 2024-09-14T06:17:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181257+0000", 2024-09-14T06:17:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9660000000000002e-05 2024-09-14T06:17:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.365 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13236 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.217652723, 2024-09-14T06:17:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082010939000000005, 2024-09-14T06:17:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13236 2024-09-14T06:17:14.366 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:14.367 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.61e-07 2024-09-14T06:17:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099382+0000", 2024-09-14T06:17:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3589999999999999e-06 2024-09-14T06:17:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099388+0000", 2024-09-14T06:17:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9619999999999998e-06 2024-09-14T06:17:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099393+0000", 2024-09-14T06:17:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3410000000000002e-06 2024-09-14T06:17:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.369 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165553+0000", 2024-09-14T06:17:14.370 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066159604999999996 2024-09-14T06:17:14.370 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.370 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.370 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.370 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165603+0000", 2024-09-14T06:17:14.370 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0427999999999999e-05 2024-09-14T06:17:14.370 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.370 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.370 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.370 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165827+0000", 2024-09-14T06:17:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022358299999999999 2024-09-14T06:17:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177005+0000", 2024-09-14T06:17:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011178044 2024-09-14T06:17:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181302+0000", 2024-09-14T06:17:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042970669999999999 2024-09-14T06:17:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.371 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181349+0000", 2024-09-14T06:17:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7287999999999999e-05 2024-09-14T06:17:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181387+0000", 2024-09-14T06:17:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7601000000000001e-05 2024-09-14T06:17:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.372 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.373 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.373 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.373 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.373 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13237 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.373 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:14.373 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.217494576, 2024-09-14T06:17:14.373 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081927299999999995, 2024-09-14T06:17:14.373 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.373 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.374 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.374 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.374 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.374 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.374 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13237 2024-09-14T06:17:14.374 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.374 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.374 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.374 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.374 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099535+0000", 2024-09-14T06:17:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:17:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.375 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099538+0000", 2024-09-14T06:17:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:17:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099545+0000", 2024-09-14T06:17:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:17:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099551+0000", 2024-09-14T06:17:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1770000000000003e-06 2024-09-14T06:17:14.376 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166131+0000", 2024-09-14T06:17:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066580499000000001 2024-09-14T06:17:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166221+0000", 2024-09-14T06:17:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0000000000000006e-05 2024-09-14T06:17:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.377 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166622+0000", 2024-09-14T06:17:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040029799999999999 2024-09-14T06:17:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177008+0000", 2024-09-14T06:17:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010386772000000001 2024-09-14T06:17:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181420+0000", 2024-09-14T06:17:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044119190000000003 2024-09-14T06:17:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.378 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181443+0000", 2024-09-14T06:17:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2603999999999999e-05 2024-09-14T06:17:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181461+0000", 2024-09-14T06:17:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8405000000000001e-05 2024-09-14T06:17:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.379 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13238 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.217458268000001, 2024-09-14T06:17:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082014927000000001, 2024-09-14T06:17:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.380 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.381 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-09-14T06:17:14.382 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.382 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.382 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13238 2024-09-14T06:17:14.382 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.382 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.382 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.382 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099571+0000", 2024-09-14T06:17:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9699999999999996e-07 2024-09-14T06:17:14.383 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099577+0000", 2024-09-14T06:17:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2890000000000003e-06 2024-09-14T06:17:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099587+0000", 2024-09-14T06:17:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0002e-05 2024-09-14T06:17:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099593+0000", 2024-09-14T06:17:14.384 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4539999999999998e-06 2024-09-14T06:17:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167024+0000", 2024-09-14T06:17:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067431480000000002 2024-09-14T06:17:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167074+0000", 2024-09-14T06:17:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9996e-05 2024-09-14T06:17:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.385 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167288+0000", 2024-09-14T06:17:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021410900000000001 2024-09-14T06:17:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177997+0000", 2024-09-14T06:17:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010708756 2024-09-14T06:17:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.386 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181508+0000", 2024-09-14T06:17:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003511363 2024-09-14T06:17:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181552+0000", 2024-09-14T06:17:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3609999999999998e-05 2024-09-14T06:17:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.387 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181585+0000", 2024-09-14T06:17:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3271e-05 2024-09-14T06:17:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13239 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.21736645, 2024-09-14T06:17:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081998772999999997, 2024-09-14T06:17:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.388 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13239 2024-09-14T06:17:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.389 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099663+0000", 2024-09-14T06:17:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6100000000000001e-07 2024-09-14T06:17:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099666+0000", 2024-09-14T06:17:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.056e-06 2024-09-14T06:17:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099672+0000", 2024-09-14T06:17:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8120000000000002e-06 2024-09-14T06:17:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099676+0000", 2024-09-14T06:17:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:17:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.391 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.392 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167602+0000", 2024-09-14T06:17:14.392 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067925867000000001 2024-09-14T06:17:14.392 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.392 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.392 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.392 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167693+0000", 2024-09-14T06:17:14.392 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0736000000000007e-05 2024-09-14T06:17:14.392 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168065+0000", 2024-09-14T06:17:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037245399999999998 2024-09-14T06:17:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178003+0000", 2024-09-14T06:17:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0099372360000000003 2024-09-14T06:17:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.393 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181621+0000", 2024-09-14T06:17:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036178489999999998 2024-09-14T06:17:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181643+0000", 2024-09-14T06:17:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2155e-05 2024-09-14T06:17:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181661+0000", 2024-09-14T06:17:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8147000000000001e-05 2024-09-14T06:17:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.394 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13240 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.217247356999998, 2024-09-14T06:17:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081942607000000001, 2024-09-14T06:17:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.395 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13240 2024-09-14T06:17:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099782+0000", 2024-09-14T06:17:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3900000000000004e-07 2024-09-14T06:17:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.397 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099787+0000", 2024-09-14T06:17:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6189999999999999e-06 2024-09-14T06:17:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099793+0000", 2024-09-14T06:17:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.998e-06 2024-09-14T06:17:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099797+0000", 2024-09-14T06:17:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8449999999999996e-06 2024-09-14T06:17:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.398 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168494+0000", 2024-09-14T06:17:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.068696500999999993 2024-09-14T06:17:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168590+0000", 2024-09-14T06:17:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6733999999999999e-05 2024-09-14T06:17:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168973+0000", 2024-09-14T06:17:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038229399999999998 2024-09-14T06:17:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178013+0000", 2024-09-14T06:17:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0090397199999999994 2024-09-14T06:17:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181683+0000", 2024-09-14T06:17:14.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036705819999999999 2024-09-14T06:17:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181705+0000", 2024-09-14T06:17:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1551000000000001e-05 2024-09-14T06:17:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.401 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181724+0000", 2024-09-14T06:17:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9123999999999999e-05 2024-09-14T06:17:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13243 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.216713626999997, 2024-09-14T06:17:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081767815999999993, 2024-09-14T06:17:14.402 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13243 2024-09-14T06:17:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.404 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.404 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:14.404 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.404 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.404 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:14.405 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1e-07 2024-09-14T06:17:14.405 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.405 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.405 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.405 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100317+0000", 2024-09-14T06:17:14.405 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.092e-06 2024-09-14T06:17:14.405 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.405 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100321+0000", 2024-09-14T06:17:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7809999999999999e-06 2024-09-14T06:17:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100324+0000", 2024-09-14T06:17:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.117e-06 2024-09-14T06:17:14.406 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171113+0000", 2024-09-14T06:17:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.070788816000000004 2024-09-14T06:17:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171202+0000", 2024-09-14T06:17:14.407 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8843000000000003e-05 2024-09-14T06:17:14.408 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.408 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.408 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.408 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171584+0000", 2024-09-14T06:17:14.408 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038231099999999998 2024-09-14T06:17:14.408 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.408 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.408 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.408 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178101+0000", 2024-09-14T06:17:14.409 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0065168329999999997 2024-09-14T06:17:14.409 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.409 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.409 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.409 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182041+0000", 2024-09-14T06:17:14.409 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039401810000000001 2024-09-14T06:17:14.409 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.409 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.410 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.410 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182065+0000", 2024-09-14T06:17:14.410 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3623000000000001e-05 2024-09-14T06:17:14.410 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.410 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.410 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.410 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182083+0000", 2024-09-14T06:17:14.410 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7909000000000001e-05 2024-09-14T06:17:14.411 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.411 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.411 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.411 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.411 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.411 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13244 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.411 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:14.411 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.216698807, 2024-09-14T06:17:14.411 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081837800000000002, 2024-09-14T06:17:14.412 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.412 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.412 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.412 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.412 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.412 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.412 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13244 2024-09-14T06:17:14.412 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.413 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.413 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.413 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.413 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:14.413 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.413 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.413 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.413 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.414 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:14.414 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.414 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.414 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.414 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.414 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:14.414 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:17:14.414 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.414 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.415 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.415 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100337+0000", 2024-09-14T06:17:14.415 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.939e-06 2024-09-14T06:17:14.415 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.415 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.415 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.415 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100341+0000", 2024-09-14T06:17:14.415 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2320000000000003e-06 2024-09-14T06:17:14.416 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.416 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.416 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.416 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100344+0000", 2024-09-14T06:17:14.416 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0759999999999999e-06 2024-09-14T06:17:14.416 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.416 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.416 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.417 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171996+0000", 2024-09-14T06:17:14.417 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.071651919999999994 2024-09-14T06:17:14.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.417 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.417 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.417 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172079+0000", 2024-09-14T06:17:14.417 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2579999999999999e-05 2024-09-14T06:17:14.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172457+0000", 2024-09-14T06:17:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037785100000000001 2024-09-14T06:17:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178104+0000", 2024-09-14T06:17:14.418 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056471730000000001 2024-09-14T06:17:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182123+0000", 2024-09-14T06:17:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040190019999999998 2024-09-14T06:17:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182147+0000", 2024-09-14T06:17:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4241000000000001e-05 2024-09-14T06:17:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.419 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182168+0000", 2024-09-14T06:17:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0492999999999998e-05 2024-09-14T06:17:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13245 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.216679300000003, 2024-09-14T06:17:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081893899000000006, 2024-09-14T06:17:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.420 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13245 2024-09-14T06:17:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.421 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100350+0000", 2024-09-14T06:17:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5499999999999999e-07 2024-09-14T06:17:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.422 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100353+0000", 2024-09-14T06:17:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:17:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100357+0000", 2024-09-14T06:17:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8500000000000004e-06 2024-09-14T06:17:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100360+0000", 2024-09-14T06:17:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8700000000000001e-06 2024-09-14T06:17:14.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172862+0000", 2024-09-14T06:17:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.072501992000000001 2024-09-14T06:17:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172950+0000", 2024-09-14T06:17:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8988999999999998e-05 2024-09-14T06:17:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173324+0000", 2024-09-14T06:17:14.424 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037366199999999997 2024-09-14T06:17:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178107+0000", 2024-09-14T06:17:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047830520000000003 2024-09-14T06:17:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182202+0000", 2024-09-14T06:17:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040949280000000003 2024-09-14T06:17:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182226+0000", 2024-09-14T06:17:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3535000000000001e-05 2024-09-14T06:17:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182243+0000", 2024-09-14T06:17:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7445e-05 2024-09-14T06:17:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.426 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.427 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13246 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.427 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:14.427 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.216664977999997, 2024-09-14T06:17:14.427 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081962276000000001, 2024-09-14T06:17:14.427 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.427 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.427 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13246 2024-09-14T06:17:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:14.428 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0800000000000001e-07 2024-09-14T06:17:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.429 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100367+0000", 2024-09-14T06:17:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8439999999999998e-06 2024-09-14T06:17:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100372+0000", 2024-09-14T06:17:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4009999999999996e-06 2024-09-14T06:17:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100378+0000", 2024-09-14T06:17:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7180000000000001e-06 2024-09-14T06:17:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.430 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173780+0000", 2024-09-14T06:17:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.073401991 2024-09-14T06:17:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173879+0000", 2024-09-14T06:17:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9449000000000003e-05 2024-09-14T06:17:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174240+0000", 2024-09-14T06:17:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036118100000000002 2024-09-14T06:17:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.431 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178109+0000", 2024-09-14T06:17:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038690619999999999 2024-09-14T06:17:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182281+0000", 2024-09-14T06:17:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041715900000000002 2024-09-14T06:17:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182306+0000", 2024-09-14T06:17:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4477e-05 2024-09-14T06:17:14.432 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.433 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.433 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.433 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182326+0000", 2024-09-14T06:17:14.433 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0254999999999998e-05 2024-09-14T06:17:14.433 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.433 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.433 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.433 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.433 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.433 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13247 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.433 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.216567800999997, 2024-09-14T06:17:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966593000000004, 2024-09-14T06:17:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13247 2024-09-14T06:17:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.434 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100462+0000", 2024-09-14T06:17:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9800000000000002e-07 2024-09-14T06:17:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100466+0000", 2024-09-14T06:17:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4039999999999997e-06 2024-09-14T06:17:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100475+0000", 2024-09-14T06:17:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3680000000000008e-06 2024-09-14T06:17:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.436 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100482+0000", 2024-09-14T06:17:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8630000000000002e-06 2024-09-14T06:17:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174669+0000", 2024-09-14T06:17:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074186849999999999 2024-09-14T06:17:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174758+0000", 2024-09-14T06:17:14.437 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9021e-05 2024-09-14T06:17:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175137+0000", 2024-09-14T06:17:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378511 2024-09-14T06:17:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178158+0000", 2024-09-14T06:17:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003020956 2024-09-14T06:17:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.438 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182386+0000", 2024-09-14T06:17:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042278840000000003 2024-09-14T06:17:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182410+0000", 2024-09-14T06:17:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4026e-05 2024-09-14T06:17:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182427+0000", 2024-09-14T06:17:14.439 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7711999999999999e-05 2024-09-14T06:17:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13248 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.216485720000001, 2024-09-14T06:17:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966527999999997, 2024-09-14T06:17:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.440 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13248 2024-09-14T06:17:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.441 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:17:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100548+0000", 2024-09-14T06:17:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:17:14.442 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100557+0000", 2024-09-14T06:17:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:17:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100563+0000", 2024-09-14T06:17:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2519999999999996e-06 2024-09-14T06:17:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.443 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175572+0000", 2024-09-14T06:17:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075008459 2024-09-14T06:17:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175656+0000", 2024-09-14T06:17:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3906999999999995e-05 2024-09-14T06:17:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176010+0000", 2024-09-14T06:17:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00035447499999999998 2024-09-14T06:17:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.444 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178304+0000", 2024-09-14T06:17:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0022941379999999998 2024-09-14T06:17:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182467+0000", 2024-09-14T06:17:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041632020000000004 2024-09-14T06:17:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182491+0000", 2024-09-14T06:17:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3733000000000001e-05 2024-09-14T06:17:14.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182509+0000", 2024-09-14T06:17:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8113000000000001e-05 2024-09-14T06:17:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.446 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13249 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.216450786000003, 2024-09-14T06:17:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082037225000000005, 2024-09-14T06:17:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13249 2024-09-14T06:17:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.447 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4600000000000005e-07 2024-09-14T06:17:14.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100584+0000", 2024-09-14T06:17:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.947e-06 2024-09-14T06:17:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100593+0000", 2024-09-14T06:17:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9760000000000001e-06 2024-09-14T06:17:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100597+0000", 2024-09-14T06:17:14.449 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9689999999999996e-06 2024-09-14T06:17:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176461+0000", 2024-09-14T06:17:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075864132000000001 2024-09-14T06:17:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176516+0000", 2024-09-14T06:17:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4416999999999998e-05 2024-09-14T06:17:14.450 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176734+0000", 2024-09-14T06:17:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021859399999999999 2024-09-14T06:17:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178308+0000", 2024-09-14T06:17:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015733660000000001 2024-09-14T06:17:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.451 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182573+0000", 2024-09-14T06:17:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042649200000000002 2024-09-14T06:17:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182597+0000", 2024-09-14T06:17:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4355e-05 2024-09-14T06:17:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182615+0000", 2024-09-14T06:17:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8003e-05 2024-09-14T06:17:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.452 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13250 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.216286181000001, 2024-09-14T06:17:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081983890000000004, 2024-09-14T06:17:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.453 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13250 2024-09-14T06:17:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.454 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100743+0000", 2024-09-14T06:17:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:17:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100753+0000", 2024-09-14T06:17:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9979999999999998e-06 2024-09-14T06:17:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.455 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100772+0000", 2024-09-14T06:17:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8664e-05 2024-09-14T06:17:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100779+0000", 2024-09-14T06:17:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.046e-06 2024-09-14T06:17:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177049+0000", 2024-09-14T06:17:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076270819000000004 2024-09-14T06:17:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177140+0000", 2024-09-14T06:17:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0746000000000002e-05 2024-09-14T06:17:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177519+0000", 2024-09-14T06:17:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378656 2024-09-14T06:17:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178349+0000", 2024-09-14T06:17:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00083061599999999995 2024-09-14T06:17:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182679+0000", 2024-09-14T06:17:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043298379999999999 2024-09-14T06:17:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182705+0000", 2024-09-14T06:17:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5602999999999999e-05 2024-09-14T06:17:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.458 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.459 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.459 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182726+0000", 2024-09-14T06:17:14.459 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1297e-05 2024-09-14T06:17:14.459 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.459 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.459 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.459 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.459 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.459 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:14.460 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:17:14.461 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:14.458+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:14.462 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone21 from 0x3741ffe (overlap 0x3741ffe) to 0x2127998 2024-09-14T06:17:14.562 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:14.558+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:14.596 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:14.597 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:14.597 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:14.597 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:14.610 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:17:14.611 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-14T06:17:14.638 INFO:teuthology.orchestra.run.smithi039.stdout:3903 read 0x1858895 thru 0x186054e (0x7cba bytes) 2024-09-14T06:17:14.658 INFO:teuthology.orchestra.run.smithi039.stdout:3906 write 0x633f663 thru 0x63441cf (0x4b6d bytes) 2024-09-14T06:17:14.662 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:14.657+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:14.686 INFO:teuthology.orchestra.run.smithi039.stdout:3907 writesame 0xc437880 thru 0xc43a697 data_size 0x28(0x2e18 bytes) 2024-09-14T06:17:14.686 INFO:teuthology.orchestra.run.smithi039.stdout:3908 writesame 0xc0146ea thru 0xc01cbb0 data_size 0x1(0x84c7 bytes) 2024-09-14T06:17:14.708 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:14.708 INFO:teuthology.orchestra.run.smithi039.stdout:3911 read 0x526b566 thru 0x52776a5 (0xc140 bytes) 2024-09-14T06:17:14.720 INFO:teuthology.orchestra.run.smithi039.stdout:3913 writesame 0xe93799f thru 0xe93fdee data_size 0x10(0x8450 bytes) 2024-09-14T06:17:14.720 INFO:teuthology.orchestra.run.smithi039.stdout:3914 read 0x85a4e76 thru 0x85b1ec7 (0xd052 bytes) 2024-09-14T06:17:14.720 INFO:teuthology.orchestra.run.smithi039.stdout:3915 write 0x2b457a5 thru 0x2b4bcb9 (0x6515 bytes) 2024-09-14T06:17:14.726 INFO:teuthology.orchestra.run.smithi039.stdout:3916 punch from 0x6566258 to 0x656aa2c, (0x47d4 bytes) 2024-09-14T06:17:14.726 INFO:teuthology.orchestra.run.smithi039.stdout:3917 write 0xb44f966 thru 0xb45191a (0x1fb5 bytes) 2024-09-14T06:17:14.732 INFO:teuthology.orchestra.run.smithi039.stdout:3918 writesame 0xa4240d1 thru 0xa42d0d3 data_size 0x1(0x9003 bytes) 2024-09-14T06:17:14.735 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:14.735 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:14.735 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:14.735 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:14.735 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:14.753 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:17:14.762 INFO:teuthology.orchestra.run.smithi039.stdout:3919 read 0x981766b thru 0x9818180 (0xb16 bytes) 2024-09-14T06:17:14.762 INFO:teuthology.orchestra.run.smithi039.stdout:3920 write 0xb1d25ca thru 0xb1dde31 (0xb868 bytes) 2024-09-14T06:17:14.763 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:14.758+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:14.767 INFO:teuthology.orchestra.run.smithi039.stdout:3922 write 0x8a02012 thru 0x8a07800 (0x57ef bytes) 2024-09-14T06:17:14.771 INFO:teuthology.orchestra.run.smithi039.stdout:3923 writesame 0x5d2b29c thru 0x5d38810 data_size 0x5(0xd575 bytes) 2024-09-14T06:17:14.779 INFO:teuthology.orchestra.run.smithi039.stdout:3924 write 0x64f89a9 thru 0x64fa0c3 (0x171b bytes) 2024-09-14T06:17:14.783 INFO:teuthology.orchestra.run.smithi039.stdout:3925 punch from 0x7fc7114 to 0x7fc8882, (0x176e bytes) 2024-09-14T06:17:14.783 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:14.790 INFO:teuthology.orchestra.run.smithi039.stdout:3928 write 0xed7422d thru 0xed7e1b0 (0x9f84 bytes) 2024-09-14T06:17:14.794 INFO:teuthology.orchestra.run.smithi039.stdout:3930 write 0x351a3f1 thru 0x3525243 (0xae53 bytes) 2024-09-14T06:17:14.801 INFO:teuthology.orchestra.run.smithi039.stdout:3932 write 0x6af40f0 thru 0x6aff9f6 (0xb907 bytes) 2024-09-14T06:17:14.805 INFO:teuthology.orchestra.run.smithi039.stdout:3933 punch from 0x87b2955 to 0x87b3033, (0x6de bytes) 2024-09-14T06:17:14.806 INFO:teuthology.orchestra.run.smithi039.stdout:3934 write 0xba3615c thru 0xba3e030 (0x7ed5 bytes) 2024-09-14T06:17:14.810 INFO:teuthology.orchestra.run.smithi039.stdout:3935 write 0x787b398 thru 0x78899ca (0xe633 bytes) 2024-09-14T06:17:14.816 INFO:teuthology.orchestra.run.smithi039.stdout:3936 punch from 0xbf23f24 to 0xbf2d404, (0x94e0 bytes) 2024-09-14T06:17:14.816 INFO:teuthology.orchestra.run.smithi039.stdout:3937 read 0x9d5f74c thru 0x9d6f200 (0xfab5 bytes) 2024-09-14T06:17:14.816 INFO:teuthology.orchestra.run.smithi039.stdout:3939 write 0x11758d1 thru 0x11843d2 (0xeb02 bytes) 2024-09-14T06:17:14.834 INFO:teuthology.orchestra.run.smithi039.stdout:3940 trunc from 0xed7e1b1 to 0x72304ff 2024-09-14T06:17:14.849 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:14.864 INFO:teuthology.orchestra.run.smithi039.stdout:3942 write 0xbc91aec thru 0xbca052a (0xea3f bytes) 2024-09-14T06:17:14.864 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:14.862+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:14.869 INFO:teuthology.orchestra.run.smithi039.stdout:3943 writesame 0xafb9184 thru 0xafc4df1 data_size 0x2(0xbc6e bytes) 2024-09-14T06:17:14.880 INFO:teuthology.orchestra.run.smithi039.stdout:3945 punch from 0x4a18201 to 0x4a19355, (0x1154 bytes) 2024-09-14T06:17:14.880 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:14.880 INFO:teuthology.orchestra.run.smithi039.stdout:3947 punch from 0x274e55e to 0x2754d79, (0x681b bytes) 2024-09-14T06:17:14.880 INFO:teuthology.orchestra.run.smithi039.stdout:3948 write 0x84eda0f thru 0x84f6197 (0x8789 bytes) 2024-09-14T06:17:14.890 INFO:teuthology.orchestra.run.smithi039.stdout:3949 trunc from 0xbca052b to 0x83a6991 2024-09-14T06:17:14.905 INFO:teuthology.orchestra.run.smithi039.stdout:3950 punch from 0x4acda5f to 0x4ad58cb, (0x7e6c bytes) 2024-09-14T06:17:14.905 INFO:teuthology.orchestra.run.smithi039.stdout:3951 read 0x3ad768a thru 0x3ae14b5 (0x9e2c bytes) 2024-09-14T06:17:14.917 INFO:teuthology.orchestra.run.smithi039.stdout:3952 write 0x91f4c66 thru 0x91f6894 (0x1c2f bytes) 2024-09-14T06:17:14.923 INFO:teuthology.orchestra.run.smithi039.stdout:3953 write 0x7bbd9d0 thru 0x7bc8049 (0xa67a bytes) 2024-09-14T06:17:14.931 INFO:teuthology.orchestra.run.smithi039.stdout:3955 write 0x3fd9dfb thru 0x3fe7e13 (0xe019 bytes) 2024-09-14T06:17:14.936 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:14.936 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:14.936 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:30547 3.1f 3:f84dfb53:::rbd_data.11529eb229bd.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write 16226533~50642] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T06:17:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.745057045999999, 2024-09-14T06:17:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0079770020000001, 2024-09-14T06:17:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 30547 2024-09-14T06:17:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189811+0000", 2024-09-14T06:17:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2060000000000001e-06 2024-09-14T06:17:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189871+0000", 2024-09-14T06:17:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9862999999999999e-05 2024-09-14T06:17:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189899+0000", 2024-09-14T06:17:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7849000000000001e-05 2024-09-14T06:17:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189915+0000", 2024-09-14T06:17:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:17:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190024+0000", 2024-09-14T06:17:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000109202 2024-09-14T06:17:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:17:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190041+0000", 2024-09-14T06:17:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7665000000000001e-05 2024-09-14T06:17:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188512+0000", 2024-09-14T06:17:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.99847100700000002 2024-09-14T06:17:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for missing object", 2024-09-14T06:17:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188646+0000", 2024-09-14T06:17:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000133986 2024-09-14T06:17:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193097+0000", 2024-09-14T06:17:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044507059999999996 2024-09-14T06:17:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193183+0000", 2024-09-14T06:17:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5496999999999998e-05 2024-09-14T06:17:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:17:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193307+0000", 2024-09-14T06:17:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000124611 2024-09-14T06:17:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.194318+0000", 2024-09-14T06:17:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0010106 2024-09-14T06:17:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197670+0000", 2024-09-14T06:17:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033523170000000001 2024-09-14T06:17:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197712+0000", 2024-09-14T06:17:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1681000000000003e-05 2024-09-14T06:17:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197786+0000", 2024-09-14T06:17:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4148999999999998e-05 2024-09-14T06:17:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32022 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 11.066676835000001, 2024-09-14T06:17:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095598104000000003, 2024-09-14T06:17:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32022 2024-09-14T06:17:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868190+0000", 2024-09-14T06:17:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6000000000000003e-07 2024-09-14T06:17:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868243+0000", 2024-09-14T06:17:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3443999999999999e-05 2024-09-14T06:17:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868260+0000", 2024-09-14T06:17:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6799e-05 2024-09-14T06:17:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868271+0000", 2024-09-14T06:17:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0523000000000001e-05 2024-09-14T06:17:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963502+0000", 2024-09-14T06:17:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095231466000000001 2024-09-14T06:17:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963636+0000", 2024-09-14T06:17:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013388699999999999 2024-09-14T06:17:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963787+0000", 2024-09-14T06:17:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000151225 2024-09-14T06:17:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32023 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5816320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 11.066581708999999, 2024-09-14T06:17:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095879753999999998, 2024-09-14T06:17:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32023 2024-09-14T06:17:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868285+0000", 2024-09-14T06:17:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6299999999999993e-07 2024-09-14T06:17:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868288+0000", 2024-09-14T06:17:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.18e-06 2024-09-14T06:17:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868298+0000", 2024-09-14T06:17:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0029e-05 2024-09-14T06:17:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868307+0000", 2024-09-14T06:17:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7229999999999993e-06 2024-09-14T06:17:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963871+0000", 2024-09-14T06:17:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095563876000000006 2024-09-14T06:17:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964015+0000", 2024-09-14T06:17:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143573 2024-09-14T06:17:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964164+0000", 2024-09-14T06:17:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014940999999999999 2024-09-14T06:17:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32024 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 11.066548565, 2024-09-14T06:17:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096230231999999999, 2024-09-14T06:17:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32024 2024-09-14T06:17:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868318+0000", 2024-09-14T06:17:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1699999999999997e-07 2024-09-14T06:17:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868321+0000", 2024-09-14T06:17:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6769999999999999e-06 2024-09-14T06:17:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868334+0000", 2024-09-14T06:17:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2928999999999999e-05 2024-09-14T06:17:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868341+0000", 2024-09-14T06:17:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9800000000000001e-06 2024-09-14T06:17:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964258+0000", 2024-09-14T06:17:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095917099000000006 2024-09-14T06:17:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964392+0000", 2024-09-14T06:17:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013396999999999999 2024-09-14T06:17:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964548+0000", 2024-09-14T06:17:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015585999999999999 2024-09-14T06:17:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32025 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 11.066515909, 2024-09-14T06:17:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096576591000000003, 2024-09-14T06:17:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32025 2024-09-14T06:17:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868351+0000", 2024-09-14T06:17:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1600000000000001e-07 2024-09-14T06:17:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868353+0000", 2024-09-14T06:17:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.627e-06 2024-09-14T06:17:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868364+0000", 2024-09-14T06:17:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0893000000000001e-05 2024-09-14T06:17:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868381+0000", 2024-09-14T06:17:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6769000000000002e-05 2024-09-14T06:17:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964632+0000", 2024-09-14T06:17:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096250589999999997 2024-09-14T06:17:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964772+0000", 2024-09-14T06:17:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014045299999999999 2024-09-14T06:17:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964927+0000", 2024-09-14T06:17:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015454299999999999 2024-09-14T06:17:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32026 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 11.066474958000001, 2024-09-14T06:17:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096901622000000007, 2024-09-14T06:17:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32026 2024-09-14T06:17:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868392+0000", 2024-09-14T06:17:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0699999999999996e-07 2024-09-14T06:17:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868395+0000", 2024-09-14T06:17:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9859999999999999e-06 2024-09-14T06:17:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868405+0000", 2024-09-14T06:17:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9229999999999997e-06 2024-09-14T06:17:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868412+0000", 2024-09-14T06:17:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9670000000000004e-06 2024-09-14T06:17:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965010+0000", 2024-09-14T06:17:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096598562999999998 2024-09-14T06:17:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965143+0000", 2024-09-14T06:17:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013247700000000001 2024-09-14T06:17:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965293+0000", 2024-09-14T06:17:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000149899 2024-09-14T06:17:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32027 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 11.066406621, 2024-09-14T06:17:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097211306999999997, 2024-09-14T06:17:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32027 2024-09-14T06:17:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868461+0000", 2024-09-14T06:17:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.296e-06 2024-09-14T06:17:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868465+0000", 2024-09-14T06:17:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9570000000000004e-06 2024-09-14T06:17:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868476+0000", 2024-09-14T06:17:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1365999999999999e-05 2024-09-14T06:17:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868483+0000", 2024-09-14T06:17:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2529999999999998e-06 2024-09-14T06:17:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965375+0000", 2024-09-14T06:17:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096891973000000006 2024-09-14T06:17:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965518+0000", 2024-09-14T06:17:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143017 2024-09-14T06:17:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965671+0000", 2024-09-14T06:17:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015244499999999999 2024-09-14T06:17:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32028 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 11.066371793, 2024-09-14T06:17:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097841480999999994, 2024-09-14T06:17:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32028 2024-09-14T06:17:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868495+0000", 2024-09-14T06:17:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6400000000000001e-07 2024-09-14T06:17:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868498+0000", 2024-09-14T06:17:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7760000000000002e-06 2024-09-14T06:17:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868507+0000", 2024-09-14T06:17:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1530000000000003e-06 2024-09-14T06:17:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868516+0000", 2024-09-14T06:17:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8929999999999994e-06 2024-09-14T06:17:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965753+0000", 2024-09-14T06:17:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097236970000000006 2024-09-14T06:17:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965890+0000", 2024-09-14T06:17:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013677599999999999 2024-09-14T06:17:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966336+0000", 2024-09-14T06:17:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00044614900000000001 2024-09-14T06:17:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32029 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 11.066340717999999, 2024-09-14T06:17:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098577779000000004, 2024-09-14T06:17:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32029 2024-09-14T06:17:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868526+0000", 2024-09-14T06:17:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9300000000000006e-07 2024-09-14T06:17:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868529+0000", 2024-09-14T06:17:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5969999999999999e-06 2024-09-14T06:17:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868538+0000", 2024-09-14T06:17:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1700000000000003e-06 2024-09-14T06:17:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868560+0000", 2024-09-14T06:17:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1596000000000001e-05 2024-09-14T06:17:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966423+0000", 2024-09-14T06:17:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097863396000000005 2024-09-14T06:17:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966578+0000", 2024-09-14T06:17:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000154792 2024-09-14T06:17:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967103+0000", 2024-09-14T06:17:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00052523499999999998 2024-09-14T06:17:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32030 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 11.066295800000001, 2024-09-14T06:17:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099324030999999993, 2024-09-14T06:17:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32030 2024-09-14T06:17:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868571+0000", 2024-09-14T06:17:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9699999999999995e-07 2024-09-14T06:17:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868574+0000", 2024-09-14T06:17:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9529999999999999e-06 2024-09-14T06:17:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868584+0000", 2024-09-14T06:17:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7860000000000008e-06 2024-09-14T06:17:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868591+0000", 2024-09-14T06:17:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0500000000000003e-06 2024-09-14T06:17:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967191+0000", 2024-09-14T06:17:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098600035000000003 2024-09-14T06:17:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967333+0000", 2024-09-14T06:17:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014190599999999999 2024-09-14T06:17:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967894+0000", 2024-09-14T06:17:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000561504 2024-09-14T06:17:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32031 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 11.066266238000001, 2024-09-14T06:17:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.10010279, 2024-09-14T06:17:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32031 2024-09-14T06:17:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:17:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868603+0000", 2024-09-14T06:17:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.599e-06 2024-09-14T06:17:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868612+0000", 2024-09-14T06:17:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.986e-06 2024-09-14T06:17:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868619+0000", 2024-09-14T06:17:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8900000000000001e-06 2024-09-14T06:17:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967983+0000", 2024-09-14T06:17:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099363612000000004 2024-09-14T06:17:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968140+0000", 2024-09-14T06:17:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015788600000000001 2024-09-14T06:17:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968703+0000", 2024-09-14T06:17:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00056208300000000001 2024-09-14T06:17:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32032 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6258688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 11.066237763, 2024-09-14T06:17:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100488088, 2024-09-14T06:17:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32032 2024-09-14T06:17:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868629+0000", 2024-09-14T06:17:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:17:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868631+0000", 2024-09-14T06:17:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4099999999999998e-06 2024-09-14T06:17:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868640+0000", 2024-09-14T06:17:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0939999999999997e-06 2024-09-14T06:17:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868647+0000", 2024-09-14T06:17:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9460000000000001e-06 2024-09-14T06:17:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968797+0000", 2024-09-14T06:17:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100149867 2024-09-14T06:17:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968937+0000", 2024-09-14T06:17:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013981399999999999 2024-09-14T06:17:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969116+0000", 2024-09-14T06:17:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017927399999999999 2024-09-14T06:17:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32044 3.1e 3:7e6619d4:::rbd_data.115274ed0aa8.0000000000000068:head [set-alloc-hint object_size 1048576 write_size 1048576,write 835584~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 11.058548679999999, 2024-09-14T06:17:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096039367, 2024-09-14T06:17:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32044 2024-09-14T06:17:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876320+0000", 2024-09-14T06:17:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8270000000000002e-06 2024-09-14T06:17:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876397+0000", 2024-09-14T06:17:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6985999999999998e-05 2024-09-14T06:17:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876431+0000", 2024-09-14T06:17:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3719e-05 2024-09-14T06:17:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876452+0000", 2024-09-14T06:17:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0891999999999999e-05 2024-09-14T06:17:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969186+0000", 2024-09-14T06:17:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.092734681999999999 2024-09-14T06:17:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969550+0000", 2024-09-14T06:17:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036402199999999998 2024-09-14T06:17:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:17:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969987+0000", 2024-09-14T06:17:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000436759 2024-09-14T06:17:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971569+0000", 2024-09-14T06:17:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001581821 2024-09-14T06:17:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971947+0000", 2024-09-14T06:17:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378446 2024-09-14T06:17:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972146+0000", 2024-09-14T06:17:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019819 2024-09-14T06:17:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972357+0000", 2024-09-14T06:17:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021102299999999999 2024-09-14T06:17:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40252 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 827392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 5.2179125739999996, 2024-09-14T06:17:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095633474999999996, 2024-09-14T06:17:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40252 2024-09-14T06:17:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717376+0000", 2024-09-14T06:17:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000422216 2024-09-14T06:17:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717386+0000", 2024-09-14T06:17:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0396e-05 2024-09-14T06:17:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717395+0000", 2024-09-14T06:17:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7819999999999999e-06 2024-09-14T06:17:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717400+0000", 2024-09-14T06:17:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1599999999999997e-06 2024-09-14T06:17:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.811922+0000", 2024-09-14T06:17:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.094522018999999999 2024-09-14T06:17:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812007+0000", 2024-09-14T06:17:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5328999999999996e-05 2024-09-14T06:17:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812587+0000", 2024-09-14T06:17:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000579573 2024-09-14T06:17:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40254 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 5.2174544010000004, 2024-09-14T06:17:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095562731999999997, 2024-09-14T06:17:15.004 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:15.004 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:15.004 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:15.004 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:15.004 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:15.004 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:15.004 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40254 2024-09-14T06:17:15.004 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.004 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:15.004 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.004 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:15.004 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:15.004 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:15.004 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.005 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.005 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:15.005 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:15.005 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:15.005 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.005 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.005 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:15.005 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:15.005 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1699999999999999e-07 2024-09-14T06:17:15.005 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.005 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.005 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:15.005 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717414+0000", 2024-09-14T06:17:15.005 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:17:15.005 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.006 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.006 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:15.006 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717419+0000", 2024-09-14T06:17:15.006 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2839999999999997e-06 2024-09-14T06:17:15.006 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.006 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.006 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:15.006 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717424+0000", 2024-09-14T06:17:15.006 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1549999999999998e-06 2024-09-14T06:17:15.006 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.006 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.006 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:15.006 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812619+0000", 2024-09-14T06:17:15.006 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095194940000000006 2024-09-14T06:17:15.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.007 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.007 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:15.007 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812717+0000", 2024-09-14T06:17:15.007 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7647999999999997e-05 2024-09-14T06:17:15.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.007 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.007 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:15.007 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812974+0000", 2024-09-14T06:17:15.007 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025752099999999998 2024-09-14T06:17:15.007 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.007 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:15.007 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.007 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.007 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40256 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 843776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:15.008 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:15.008 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 5.2174317969999997, 2024-09-14T06:17:15.008 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095932066999999996, 2024-09-14T06:17:15.008 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:15.008 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:15.008 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:15.008 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:15.008 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:15.008 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:15.008 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40256 2024-09-14T06:17:15.008 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.008 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:15.008 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.009 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:15.009 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:15.009 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:15.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.009 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.009 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:15.009 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:15.009 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:15.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.009 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.009 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:15.009 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717628+0000", 2024-09-14T06:17:15.009 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019339499999999999 2024-09-14T06:17:15.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.010 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.010 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:15.010 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717632+0000", 2024-09-14T06:17:15.010 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2230000000000001e-06 2024-09-14T06:17:15.010 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.010 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.010 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:15.010 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717641+0000", 2024-09-14T06:17:15.010 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6309999999999998e-06 2024-09-14T06:17:15.010 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.010 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.010 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:15.010 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717656+0000", 2024-09-14T06:17:15.010 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4724e-05 2024-09-14T06:17:15.010 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.011 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.011 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:15.011 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813006+0000", 2024-09-14T06:17:15.011 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095349801999999997 2024-09-14T06:17:15.011 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.011 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.011 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:15.011 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813107+0000", 2024-09-14T06:17:15.011 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000100973 2024-09-14T06:17:15.011 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.011 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.011 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:15.011 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813366+0000", 2024-09-14T06:17:15.011 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025931899999999999 2024-09-14T06:17:15.011 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.011 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:15.012 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.012 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.012 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.012 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40258 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 851968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:15.012 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:15.012 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 5.2171956970000002, 2024-09-14T06:17:15.012 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096062543, 2024-09-14T06:17:15.012 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:15.012 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:15.012 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:15.012 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:15.012 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:15.012 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:15.013 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40258 2024-09-14T06:17:15.013 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.013 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:15.013 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.013 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:15.013 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:15.013 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:15.013 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.013 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.013 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:15.013 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:15.013 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:15.013 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.013 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.014 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:15.014 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717671+0000", 2024-09-14T06:17:15.014 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8299999999999997e-07 2024-09-14T06:17:15.014 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.014 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.014 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:15.014 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717674+0000", 2024-09-14T06:17:15.014 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7080000000000002e-06 2024-09-14T06:17:15.014 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.014 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.014 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:15.014 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717681+0000", 2024-09-14T06:17:15.014 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7889999999999995e-06 2024-09-14T06:17:15.014 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.014 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.015 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:15.015 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717687+0000", 2024-09-14T06:17:15.015 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7529999999999998e-06 2024-09-14T06:17:15.015 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.015 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.015 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:15.015 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813408+0000", 2024-09-14T06:17:15.015 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095720451999999998 2024-09-14T06:17:15.015 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.015 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.015 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:15.015 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813499+0000", 2024-09-14T06:17:15.015 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0933000000000005e-05 2024-09-14T06:17:15.015 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.016 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.016 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:15.016 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813733+0000", 2024-09-14T06:17:15.016 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00023432499999999999 2024-09-14T06:17:15.016 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.016 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:15.016 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.016 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.016 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.016 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40260 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 860160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:15.016 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:15.016 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 5.2171698480000002, 2024-09-14T06:17:15.016 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096337381999999999, 2024-09-14T06:17:15.016 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:15.017 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:15.017 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:15.017 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:15.017 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:15.017 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:15.017 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40260 2024-09-14T06:17:15.017 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.017 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:15.017 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.017 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:15.017 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:15.017 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:15.017 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.017 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.017 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:15.018 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:15.018 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:15.018 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.018 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.018 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:15.018 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718008+0000", 2024-09-14T06:17:15.018 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00031192699999999998 2024-09-14T06:17:15.018 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.018 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.018 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:15.018 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718013+0000", 2024-09-14T06:17:15.018 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1830000000000002e-06 2024-09-14T06:17:15.018 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.018 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.019 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:15.019 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718036+0000", 2024-09-14T06:17:15.019 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2721000000000002e-05 2024-09-14T06:17:15.019 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.019 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.019 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:15.019 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718053+0000", 2024-09-14T06:17:15.019 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6736e-05 2024-09-14T06:17:15.019 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.019 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.019 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:15.019 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813767+0000", 2024-09-14T06:17:15.019 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095714658999999994 2024-09-14T06:17:15.019 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.020 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.020 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:15.020 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813816+0000", 2024-09-14T06:17:15.020 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8848999999999999e-05 2024-09-14T06:17:15.020 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.020 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.020 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:15.020 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814034+0000", 2024-09-14T06:17:15.020 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000217307 2024-09-14T06:17:15.020 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.020 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:15.020 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.020 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.021 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.021 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40262 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 868352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:15.021 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:15.021 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 5.2167701649999998, 2024-09-14T06:17:15.021 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096354165000000006, 2024-09-14T06:17:15.021 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:15.021 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:15.021 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:15.021 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:15.021 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:15.021 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:15.021 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40262 2024-09-14T06:17:15.021 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.022 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:15.022 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.022 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:15.022 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:15.022 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:15.022 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.022 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.022 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:15.022 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:15.022 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:15.022 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.022 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.022 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:15.022 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718281+0000", 2024-09-14T06:17:15.023 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000185069 2024-09-14T06:17:15.023 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.023 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.023 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:15.023 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718286+0000", 2024-09-14T06:17:15.023 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.118e-06 2024-09-14T06:17:15.023 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.023 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.023 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:15.023 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718295+0000", 2024-09-14T06:17:15.023 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0429999999999996e-06 2024-09-14T06:17:15.023 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.023 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.024 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:15.024 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718302+0000", 2024-09-14T06:17:15.024 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2790000000000001e-06 2024-09-14T06:17:15.024 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.024 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.024 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:15.024 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814076+0000", 2024-09-14T06:17:15.024 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095774076999999999 2024-09-14T06:17:15.024 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.024 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.024 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:15.024 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814191+0000", 2024-09-14T06:17:15.024 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00011500700000000001 2024-09-14T06:17:15.025 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.025 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.025 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:15.025 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814450+0000", 2024-09-14T06:17:15.025 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025857199999999999 2024-09-14T06:17:15.025 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.025 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:15.025 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.025 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.025 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.025 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40264 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 876544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:15.025 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:15.025 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 5.2164130970000002, 2024-09-14T06:17:15.025 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096372478999999997, 2024-09-14T06:17:15.025 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:15.026 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:15.026 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:15.026 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:15.026 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:15.026 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:15.026 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40264 2024-09-14T06:17:15.026 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.026 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:15.026 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.026 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:15.026 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:15.026 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:15.026 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.026 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.027 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:15.027 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:15.027 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:15.027 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.027 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.027 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:15.027 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718793+0000", 2024-09-14T06:17:15.027 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00033991099999999998 2024-09-14T06:17:15.027 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.027 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.027 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:15.027 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718805+0000", 2024-09-14T06:17:15.027 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2514000000000001e-05 2024-09-14T06:17:15.027 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.027 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.028 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:15.028 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718815+0000", 2024-09-14T06:17:15.028 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9590000000000007e-06 2024-09-14T06:17:15.028 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.028 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.028 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:15.028 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718822+0000", 2024-09-14T06:17:15.028 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2030000000000003e-06 2024-09-14T06:17:15.028 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.028 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.028 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:15.028 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814490+0000", 2024-09-14T06:17:15.028 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095667346 2024-09-14T06:17:15.028 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.029 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.029 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:15.029 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814577+0000", 2024-09-14T06:17:15.029 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6920000000000001e-05 2024-09-14T06:17:15.029 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.029 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.029 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:15.029 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814825+0000", 2024-09-14T06:17:15.029 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000248626 2024-09-14T06:17:15.029 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.030 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:15.030 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.030 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.030 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:15.030 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:15.032 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:17:15.032 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:15.029+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:15.033 INFO:teuthology.orchestra.run.smithi039.stdout:3956 read 0x3e4f879 thru 0x3e58616 (0x8d9e bytes) 2024-09-14T06:17:15.033 INFO:teuthology.orchestra.run.smithi039.stdout:3957 punch from 0x1e02810 to 0x1e0b63f, (0x8e2f bytes) 2024-09-14T06:17:15.033 INFO:teuthology.orchestra.run.smithi039.stdout:3959 trunc from 0x91f6895 to 0x7d31727 2024-09-14T06:17:15.033 INFO:teuthology.orchestra.run.smithi039.stdout:3960 write 0x7621b36 thru 0x76248dd (0x2da8 bytes) 2024-09-14T06:17:15.033 INFO:teuthology.orchestra.run.smithi039.stdout:3961 write 0x6072e32 thru 0x60777f0 (0x49bf bytes) 2024-09-14T06:17:15.033 INFO:teuthology.orchestra.run.smithi039.stdout:3962 write 0xdaba275 thru 0xdac1cd0 (0x7a5c bytes) 2024-09-14T06:17:15.033 INFO:teuthology.orchestra.run.smithi039.stdout:3963 write 0x13e31d8 thru 0x13ef0e0 (0xbf09 bytes) 2024-09-14T06:17:15.033 INFO:teuthology.orchestra.run.smithi039.stdout:3966 write 0x8bad9e3 thru 0x8bae831 (0xe4f bytes) 2024-09-14T06:17:15.033 INFO:teuthology.orchestra.run.smithi039.stdout:3967 punch from 0x43d65b to 0x44b5d1, (0xdf76 bytes) 2024-09-14T06:17:15.034 INFO:teuthology.orchestra.run.smithi039.stdout:3969 writesame 0x2c29596 thru 0x2c3923c data_size 0x1(0xfca7 bytes) 2024-09-14T06:17:15.113 INFO:teuthology.orchestra.run.smithi039.stdout:3970 punch from 0x5d9fdb2 to 0x5daa83c, (0xaa8a bytes) 2024-09-14T06:17:15.113 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:15.113 INFO:teuthology.orchestra.run.smithi039.stdout:3973 write 0x7412b92 thru 0x7417aec (0x4f5b bytes) 2024-09-14T06:17:15.126 INFO:teuthology.orchestra.run.smithi039.stdout:3974 trunc from 0xdac1cd1 to 0xa49c600 2024-09-14T06:17:15.132 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:15.129+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:15.143 INFO:teuthology.orchestra.run.smithi039.stdout:3975 write 0x1ff9d0c thru 0x2003107 (0x93fc bytes) 2024-09-14T06:17:15.154 INFO:teuthology.orchestra.run.smithi039.stdout:3977 read 0xbe0a44 thru 0xbe9e25 (0x93e2 bytes) 2024-09-14T06:17:15.155 INFO:teuthology.orchestra.run.smithi039.stdout:3978 write 0xa2b1f04 thru 0xa2b5ec1 (0x3fbe bytes) 2024-09-14T06:17:15.161 INFO:teuthology.orchestra.run.smithi039.stdout:3979 read 0x3049cf7 thru 0x3055363 (0xb66d bytes) 2024-09-14T06:17:15.162 INFO:teuthology.orchestra.run.smithi039.stdout:3980 read 0x1fad661 thru 0x1fb04da (0x2e7a bytes) 2024-09-14T06:17:15.166 INFO:teuthology.orchestra.run.smithi039.stdout:3981 trunc from 0xa49c600 to 0x8f6d3b6 2024-09-14T06:17:15.188 INFO:teuthology.orchestra.run.smithi039.stdout:3982 read 0x6a98e23 thru 0x6aa203d (0x921b bytes) 2024-09-14T06:17:15.188 INFO:teuthology.orchestra.run.smithi039.stdout:3983 write 0x8dce51a thru 0x8dd7444 (0x8f2b bytes) 2024-09-14T06:17:15.195 INFO:teuthology.orchestra.run.smithi039.stdout:3984 read 0x493735d thru 0x493cf3f (0x5be3 bytes) 2024-09-14T06:17:15.195 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:15.195 INFO:teuthology.orchestra.run.smithi039.stdout:3987 read 0x157f85 thru 0x160d73 (0x8def bytes) 2024-09-14T06:17:15.217 INFO:teuthology.orchestra.run.smithi039.stdout:3988 write 0xa299665 thru 0xa2a1284 (0x7c20 bytes) 2024-09-14T06:17:15.224 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:15.224 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:15.224 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.224 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:51122 3.b e67/23 3:d782bfdb:::rbd_data.11522fd7a0c5.0000000000000142:head v 67'128, mlcod=67'126)", 2024-09-14T06:17:15.225 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:15.223859+0000", 2024-09-14T06:17:15.225 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 0.0016652920000000001, 2024-09-14T06:17:15.225 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0017049000000000001, 2024-09-14T06:17:15.225 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:15.225 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:15.225 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:15.225 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:15.225 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.225 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:15.225 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:15.223859+0000", 2024-09-14T06:17:15.225 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:15.225 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.225 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.225 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:15.226 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:15.223859+0000", 2024-09-14T06:17:15.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:15.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.226 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:15.226 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:15.223860+0000", 2024-09-14T06:17:15.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0100000000000003e-07 2024-09-14T06:17:15.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.226 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:15.226 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:15.223885+0000", 2024-09-14T06:17:15.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:17:15.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.226 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:15.227 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:15.223899+0000", 2024-09-14T06:17:15.227 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4071e-05 2024-09-14T06:17:15.227 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.227 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.227 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:15.227 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:15.223908+0000", 2024-09-14T06:17:15.227 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9889999999999998e-06 2024-09-14T06:17:15.227 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.227 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.227 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:15.227 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:15.224029+0000", 2024-09-14T06:17:15.227 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000121204 2024-09-14T06:17:15.227 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:15.227 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:15.228 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:15.228 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:15.224100+0000", 2024-09-14T06:17:15.228 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0803999999999995e-05 2024-09-14T06:17:15.228 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.228 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:15.228 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.228 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:15.228 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-09-14T06:17:15.228 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 1 2024-09-14T06:17:15.228 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:3989 read 0x90984bf thru 0x90999ce (0x1510 bytes) 2024-09-14T06:17:15.233 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:15.229+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:15.239 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:17:15.244 INFO:teuthology.orchestra.run.smithi039.stdout:3990 write 0xcb7eb3d thru 0xcb88cdf (0xa1a3 bytes) 2024-09-14T06:17:15.253 INFO:teuthology.orchestra.run.smithi039.stdout:3992 write 0xb7b5629 thru 0xb7c00e7 (0xaabf bytes) 2024-09-14T06:17:15.261 INFO:teuthology.orchestra.run.smithi039.stdout:3993 write 0x1ad6a84 thru 0x1ae5d5d (0xf2da bytes) 2024-09-14T06:17:15.272 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:15.272 INFO:teuthology.orchestra.run.smithi039.stdout:3995 write 0xc829728 thru 0xc82e5c0 (0x4e99 bytes) 2024-09-14T06:17:15.275 INFO:teuthology.orchestra.run.smithi039.stdout:3997 punch from 0x7e47f24 to 0x7e4eb73, (0x6c4f bytes) 2024-09-14T06:17:15.275 INFO:teuthology.orchestra.run.smithi039.stdout:3998 punch from 0x20efc8f to 0x20f477a, (0x4aeb bytes) 2024-09-14T06:17:15.275 INFO:teuthology.orchestra.run.smithi039.stdout:3999 punch from 0xb4e2e1 to 0xb5380a, (0x5529 bytes) 2024-09-14T06:17:15.275 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:15.275 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:15.275 INFO:teuthology.orchestra.run.smithi039.stdout:4002 read 0x4a87bc1 thru 0x4a8a730 (0x2b70 bytes) 2024-09-14T06:17:15.275 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:15.295 INFO:teuthology.orchestra.run.smithi039.stdout:4004 trunc from 0xcb88ce0 to 0x5315621 2024-09-14T06:17:15.317 INFO:teuthology.orchestra.run.smithi039.stdout:4005 trunc from 0x5315621 to 0x56f7d5f 2024-09-14T06:17:15.327 INFO:teuthology.orchestra.run.smithi039.stdout:4006 trunc from 0x56f7d5f to 0x4bdc49c 2024-09-14T06:17:15.333 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:15.329+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:15.336 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:15.343 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:17:15.347 INFO:teuthology.orchestra.run.smithi039.stdout:4009 write 0x5ddf5d0 thru 0x5de55e0 (0x6011 bytes) 2024-09-14T06:17:15.378 INFO:teuthology.orchestra.run.smithi039.stdout:4010 writesame 0x9dac57a thru 0x9db79eb data_size 0x3(0xb472 bytes) 2024-09-14T06:17:15.378 INFO:teuthology.orchestra.run.smithi039.stdout:4011 read 0x9030965 thru 0x903f069 (0xe705 bytes) 2024-09-14T06:17:15.397 INFO:teuthology.orchestra.run.smithi039.stdout:4013 write 0xe7d8aa7 thru 0xe7dfc4c (0x71a6 bytes) 2024-09-14T06:17:15.402 INFO:teuthology.orchestra.run.smithi039.stdout:4014 read 0x40e8ff1 thru 0x40f7d96 (0xeda6 bytes) 2024-09-14T06:17:15.402 INFO:teuthology.orchestra.run.smithi039.stdout:4015 writesame 0x6871152 thru 0x6879343 data_size 0x1(0x81f2 bytes) 2024-09-14T06:17:15.431 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:15.431 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:15.431 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:15.431 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:15.431 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:15.435 INFO:teuthology.orchestra.run.smithi039.stdout:4016 read 0x3825593 thru 0x382d4b8 (0x7f26 bytes) 2024-09-14T06:17:15.435 INFO:teuthology.orchestra.run.smithi039.stdout:4017 write 0x2c349e3 thru 0x2c3be69 (0x7487 bytes) 2024-09-14T06:17:15.435 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:15.433+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:15.447 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:17:15.500 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:15.500 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:15.516 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:15.517 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:15.533 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:15.533 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:15.535 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:15.533+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:15.549 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:15.549 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:15.568 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:15.568 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:15.587 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:15.587 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:15.608 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:15.609 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:15.609 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:15.609 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:15.623 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:17:15.628 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:15.629 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:15.636 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:15.634+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:15.738 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:15.734+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:15.794 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:15.795 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:15.808 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:17:15.855 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:15.853+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:15.933 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:15.933 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:15.939 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:15.937+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:15.947 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:17:16.040 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:16.038+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:16.109 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:16.109 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:16.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:17:16.141 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:16.137+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:16.241 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:16.237+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:16.257 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:16.257 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:16.271 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:17:16.343 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:16.341+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:16.387 INFO:teuthology.orchestra.run.smithi039.stdout:4018 punch from 0xd4bbb0 to 0xd5503e, (0x948e bytes) 2024-09-14T06:17:16.388 INFO:teuthology.orchestra.run.smithi039.stdout:4019 read 0x3385f09 thru 0x338672b (0x823 bytes) 2024-09-14T06:17:16.389 INFO:teuthology.orchestra.run.smithi039.stdout:4020 write 0x2b0e1de thru 0x2b166b1 (0x84d4 bytes) 2024-09-14T06:17:16.400 INFO:teuthology.orchestra.run.smithi039.stdout:4021 write 0x5105c80 thru 0x510f493 (0x9814 bytes) 2024-09-14T06:17:16.407 INFO:teuthology.orchestra.run.smithi039.stdout:4023 read 0xa11bbdc thru 0xa11efec (0x3411 bytes) 2024-09-14T06:17:16.407 INFO:teuthology.orchestra.run.smithi039.stdout:4024 punch from 0x4e202a6 to 0x4e2b949, (0xb6a3 bytes) 2024-09-14T06:17:16.407 INFO:teuthology.orchestra.run.smithi039.stdout:4025 read 0x501af51 thru 0x5024737 (0x97e7 bytes) 2024-09-14T06:17:16.407 INFO:teuthology.orchestra.run.smithi039.stdout:4026 read 0x6584299 thru 0x6586c9f (0x2a07 bytes) 2024-09-14T06:17:16.407 INFO:teuthology.orchestra.run.smithi039.stdout:4027 writesame 0xe794171 thru 0xe7a3442 data_size 0x1(0xf2d2 bytes) 2024-09-14T06:17:16.410 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:16.410 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:16.431 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:17:16.443 INFO:teuthology.orchestra.run.smithi039.stdout:4028 punch from 0x54f97da to 0x5500c84, (0x74aa bytes) 2024-09-14T06:17:16.443 INFO:teuthology.orchestra.run.smithi039.stdout:4033 write 0x711d866 thru 0x711fac9 (0x2264 bytes) 2024-09-14T06:17:16.444 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:16.442+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:16.462 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:17:16.466 INFO:teuthology.orchestra.run.smithi039.stdout:4034 trunc from 0xe7dfc4d to 0x49f09a4 2024-09-14T06:17:16.490 INFO:teuthology.orchestra.run.smithi039.stdout:4035 read 0x56760b thru 0x56a7f9 (0x31ef bytes) 2024-09-14T06:17:16.491 INFO:teuthology.orchestra.run.smithi039.stdout:4036 read 0x2f27ce1 thru 0x2f33bb7 (0xbed7 bytes) 2024-09-14T06:17:16.492 INFO:teuthology.orchestra.run.smithi039.stdout:4037 read 0x20ea05 thru 0x219ee2 (0xb4de bytes) 2024-09-14T06:17:16.507 INFO:teuthology.orchestra.run.smithi039.stdout:4043 trunc from 0x49f09a4 to 0xe432e84 2024-09-14T06:17:16.518 INFO:teuthology.orchestra.run.smithi039.stdout:4046 read 0xda7a79a thru 0xda810f6 (0x695d bytes) 2024-09-14T06:17:16.519 INFO:teuthology.orchestra.run.smithi039.stdout:4051 writesame 0xa4f038d thru 0xa4fba0a data_size 0x5e(0xb67e bytes) 2024-09-14T06:17:16.525 INFO:teuthology.orchestra.run.smithi039.stdout:4053 writesame 0xc47ecc9 thru 0xc4850bd data_size 0x1(0x63f5 bytes) 2024-09-14T06:17:16.544 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:16.542+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:16.559 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:16.559 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:16.561 INFO:teuthology.orchestra.run.smithi039.stdout:4055 write 0xdd21427 thru 0xdd28dc3 (0x799d bytes) 2024-09-14T06:17:16.568 INFO:teuthology.orchestra.run.smithi039.stdout:4056 write 0x97de13b thru 0x97dfac7 (0x198d bytes) 2024-09-14T06:17:16.575 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:17:16.576 INFO:teuthology.orchestra.run.smithi039.stdout:4057 punch from 0xdf16f4a to 0xdf19a95, (0x2b4b bytes) 2024-09-14T06:17:16.576 INFO:teuthology.orchestra.run.smithi039.stdout:4058 write 0xb1d3e42 thru 0xb1dde3c (0x9ffb bytes) 2024-09-14T06:17:16.582 INFO:teuthology.orchestra.run.smithi039.stdout:4061 read 0xd5db3ba thru 0xd5e888a (0xd4d1 bytes) 2024-09-14T06:17:16.582 INFO:teuthology.orchestra.run.smithi039.stdout:4062 clone 23 order 25 su 2097152 sc 4 2024-09-14T06:17:16.645 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:16.642+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:16.694 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:16.694 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:16.709 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:17:16.752 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:16.750+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:16.846 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:16.842+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:16.883 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:16.884 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:16.897 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:17:16.948 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:16.945+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:17.049 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:17.049+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:17.073 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:17.073 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:17.088 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:17:17.150 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:17.149+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:17.217 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:17.217 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:17.232 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:17:17.283 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:17.282+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:17.342 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:17:17.342 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:17:17.342 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-14T06:17:17.342 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T06:17:17.342 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T06:17:17.343 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-14T06:17:17.359 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:17.359 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:17.359 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:17.358+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:17.375 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:17:17.392 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-09-14T06:17:17.453 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:17.449+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:17.481 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:17.481 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T06:17:17.481 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:17.524 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:17.524 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:17.527 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone22 from 0xe432e84 (overlap 0x3741ffe) to 0x2127998 2024-09-14T06:17:17.543 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:17:17.587 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:17.586+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:17.647 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:17:17.695 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:17.694+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:17.720 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:17.721 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:17.737 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:17:17.750 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-14T06:17:17.796 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:17.794+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:17.803 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:17.804 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:17.822 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:17.822 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:17.839 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:17.839 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:17.854 INFO:teuthology.orchestra.run.smithi039.stdout:4063 read 0x8194c92 thru 0x819a065 (0x53d4 bytes) 2024-09-14T06:17:17.855 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:17.853+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:17.857 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:17.857 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:17.877 INFO:teuthology.orchestra.run.smithi039.stdout:4064 trunc from 0xe432e84 to 0x29a1bc8 2024-09-14T06:17:17.899 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:17.900 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:17.909 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:17.909 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:17.909 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:17.909 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:17.909 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:17.926 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:17.926 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:17.931 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:17:17.950 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:17.951 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:17.980 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:17.980 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:17.980 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:17.978+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:18.011 INFO:teuthology.orchestra.run.smithi039.stdout:4065 read 0x85a003 thru 0x865921 (0xb91f bytes) 2024-09-14T06:17:18.011 INFO:teuthology.orchestra.run.smithi039.stdout:4066 read 0x154f0e7 thru 0x1550100 (0x101a bytes) 2024-09-14T06:17:18.032 INFO:teuthology.orchestra.run.smithi039.stdout:4067 write 0xa264ac4 thru 0xa272a68 (0xdfa5 bytes) 2024-09-14T06:17:18.038 INFO:teuthology.orchestra.run.smithi039.stdout:4069 punch from 0x3e3bb75 to 0x3e48f02, (0xd38d bytes) 2024-09-14T06:17:18.052 INFO:teuthology.orchestra.run.smithi039.stdout:4072 trunc from 0xa272a69 to 0x19f11d6 2024-09-14T06:17:18.057 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:18.054+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:18.073 INFO:teuthology.orchestra.run.smithi039.stdout:4073 read 0x64a968 thru 0x653022 (0x86bb bytes) 2024-09-14T06:17:18.073 INFO:teuthology.orchestra.run.smithi039.stdout:4074 read 0xbaad49 thru 0xbabad0 (0xd88 bytes) 2024-09-14T06:17:18.074 INFO:teuthology.orchestra.run.smithi039.stdout:4075 read 0x1760db thru 0x179877 (0x379d bytes) 2024-09-14T06:17:18.078 INFO:teuthology.orchestra.run.smithi039.stdout:4076 trunc from 0x19f11d6 to 0x4553f54 2024-09-14T06:17:18.087 INFO:teuthology.orchestra.run.smithi039.stdout:4077 read 0xed7d41 thru 0xee066e (0x892e bytes) 2024-09-14T06:17:18.093 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:18.093 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:18.093 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:18.093 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:18.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.093 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11263 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.093 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:18.093 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.099659963000001, 2024-09-14T06:17:18.094 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049944574999999998, 2024-09-14T06:17:18.094 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.094 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.094 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:18.094 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.094 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.094 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.094 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11263 2024-09-14T06:17:18.094 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.094 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.094 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.094 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:18.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:18.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:18.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.51e-07 2024-09-14T06:17:18.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.096 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.096 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.096 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991560+0000", 2024-09-14T06:17:18.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6719999999999998e-06 2024-09-14T06:17:18.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.096 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.096 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.096 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991567+0000", 2024-09-14T06:17:18.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1470000000000004e-06 2024-09-14T06:17:18.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.097 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.097 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.097 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991572+0000", 2024-09-14T06:17:18.097 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.237e-06 2024-09-14T06:17:18.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.097 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.097 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.097 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041149+0000", 2024-09-14T06:17:18.097 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049576988000000002 2024-09-14T06:17:18.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.097 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.097 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.098 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041277+0000", 2024-09-14T06:17:18.098 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000127687 2024-09-14T06:17:18.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.098 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.098 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.098 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041497+0000", 2024-09-14T06:17:18.098 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022039299999999999 2024-09-14T06:17:18.098 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.098 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.098 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.098 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.098 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11264 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 327680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.098 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:18.099 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.099633621999999, 2024-09-14T06:17:18.099 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050248567000000001, 2024-09-14T06:17:18.099 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.099 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.099 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:18.099 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.099 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.099 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.099 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11264 2024-09-14T06:17:18.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.099 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:18.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:18.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:18.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.41e-07 2024-09-14T06:17:18.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991581+0000", 2024-09-14T06:17:18.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3740000000000001e-06 2024-09-14T06:17:18.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991585+0000", 2024-09-14T06:17:18.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9400000000000004e-06 2024-09-14T06:17:18.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991587+0000", 2024-09-14T06:17:18.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.694e-06 2024-09-14T06:17:18.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041527+0000", 2024-09-14T06:17:18.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049939349000000001 2024-09-14T06:17:18.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041598+0000", 2024-09-14T06:17:18.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1633999999999999e-05 2024-09-14T06:17:18.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041827+0000", 2024-09-14T06:17:18.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000229235 2024-09-14T06:17:18.103 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.103 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.103 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.103 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.103 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:18.103 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.099620281999997, 2024-09-14T06:17:18.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:17:18.103 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.103 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.103 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:18.103 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.104 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.104 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.104 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:17:18.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.104 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:18.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:18.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.105 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:18.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:17:18.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.105 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:17:18.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:17:18.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.105 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.106 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:17:18.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:17:18.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.106 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.106 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:17:18.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:17:18.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.106 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.106 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:17:18.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:17:18.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:17:18.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:17:18.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:17:18.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:17:18.107 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.107 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.107 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.107 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.108 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:18.108 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.099607210000002, 2024-09-14T06:17:18.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:17:18.108 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.108 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.108 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:18.108 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.108 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.108 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.108 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:17:18.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.108 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:18.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:18.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:17:18.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:17:18.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.110 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.110 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:17:18.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:17:18.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.110 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.110 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:17:18.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:17:18.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.110 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:17:18.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:17:18.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.111 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:17:18.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:17:18.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.111 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:17:18.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:17:18.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.112 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.112 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:17:18.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:17:18.112 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.112 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.112 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.112 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.112 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:18.112 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.099589778000002, 2024-09-14T06:17:18.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:17:18.112 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.113 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.113 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:18.113 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.113 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.113 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.113 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:17:18.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.113 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:18.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:18.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:18.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:17:18.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:17:18.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:17:18.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:17:18.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:17:18.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:17:18.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:17:18.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:17:18.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:17:18.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.116 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:17:18.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:17:18.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.116 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:17:18.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:17:18.116 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.116 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.117 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.117 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.117 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:18.117 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.099568673, 2024-09-14T06:17:18.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:17:18.117 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.117 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.117 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:18.117 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.117 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.117 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.117 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:17:18.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.118 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.118 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.118 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:18.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.118 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.118 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:18.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.118 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.118 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:18.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:17:18.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:17:18.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:17:18.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:17:18.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:17:18.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:17:18.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:17:18.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:17:18.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:17:18.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:17:18.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:17:18.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.121 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.121 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:17:18.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:17:18.121 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.121 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.121 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.121 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.121 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:18.121 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.099547565000002, 2024-09-14T06:17:18.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:17:18.121 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.121 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.122 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:18.122 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.122 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.122 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.122 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:17:18.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.122 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.122 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.122 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:18.122 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.122 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.122 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:18.122 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:18.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:17:18.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:17:18.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:17:18.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:17:18.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:17:18.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:17:18.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:17:18.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:17:18.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:17:18.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:17:18.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:17:18.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:17:18.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:17:18.125 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.125 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.125 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.125 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48234 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:18.125 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:18.125 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.0582762730000006, 2024-09-14T06:17:18.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051730353999999999, 2024-09-14T06:17:18.126 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.126 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.126 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.126 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.126 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.126 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.126 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48234 2024-09-14T06:17:18.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.126 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.126 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.126 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.126 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:18.126 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.126 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.127 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:18.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.127 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032937+0000", 2024-09-14T06:17:18.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:17:18.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.127 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032940+0000", 2024-09-14T06:17:18.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0580000000000002e-06 2024-09-14T06:17:18.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.128 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.128 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032946+0000", 2024-09-14T06:17:18.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2960000000000004e-06 2024-09-14T06:17:18.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.128 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.128 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032951+0000", 2024-09-14T06:17:18.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.938e-06 2024-09-14T06:17:18.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074419+0000", 2024-09-14T06:17:18.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041467979000000002 2024-09-14T06:17:18.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074490+0000", 2024-09-14T06:17:18.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1324999999999999e-05 2024-09-14T06:17:18.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:18.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074859+0000", 2024-09-14T06:17:18.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036914899999999999 2024-09-14T06:17:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078464+0000", 2024-09-14T06:17:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036043170000000001 2024-09-14T06:17:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084547+0000", 2024-09-14T06:17:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006083097 2024-09-14T06:17:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084614+0000", 2024-09-14T06:17:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7509999999999996e-05 2024-09-14T06:17:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084667+0000", 2024-09-14T06:17:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:17:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48237 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.0581965289999999, 2024-09-14T06:17:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051772830999999998, 2024-09-14T06:17:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48237 2024-09-14T06:17:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.133 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.133 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:18.133 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.133 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.133 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:18.133 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.133 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.133 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:18.133 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0499999999999999e-07 2024-09-14T06:17:18.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033018+0000", 2024-09-14T06:17:18.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0949999999999998e-06 2024-09-14T06:17:18.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033023+0000", 2024-09-14T06:17:18.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1620000000000001e-06 2024-09-14T06:17:18.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033026+0000", 2024-09-14T06:17:18.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0120000000000001e-06 2024-09-14T06:17:18.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075349+0000", 2024-09-14T06:17:18.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042323373999999997 2024-09-14T06:17:18.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075394+0000", 2024-09-14T06:17:18.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4861000000000003e-05 2024-09-14T06:17:18.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:18.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075578+0000", 2024-09-14T06:17:18.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00018414899999999999 2024-09-14T06:17:18.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.136 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078894+0000", 2024-09-14T06:17:18.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033158480000000001 2024-09-14T06:17:18.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.136 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:18.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084677+0000", 2024-09-14T06:17:18.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0057832120000000002 2024-09-14T06:17:18.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.137 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:18.137 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084752+0000", 2024-09-14T06:17:18.137 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5123000000000006e-05 2024-09-14T06:17:18.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.137 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.137 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084789+0000", 2024-09-14T06:17:18.137 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6690000000000003e-05 2024-09-14T06:17:18.137 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.137 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.137 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.138 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48243 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:18.138 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:18.138 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.0581154040000005, 2024-09-14T06:17:18.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051780221000000001, 2024-09-14T06:17:18.138 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.138 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.138 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.138 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.138 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.138 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.138 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48243 2024-09-14T06:17:18.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.139 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:18.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:18.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:18.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:17:18.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033100+0000", 2024-09-14T06:17:18.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:17:18.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033108+0000", 2024-09-14T06:17:18.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1760000000000004e-06 2024-09-14T06:17:18.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.141 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.141 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033111+0000", 2024-09-14T06:17:18.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0790000000000001e-06 2024-09-14T06:17:18.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.141 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.141 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075903+0000", 2024-09-14T06:17:18.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042791942999999999 2024-09-14T06:17:18.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.141 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075987+0000", 2024-09-14T06:17:18.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4467000000000003e-05 2024-09-14T06:17:18.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.142 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:18.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.076259+0000", 2024-09-14T06:17:18.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027201900000000003 2024-09-14T06:17:18.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.142 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078947+0000", 2024-09-14T06:17:18.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026875879999999999 2024-09-14T06:17:18.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:18.143 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084797+0000", 2024-09-14T06:17:18.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0058500380000000001 2024-09-14T06:17:18.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:18.143 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084843+0000", 2024-09-14T06:17:18.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6298999999999998e-05 2024-09-14T06:17:18.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.143 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084877+0000", 2024-09-14T06:17:18.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4378999999999997e-05 2024-09-14T06:17:18.144 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.144 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.144 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.144 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48287 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:18.144 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:18.144 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.0580756040000008, 2024-09-14T06:17:18.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051924706000000001, 2024-09-14T06:17:18.144 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.144 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.144 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.144 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.145 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.145 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.145 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48287 2024-09-14T06:17:18.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.145 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:18.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:18.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033138+0000", 2024-09-14T06:17:18.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6599999999999996e-07 2024-09-14T06:17:18.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033141+0000", 2024-09-14T06:17:18.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0879999999999999e-06 2024-09-14T06:17:18.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033147+0000", 2024-09-14T06:17:18.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2249999999999997e-06 2024-09-14T06:17:18.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033153+0000", 2024-09-14T06:17:18.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.066e-06 2024-09-14T06:17:18.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077790+0000", 2024-09-14T06:17:18.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.044637255000000001 2024-09-14T06:17:18.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077845+0000", 2024-09-14T06:17:18.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4410999999999998e-05 2024-09-14T06:17:18.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:18.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078062+0000", 2024-09-14T06:17:18.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021755599999999999 2024-09-14T06:17:18.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.149 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.080991+0000", 2024-09-14T06:17:18.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029282969999999998 2024-09-14T06:17:18.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.150 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:18.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084885+0000", 2024-09-14T06:17:18.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003894495 2024-09-14T06:17:18.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.150 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:18.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085020+0000", 2024-09-14T06:17:18.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013500799999999999 2024-09-14T06:17:18.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.151 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085062+0000", 2024-09-14T06:17:18.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1638999999999999e-05 2024-09-14T06:17:18.151 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.151 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.151 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.151 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48288 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:18.151 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:18.151 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.0580519049999992, 2024-09-14T06:17:18.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051994018000000003, 2024-09-14T06:17:18.152 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.152 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.152 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.152 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.152 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.152 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.152 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48288 2024-09-14T06:17:18.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.152 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:18.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:18.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:18.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.89e-07 2024-09-14T06:17:18.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033164+0000", 2024-09-14T06:17:18.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.779e-06 2024-09-14T06:17:18.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033171+0000", 2024-09-14T06:17:18.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6279999999999999e-06 2024-09-14T06:17:18.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033176+0000", 2024-09-14T06:17:18.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1610000000000002e-06 2024-09-14T06:17:18.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078991+0000", 2024-09-14T06:17:18.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045815348999999998 2024-09-14T06:17:18.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079086+0000", 2024-09-14T06:17:18.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5282e-05 2024-09-14T06:17:18.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:18.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079397+0000", 2024-09-14T06:17:18.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031055999999999998 2024-09-14T06:17:18.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.156 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084432+0000", 2024-09-14T06:17:18.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050355599999999997 2024-09-14T06:17:18.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.156 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:18.156 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085073+0000", 2024-09-14T06:17:18.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064015199999999995 2024-09-14T06:17:18.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.156 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:18.156 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085120+0000", 2024-09-14T06:17:18.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7107000000000001e-05 2024-09-14T06:17:18.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085155+0000", 2024-09-14T06:17:18.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4950999999999998e-05 2024-09-14T06:17:18.157 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.157 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.157 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.157 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48416 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:18.157 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:18.157 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.0558114930000002, 2024-09-14T06:17:18.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053690144000000002, 2024-09-14T06:17:18.157 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.157 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.158 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.158 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.158 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.158 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.158 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48416 2024-09-14T06:17:18.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.158 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:18.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:18.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035402+0000", 2024-09-14T06:17:18.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7999999999999995e-07 2024-09-14T06:17:18.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035405+0000", 2024-09-14T06:17:18.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3960000000000002e-06 2024-09-14T06:17:18.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035413+0000", 2024-09-14T06:17:18.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2209999999999997e-06 2024-09-14T06:17:18.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035420+0000", 2024-09-14T06:17:18.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1539999999999996e-06 2024-09-14T06:17:18.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081021+0000", 2024-09-14T06:17:18.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045600389999999998 2024-09-14T06:17:18.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081091+0000", 2024-09-14T06:17:18.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0325000000000002e-05 2024-09-14T06:17:18.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:18.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081434+0000", 2024-09-14T06:17:18.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000342519 2024-09-14T06:17:18.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:18.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085254+0000", 2024-09-14T06:17:18.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038201609999999999 2024-09-14T06:17:18.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088977+0000", 2024-09-14T06:17:18.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003722805 2024-09-14T06:17:18.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:18.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089048+0000", 2024-09-14T06:17:18.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1266999999999994e-05 2024-09-14T06:17:18.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089091+0000", 2024-09-14T06:17:18.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3325999999999998e-05 2024-09-14T06:17:18.163 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.163 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.163 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.163 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48417 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:18.163 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:18.163 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.0557828239999996, 2024-09-14T06:17:18.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053805032000000003, 2024-09-14T06:17:18.163 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.163 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.163 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.163 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.163 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.164 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.164 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48417 2024-09-14T06:17:18.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.164 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:18.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:18.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:18.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9500000000000003e-07 2024-09-14T06:17:18.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035433+0000", 2024-09-14T06:17:18.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.796e-06 2024-09-14T06:17:18.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035442+0000", 2024-09-14T06:17:18.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7120000000000006e-06 2024-09-14T06:17:18.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035448+0000", 2024-09-14T06:17:18.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.015e-06 2024-09-14T06:17:18.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081855+0000", 2024-09-14T06:17:18.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.046407634000000003 2024-09-14T06:17:18.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081938+0000", 2024-09-14T06:17:18.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2687000000000003e-05 2024-09-14T06:17:18.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:18.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082313+0000", 2024-09-14T06:17:18.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000374872 2024-09-14T06:17:18.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:18.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087616+0000", 2024-09-14T06:17:18.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053032670000000004 2024-09-14T06:17:18.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089140+0000", 2024-09-14T06:17:18.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0015234350000000001 2024-09-14T06:17:18.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:18.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089184+0000", 2024-09-14T06:17:18.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4804e-05 2024-09-14T06:17:18.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089235+0000", 2024-09-14T06:17:18.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0315000000000002e-05 2024-09-14T06:17:18.169 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.169 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.169 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.169 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48418 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:18.169 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:18.169 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.0557386849999997, 2024-09-14T06:17:18.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053936425000000003, 2024-09-14T06:17:18.169 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.169 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.169 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.169 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.169 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.170 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.170 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48418 2024-09-14T06:17:18.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.170 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.170 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:18.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.170 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:18.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035475+0000", 2024-09-14T06:17:18.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7000000000000003e-07 2024-09-14T06:17:18.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035479+0000", 2024-09-14T06:17:18.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2350000000000001e-06 2024-09-14T06:17:18.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035488+0000", 2024-09-14T06:17:18.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8820000000000007e-06 2024-09-14T06:17:18.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035494+0000", 2024-09-14T06:17:18.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3060000000000003e-06 2024-09-14T06:17:18.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082826+0000", 2024-09-14T06:17:18.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.047332008000000002 2024-09-14T06:17:18.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.173 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082883+0000", 2024-09-14T06:17:18.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7124999999999999e-05 2024-09-14T06:17:18.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.173 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:18.173 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.083385+0000", 2024-09-14T06:17:18.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050134099999999998 2024-09-14T06:17:18.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.173 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:18.173 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088888+0000", 2024-09-14T06:17:18.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005503127 2024-09-14T06:17:18.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.174 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089294+0000", 2024-09-14T06:17:18.174 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040669700000000003 2024-09-14T06:17:18.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:18.174 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089348+0000", 2024-09-14T06:17:18.174 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3729e-05 2024-09-14T06:17:18.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.174 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089410+0000", 2024-09-14T06:17:18.174 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2105000000000003e-05 2024-09-14T06:17:18.175 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.175 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.175 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.175 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48419 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:18.175 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:18.175 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.0556610549999998, 2024-09-14T06:17:18.175 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054007545999999997, 2024-09-14T06:17:18.175 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.175 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.175 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.175 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.175 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.176 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.176 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48419 2024-09-14T06:17:18.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.176 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:18.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:18.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.177 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.177 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.177 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035552+0000", 2024-09-14T06:17:18.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4099999999999998e-07 2024-09-14T06:17:18.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.177 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.177 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.177 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035557+0000", 2024-09-14T06:17:18.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1819999999999997e-06 2024-09-14T06:17:18.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.177 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.178 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.178 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035566+0000", 2024-09-14T06:17:18.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1120000000000002e-06 2024-09-14T06:17:18.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.178 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.178 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035572+0000", 2024-09-14T06:17:18.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0430000000000004e-06 2024-09-14T06:17:18.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.178 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.178 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085304+0000", 2024-09-14T06:17:18.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049731707 2024-09-14T06:17:18.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.179 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085371+0000", 2024-09-14T06:17:18.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6771000000000002e-05 2024-09-14T06:17:18.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:18.179 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085601+0000", 2024-09-14T06:17:18.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022968000000000001 2024-09-14T06:17:18.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:18.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088893+0000", 2024-09-14T06:17:18.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032927780000000001 2024-09-14T06:17:18.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089467+0000", 2024-09-14T06:17:18.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057330399999999996 2024-09-14T06:17:18.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:18.181 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089520+0000", 2024-09-14T06:17:18.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3133000000000001e-05 2024-09-14T06:17:18.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.181 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.181 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.181 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089559+0000", 2024-09-14T06:17:18.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9195e-05 2024-09-14T06:17:18.181 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.181 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.181 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.181 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48420 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.0555718249999995, 2024-09-14T06:17:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056401932000000002, 2024-09-14T06:17:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48420 2024-09-14T06:17:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.183 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.183 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.183 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:18.183 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.183 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.183 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.183 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:18.183 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.183 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.183 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.184 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035642+0000", 2024-09-14T06:17:18.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1519999999999999e-06 2024-09-14T06:17:18.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.184 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.184 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035647+0000", 2024-09-14T06:17:18.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2299999999999999e-06 2024-09-14T06:17:18.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.184 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.184 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035660+0000", 2024-09-14T06:17:18.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3166999999999999e-05 2024-09-14T06:17:18.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.185 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035666+0000", 2024-09-14T06:17:18.185 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1720000000000004e-06 2024-09-14T06:17:18.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.185 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086104+0000", 2024-09-14T06:17:18.185 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050437956999999999 2024-09-14T06:17:18.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.186 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086164+0000", 2024-09-14T06:17:18.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9592999999999997e-05 2024-09-14T06:17:18.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:18.186 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086402+0000", 2024-09-14T06:17:18.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000238314 2024-09-14T06:17:18.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:18.186 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089103+0000", 2024-09-14T06:17:18.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027011600000000002 2024-09-14T06:17:18.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091962+0000", 2024-09-14T06:17:18.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028581259999999999 2024-09-14T06:17:18.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:18.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092005+0000", 2024-09-14T06:17:18.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3111000000000001e-05 2024-09-14T06:17:18.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092043+0000", 2024-09-14T06:17:18.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7950000000000001e-05 2024-09-14T06:17:18.188 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.188 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.188 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.188 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48424 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:18.188 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:18.188 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.0553078800000009, 2024-09-14T06:17:18.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056273173000000003, 2024-09-14T06:17:18.188 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.188 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.189 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.189 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.189 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.189 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.189 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48424 2024-09-14T06:17:18.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.189 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.189 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.189 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:18.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.190 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.190 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:18.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.190 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.190 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:18.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2900000000000003e-07 2024-09-14T06:17:18.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035909+0000", 2024-09-14T06:17:18.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6490000000000001e-06 2024-09-14T06:17:18.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035918+0000", 2024-09-14T06:17:18.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5089999999999999e-06 2024-09-14T06:17:18.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035924+0000", 2024-09-14T06:17:18.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7089999999999999e-06 2024-09-14T06:17:18.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087662+0000", 2024-09-14T06:17:18.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737621999999997 2024-09-14T06:17:18.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087744+0000", 2024-09-14T06:17:18.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2174000000000003e-05 2024-09-14T06:17:18.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:18.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088106+0000", 2024-09-14T06:17:18.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000361947 2024-09-14T06:17:18.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:18.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092053+0000", 2024-09-14T06:17:18.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0039473080000000001 2024-09-14T06:17:18.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092087+0000", 2024-09-14T06:17:18.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4153000000000003e-05 2024-09-14T06:17:18.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.194 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:18.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092138+0000", 2024-09-14T06:17:18.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0804999999999999e-05 2024-09-14T06:17:18.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.194 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092178+0000", 2024-09-14T06:17:18.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9668e-05 2024-09-14T06:17:18.194 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.194 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.194 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.195 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48508 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:18.195 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:18.195 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.0502327989999998, 2024-09-14T06:17:18.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054906017000000001, 2024-09-14T06:17:18.195 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.195 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.195 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.195 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.195 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.195 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.195 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48508 2024-09-14T06:17:18.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.196 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:18.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:18.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040981+0000", 2024-09-14T06:17:18.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7700000000000004e-07 2024-09-14T06:17:18.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040984+0000", 2024-09-14T06:17:18.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1480000000000002e-06 2024-09-14T06:17:18.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040991+0000", 2024-09-14T06:17:18.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4580000000000004e-06 2024-09-14T06:17:18.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040997+0000", 2024-09-14T06:17:18.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7799999999999997e-06 2024-09-14T06:17:18.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089601+0000", 2024-09-14T06:17:18.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048604264000000001 2024-09-14T06:17:18.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089708+0000", 2024-09-14T06:17:18.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010681200000000001 2024-09-14T06:17:18.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.199 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:18.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090384+0000", 2024-09-14T06:17:18.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00067557499999999998 2024-09-14T06:17:18.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.199 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093879+0000", 2024-09-14T06:17:18.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034954679999999998 2024-09-14T06:17:18.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:18.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095764+0000", 2024-09-14T06:17:18.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018848070000000001 2024-09-14T06:17:18.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:18.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095837+0000", 2024-09-14T06:17:18.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3433e-05 2024-09-14T06:17:18.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095886+0000", 2024-09-14T06:17:18.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8495000000000002e-05 2024-09-14T06:17:18.201 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.201 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.201 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.201 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48537 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:18.201 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:18.201 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 8.0488690290000005, 2024-09-14T06:17:18.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053681833999999998, 2024-09-14T06:17:18.201 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:18.202 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:18.202 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.202 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:18.202 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:18.202 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.202 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48537 2024-09-14T06:17:18.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.202 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:18.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:18.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:18.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:18.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:18.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:18.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:18.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:18.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2299999999999999e-07 2024-09-14T06:17:18.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:18.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042350+0000", 2024-09-14T06:17:18.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5690000000000004e-06 2024-09-14T06:17:18.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:18.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042361+0000", 2024-09-14T06:17:18.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1337e-05 2024-09-14T06:17:18.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042369+0000", 2024-09-14T06:17:18.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8359999999999992e-06 2024-09-14T06:17:18.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.205 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:18.205 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090914+0000", 2024-09-14T06:17:18.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048544945999999999 2024-09-14T06:17:18.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.205 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:18.205 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090995+0000", 2024-09-14T06:17:18.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0548000000000002e-05 2024-09-14T06:17:18.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:18.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091508+0000", 2024-09-14T06:17:18.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000513723 2024-09-14T06:17:18.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093900+0000", 2024-09-14T06:17:18.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023920429999999999 2024-09-14T06:17:18.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:18.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095944+0000", 2024-09-14T06:17:18.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020437379999999998 2024-09-14T06:17:18.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:18.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095987+0000", 2024-09-14T06:17:18.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2521999999999997e-05 2024-09-14T06:17:18.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:18.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:18.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:18.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.096025+0000", 2024-09-14T06:17:18.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8649000000000002e-05 2024-09-14T06:17:18.207 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.208 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.208 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.208 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:18.208 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:18.208 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:18.209 INFO:teuthology.orchestra.run.smithi039.stdout:4078 write 0xe091176 thru 0xe096f73 (0x5dfe bytes) 2024-09-14T06:17:18.210 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:17:18.212 INFO:teuthology.orchestra.run.smithi039.stdout:4079 read 0xb84846e thru 0xb851ab4 (0x9647 bytes) 2024-09-14T06:17:18.212 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:18.212 INFO:teuthology.orchestra.run.smithi039.stdout:4082 punch from 0xb0d05c9 to 0xb0d50ae, (0x4ae5 bytes) 2024-09-14T06:17:18.213 INFO:teuthology.orchestra.run.smithi039.stdout:4084 trunc from 0xe096f74 to 0x6c51078 2024-09-14T06:17:18.213 INFO:teuthology.orchestra.run.smithi039.stdout:4085 punch from 0x49cf84d to 0x49d6a05, (0x71b8 bytes) 2024-09-14T06:17:18.213 INFO:teuthology.orchestra.run.smithi039.stdout:4086 read 0x46a0f8a thru 0x46b00f9 (0xf170 bytes) 2024-09-14T06:17:18.213 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:18.213 INFO:teuthology.orchestra.run.smithi039.stdout:4088 read 0xb4814d thru 0xb51ee1 (0x9d95 bytes) 2024-09-14T06:17:18.213 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:18.213 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:18.213 INFO:teuthology.orchestra.run.smithi039.stdout:4091 punch from 0x5e0c077 to 0x5e1ab67, (0xeaf0 bytes) 2024-09-14T06:17:18.213 INFO:teuthology.orchestra.run.smithi039.stdout:4092 trunc from 0x6c51078 to 0x9e7528b 2024-09-14T06:17:18.213 INFO:teuthology.orchestra.run.smithi039.stdout:4094 read 0x14bd42f thru 0x14c1022 (0x3bf4 bytes) 2024-09-14T06:17:18.214 INFO:teuthology.orchestra.run.smithi039.stdout:4095 read 0x7942e7f thru 0x7946d6d (0x3eef bytes) 2024-09-14T06:17:18.214 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:18.214 INFO:teuthology.orchestra.run.smithi039.stdout:4098 trunc from 0x9e7528b to 0x70c6ded 2024-09-14T06:17:18.214 INFO:teuthology.orchestra.run.smithi039.stdout:4099 write 0x8bac thru 0x119ec (0x8e41 bytes) 2024-09-14T06:17:18.214 INFO:teuthology.orchestra.run.smithi039.stdout:4100 write 0x615e380 thru 0x6160e4b (0x2acc bytes) 2024-09-14T06:17:18.214 INFO:teuthology.orchestra.run.smithi039.stdout:4102 flatten 2024-09-14T06:17:18.214 INFO:teuthology.orchestra.run.smithi039.stdout:4103 punch from 0x60d8c8d to 0x60dfb2a, (0x6e9d bytes) 2024-09-14T06:17:18.214 INFO:teuthology.orchestra.run.smithi039.stdout:4104 read 0x796767 thru 0x7a47d6 (0xe070 bytes) 2024-09-14T06:17:18.214 INFO:teuthology.orchestra.run.smithi039.stdout:4105 read 0x229b415 thru 0x229f7e7 (0x43d3 bytes) 2024-09-14T06:17:18.214 INFO:teuthology.orchestra.run.smithi039.stdout:4106 punch from 0x10e5592 to 0x10f4d05, (0xf773 bytes) 2024-09-14T06:17:18.214 INFO:teuthology.orchestra.run.smithi039.stdout:4109 punch from 0x542fcf3 to 0x54323fb, (0x2708 bytes) 2024-09-14T06:17:18.214 INFO:teuthology.orchestra.run.smithi039.stdout:4110 read 0x922e6c thru 0x92d3c1 (0xa556 bytes) 2024-09-14T06:17:18.215 INFO:teuthology.orchestra.run.smithi039.stdout:4111 trunc from 0x70c6ded to 0x95455d2 2024-09-14T06:17:18.215 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:18.205+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:18.224 INFO:teuthology.orchestra.run.smithi039.stdout:4112 punch from 0x3e661a6 to 0x3e69abf, (0x3919 bytes) 2024-09-14T06:17:18.224 INFO:teuthology.orchestra.run.smithi039.stdout:4113 read 0x92df480 thru 0x92e6bc2 (0x7743 bytes) 2024-09-14T06:17:18.224 INFO:teuthology.orchestra.run.smithi039.stdout:4114 punch from 0x68e9fa9 to 0x68ee21a, (0x4271 bytes) 2024-09-14T06:17:18.225 INFO:teuthology.orchestra.run.smithi039.stdout:4116 read 0x2f21871 thru 0x2f2f290 (0xda20 bytes) 2024-09-14T06:17:18.225 INFO:teuthology.orchestra.run.smithi039.stdout:4117 read 0xe673ea thru 0xe67fbb (0xbd2 bytes) 2024-09-14T06:17:18.251 INFO:teuthology.orchestra.run.smithi039.stdout:4118 write 0xe46362b thru 0xe4671f1 (0x3bc7 bytes) 2024-09-14T06:17:18.257 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:18.257 INFO:teuthology.orchestra.run.smithi039.stdout:4121 read 0x33d412 thru 0x347fc6 (0xabb5 bytes) 2024-09-14T06:17:18.258 INFO:teuthology.orchestra.run.smithi039.stdout:4124 read 0xd333ff1 thru 0xd33f04a (0xb05a bytes) 2024-09-14T06:17:18.274 INFO:teuthology.orchestra.run.smithi039.stdout:4125 trunc from 0xe4671f2 to 0x79b2cfe 2024-09-14T06:17:18.302 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:18.310 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:18.309+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:18.319 INFO:teuthology.orchestra.run.smithi039.stdout:4128 write 0x87823d6 thru 0x8783d57 (0x1982 bytes) 2024-09-14T06:17:18.326 INFO:teuthology.orchestra.run.smithi039.stdout:4129 writesame 0x425ef5d thru 0x426585a data_size 0x2(0x68fe bytes) 2024-09-14T06:17:18.349 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:18.349 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:18.349 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:18.349 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:18.363 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:17:18.371 INFO:teuthology.orchestra.run.smithi039.stdout:4130 writesame 0xc0f4f31 thru 0xc0f4f8e data_size 0x1(0x5e bytes) 2024-09-14T06:17:18.371 INFO:teuthology.orchestra.run.smithi039.stdout:4131 clone 24 order 19 su 32768 sc 13 2024-09-14T06:17:18.411 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:18.406+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:18.512 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:18.510+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:18.528 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:18.528 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:18.528 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:18.528 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:18.528 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:18.542 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:17:18.613 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:18.609+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:18.665 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:18.665 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:18.665 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:18.665 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:18.665 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.666 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13229 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.666 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:18.666 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.565527496000001, 2024-09-14T06:17:18.666 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081798868999999996, 2024-09-14T06:17:18.666 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.666 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.666 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.666 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.666 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.666 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.666 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13229 2024-09-14T06:17:18.666 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.667 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.667 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.667 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.667 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:18.667 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.667 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.667 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.667 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.667 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:18.667 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.667 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.667 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.667 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.668 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098710+0000", 2024-09-14T06:17:18.668 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8199999999999999e-07 2024-09-14T06:17:18.668 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.668 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.668 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.668 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098714+0000", 2024-09-14T06:17:18.668 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:17:18.668 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.668 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.668 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.668 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098727+0000", 2024-09-14T06:17:18.668 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2520999999999999e-05 2024-09-14T06:17:18.668 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.668 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.669 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.669 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098749+0000", 2024-09-14T06:17:18.669 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2773999999999999e-05 2024-09-14T06:17:18.669 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.669 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.669 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.669 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160234+0000", 2024-09-14T06:17:18.669 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.061484430999999999 2024-09-14T06:17:18.669 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.669 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.669 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.669 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160281+0000", 2024-09-14T06:17:18.670 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7437e-05 2024-09-14T06:17:18.670 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.670 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.670 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.670 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160569+0000", 2024-09-14T06:17:18.670 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028788700000000002 2024-09-14T06:17:18.670 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.670 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.670 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.670 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163121+0000", 2024-09-14T06:17:18.670 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025514090000000001 2024-09-14T06:17:18.670 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.670 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.671 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.671 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180410+0000", 2024-09-14T06:17:18.671 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.017289098999999999 2024-09-14T06:17:18.671 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.671 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.671 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.671 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180472+0000", 2024-09-14T06:17:18.671 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2547999999999998e-05 2024-09-14T06:17:18.671 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.671 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.671 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.671 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180508+0000", 2024-09-14T06:17:18.671 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5534000000000002e-05 2024-09-14T06:17:18.672 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.672 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.672 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.672 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.672 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.672 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13230 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.672 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:18.672 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.565446293999997, 2024-09-14T06:17:18.672 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081851558000000005, 2024-09-14T06:17:18.672 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.672 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.672 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.672 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.673 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.673 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.673 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13230 2024-09-14T06:17:18.673 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.673 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.673 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.673 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.673 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:18.673 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.673 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.673 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.673 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.673 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:18.673 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.674 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.674 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.674 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.674 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098791+0000", 2024-09-14T06:17:18.674 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8800000000000001e-07 2024-09-14T06:17:18.674 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.674 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.674 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.674 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098795+0000", 2024-09-14T06:17:18.674 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.391e-06 2024-09-14T06:17:18.674 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.674 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.674 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.675 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098804+0000", 2024-09-14T06:17:18.675 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3440000000000001e-06 2024-09-14T06:17:18.675 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.675 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.675 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.675 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098810+0000", 2024-09-14T06:17:18.675 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2339999999999999e-06 2024-09-14T06:17:18.675 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.675 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.675 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.675 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160930+0000", 2024-09-14T06:17:18.675 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062120018999999999 2024-09-14T06:17:18.675 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.675 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.675 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.676 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161017+0000", 2024-09-14T06:17:18.676 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6868999999999998e-05 2024-09-14T06:17:18.676 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.676 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.676 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.676 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161300+0000", 2024-09-14T06:17:18.676 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028349499999999998 2024-09-14T06:17:18.676 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.676 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.676 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.676 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164612+0000", 2024-09-14T06:17:18.676 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033119680000000002 2024-09-14T06:17:18.676 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.676 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.677 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.677 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180559+0000", 2024-09-14T06:17:18.677 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015946612999999998 2024-09-14T06:17:18.677 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.677 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.677 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.677 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180605+0000", 2024-09-14T06:17:18.677 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5890000000000003e-05 2024-09-14T06:17:18.677 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.677 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.677 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.677 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180642+0000", 2024-09-14T06:17:18.677 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:17:18.678 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.678 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.678 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.678 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.678 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.678 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13231 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.678 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:18.678 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.565183879999999, 2024-09-14T06:17:18.678 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081750685000000003, 2024-09-14T06:17:18.678 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.678 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.678 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.679 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.679 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.679 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.679 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13231 2024-09-14T06:17:18.679 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.679 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.679 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.679 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.679 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:18.679 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.679 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.679 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.679 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.680 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:18.680 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.680 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.680 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.680 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.680 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099053+0000", 2024-09-14T06:17:18.680 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:17:18.680 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.680 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.680 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.680 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099059+0000", 2024-09-14T06:17:18.680 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1789999999999999e-06 2024-09-14T06:17:18.681 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.681 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.681 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.681 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099072+0000", 2024-09-14T06:17:18.681 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3179999999999999e-05 2024-09-14T06:17:18.681 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.681 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.681 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.681 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099080+0000", 2024-09-14T06:17:18.681 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9030000000000004e-06 2024-09-14T06:17:18.681 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.681 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.682 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.682 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161681+0000", 2024-09-14T06:17:18.682 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062601641999999999 2024-09-14T06:17:18.682 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.682 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.682 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.682 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161766+0000", 2024-09-14T06:17:18.682 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4909999999999998e-05 2024-09-14T06:17:18.682 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.682 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.682 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.682 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162109+0000", 2024-09-14T06:17:18.682 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00034264899999999999 2024-09-14T06:17:18.683 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.683 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.683 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.683 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164615+0000", 2024-09-14T06:17:18.683 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025061979999999998 2024-09-14T06:17:18.683 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.683 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.683 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.683 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180718+0000", 2024-09-14T06:17:18.683 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016103240000000001 2024-09-14T06:17:18.683 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.683 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.684 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.684 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180768+0000", 2024-09-14T06:17:18.684 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9781999999999999e-05 2024-09-14T06:17:18.684 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.684 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.684 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.684 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180803+0000", 2024-09-14T06:17:18.684 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5021999999999998e-05 2024-09-14T06:17:18.684 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.684 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.684 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.684 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.684 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.685 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13232 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.685 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:18.685 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.565142106000003, 2024-09-14T06:17:18.685 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081861981, 2024-09-14T06:17:18.685 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.685 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.685 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.685 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.685 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.685 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.685 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13232 2024-09-14T06:17:18.685 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.686 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.686 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.686 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.686 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:18.686 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.686 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.686 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.686 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.686 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:18.686 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.686 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.686 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.686 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.687 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099095+0000", 2024-09-14T06:17:18.687 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:17:18.687 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.687 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.687 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.687 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099098+0000", 2024-09-14T06:17:18.687 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5420000000000001e-06 2024-09-14T06:17:18.687 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.687 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.687 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.687 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099106+0000", 2024-09-14T06:17:18.687 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.824e-06 2024-09-14T06:17:18.687 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.687 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.688 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.688 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099112+0000", 2024-09-14T06:17:18.688 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9789999999999998e-06 2024-09-14T06:17:18.688 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.688 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.688 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.688 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162576+0000", 2024-09-14T06:17:18.688 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.063463723999999999 2024-09-14T06:17:18.688 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.688 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.688 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.688 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162627+0000", 2024-09-14T06:17:18.688 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1530000000000003e-05 2024-09-14T06:17:18.689 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.689 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.689 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.689 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162854+0000", 2024-09-14T06:17:18.689 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000226159 2024-09-14T06:17:18.689 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.689 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.689 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.689 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166090+0000", 2024-09-14T06:17:18.689 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032362319999999999 2024-09-14T06:17:18.689 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.689 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.689 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.690 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180877+0000", 2024-09-14T06:17:18.690 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014786726 2024-09-14T06:17:18.690 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.690 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.690 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.690 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180920+0000", 2024-09-14T06:17:18.690 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3427000000000003e-05 2024-09-14T06:17:18.690 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.690 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.690 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.690 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180956+0000", 2024-09-14T06:17:18.690 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6195999999999997e-05 2024-09-14T06:17:18.690 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.690 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.691 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.691 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.691 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.691 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13233 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.691 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:18.691 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.565116842999998, 2024-09-14T06:17:18.691 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081924860000000002, 2024-09-14T06:17:18.691 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.691 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.691 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.691 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.691 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.692 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.692 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13233 2024-09-14T06:17:18.692 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.692 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.692 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.692 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.692 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:18.692 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.692 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.692 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.692 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.692 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:18.693 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.693 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.693 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.693 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.693 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099120+0000", 2024-09-14T06:17:18.693 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:17:18.693 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.693 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.693 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.693 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099122+0000", 2024-09-14T06:17:18.693 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:17:18.693 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.693 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.693 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.694 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099126+0000", 2024-09-14T06:17:18.694 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2540000000000003e-06 2024-09-14T06:17:18.694 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.694 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.694 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.694 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099130+0000", 2024-09-14T06:17:18.694 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4060000000000001e-06 2024-09-14T06:17:18.694 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.694 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.694 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.694 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163169+0000", 2024-09-14T06:17:18.694 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064039415000000002 2024-09-14T06:17:18.694 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.695 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.695 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.695 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163260+0000", 2024-09-14T06:17:18.695 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1038000000000005e-05 2024-09-14T06:17:18.695 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.695 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.695 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.695 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163640+0000", 2024-09-14T06:17:18.695 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037954699999999999 2024-09-14T06:17:18.695 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.695 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.695 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.695 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166093+0000", 2024-09-14T06:17:18.696 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024533229999999999 2024-09-14T06:17:18.696 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.696 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.696 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.696 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180993+0000", 2024-09-14T06:17:18.696 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01489997 2024-09-14T06:17:18.696 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.696 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.696 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.696 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181020+0000", 2024-09-14T06:17:18.696 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6806999999999999e-05 2024-09-14T06:17:18.696 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.697 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.697 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.697 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181044+0000", 2024-09-14T06:17:18.697 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4405000000000001e-05 2024-09-14T06:17:18.697 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.697 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.697 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.697 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.697 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.697 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13234 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.697 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:18.697 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.565074637999999, 2024-09-14T06:17:18.697 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082015597999999995, 2024-09-14T06:17:18.698 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.698 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.698 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.698 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.698 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.698 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.698 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13234 2024-09-14T06:17:18.698 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.698 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.698 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.698 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.698 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:18.698 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.699 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.699 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.699 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.699 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:18.699 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.699 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.699 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.699 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.699 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:18.699 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0999999999999998e-07 2024-09-14T06:17:18.699 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.699 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.699 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.700 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099166+0000", 2024-09-14T06:17:18.700 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:17:18.700 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.700 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.700 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.700 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099171+0000", 2024-09-14T06:17:18.700 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6949999999999997e-06 2024-09-14T06:17:18.700 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.700 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.700 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.700 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099175+0000", 2024-09-14T06:17:18.700 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9029999999999997e-06 2024-09-14T06:17:18.700 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.701 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.701 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.701 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164063+0000", 2024-09-14T06:17:18.701 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064887608999999999 2024-09-14T06:17:18.701 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.701 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.701 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.701 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164111+0000", 2024-09-14T06:17:18.701 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7648999999999997e-05 2024-09-14T06:17:18.701 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.701 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.701 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.701 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164351+0000", 2024-09-14T06:17:18.702 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024072899999999999 2024-09-14T06:17:18.702 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.702 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.702 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.702 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167561+0000", 2024-09-14T06:17:18.702 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032094179999999999 2024-09-14T06:17:18.702 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.702 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.702 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.702 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181093+0000", 2024-09-14T06:17:18.702 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013532245 2024-09-14T06:17:18.702 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.702 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.702 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.703 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181139+0000", 2024-09-14T06:17:18.703 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6483999999999997e-05 2024-09-14T06:17:18.703 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.703 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.703 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.703 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181177+0000", 2024-09-14T06:17:18.703 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7861000000000002e-05 2024-09-14T06:17:18.703 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.703 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.703 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.703 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.703 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.703 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13235 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.704 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:18.704 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.564967363999997, 2024-09-14T06:17:18.704 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081988088000000001, 2024-09-14T06:17:18.704 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.704 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.704 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.704 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.704 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.704 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.704 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13235 2024-09-14T06:17:18.704 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.704 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.704 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.704 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.705 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:18.705 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.705 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.705 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.705 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.705 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:18.705 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.705 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.705 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.705 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.705 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099270+0000", 2024-09-14T06:17:18.705 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7400000000000003e-07 2024-09-14T06:17:18.705 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.706 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.706 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.706 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099276+0000", 2024-09-14T06:17:18.706 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:17:18.706 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.706 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.706 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.706 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099284+0000", 2024-09-14T06:17:18.706 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7979999999999995e-06 2024-09-14T06:17:18.706 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.706 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.706 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.706 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099289+0000", 2024-09-14T06:17:18.707 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:17:18.707 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.707 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.707 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.707 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164664+0000", 2024-09-14T06:17:18.707 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065374901999999999 2024-09-14T06:17:18.707 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.707 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.707 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.707 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164755+0000", 2024-09-14T06:17:18.707 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0790000000000003e-05 2024-09-14T06:17:18.707 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.707 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.707 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.708 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165143+0000", 2024-09-14T06:17:18.708 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000387904 2024-09-14T06:17:18.708 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.708 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.708 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.708 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167564+0000", 2024-09-14T06:17:18.708 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024208989999999998 2024-09-14T06:17:18.708 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.708 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.708 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.708 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181212+0000", 2024-09-14T06:17:18.708 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013648664 2024-09-14T06:17:18.708 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.709 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.709 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.709 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181237+0000", 2024-09-14T06:17:18.709 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5177999999999998e-05 2024-09-14T06:17:18.709 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.709 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.709 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.709 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181257+0000", 2024-09-14T06:17:18.709 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9660000000000002e-05 2024-09-14T06:17:18.709 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.709 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.709 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.709 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.709 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.710 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13236 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.710 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:18.710 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.564860545000002, 2024-09-14T06:17:18.710 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082010939000000005, 2024-09-14T06:17:18.710 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.710 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.710 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.710 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.710 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.710 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.710 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13236 2024-09-14T06:17:18.710 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.710 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.711 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.711 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.711 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:18.711 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.711 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.711 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.711 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.711 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:18.711 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.711 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.711 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.711 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.711 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:18.712 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.61e-07 2024-09-14T06:17:18.712 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.712 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.712 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.712 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099382+0000", 2024-09-14T06:17:18.712 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3589999999999999e-06 2024-09-14T06:17:18.712 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.712 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.712 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.713 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099388+0000", 2024-09-14T06:17:18.713 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9619999999999998e-06 2024-09-14T06:17:18.713 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.713 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.713 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.713 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099393+0000", 2024-09-14T06:17:18.713 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3410000000000002e-06 2024-09-14T06:17:18.713 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.713 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.713 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.713 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165553+0000", 2024-09-14T06:17:18.713 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066159604999999996 2024-09-14T06:17:18.713 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.713 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.714 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.714 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165603+0000", 2024-09-14T06:17:18.714 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0427999999999999e-05 2024-09-14T06:17:18.714 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.714 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.714 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.714 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165827+0000", 2024-09-14T06:17:18.714 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022358299999999999 2024-09-14T06:17:18.714 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.714 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.714 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.714 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177005+0000", 2024-09-14T06:17:18.714 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011178044 2024-09-14T06:17:18.714 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.715 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.715 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.715 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181302+0000", 2024-09-14T06:17:18.715 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042970669999999999 2024-09-14T06:17:18.715 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.715 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.715 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.715 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181349+0000", 2024-09-14T06:17:18.715 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7287999999999999e-05 2024-09-14T06:17:18.715 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.715 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.715 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.715 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181387+0000", 2024-09-14T06:17:18.716 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7601000000000001e-05 2024-09-14T06:17:18.716 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.716 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.716 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.716 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.716 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.716 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13237 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.716 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:18.716 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.564702398000001, 2024-09-14T06:17:18.716 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081927299999999995, 2024-09-14T06:17:18.716 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.716 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.716 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.716 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.717 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.717 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.717 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13237 2024-09-14T06:17:18.717 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.717 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.717 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.717 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.717 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:18.717 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.717 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.717 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.717 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.717 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:18.718 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.718 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.718 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.718 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.718 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099535+0000", 2024-09-14T06:17:18.718 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:17:18.718 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.718 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.718 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.718 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099538+0000", 2024-09-14T06:17:18.718 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:17:18.718 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.718 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.719 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.719 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099545+0000", 2024-09-14T06:17:18.719 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:17:18.719 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.719 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.719 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.719 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099551+0000", 2024-09-14T06:17:18.719 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1770000000000003e-06 2024-09-14T06:17:18.719 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.719 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.719 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.719 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166131+0000", 2024-09-14T06:17:18.720 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066580499000000001 2024-09-14T06:17:18.720 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.720 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.720 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.720 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166221+0000", 2024-09-14T06:17:18.720 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0000000000000006e-05 2024-09-14T06:17:18.720 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.720 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.720 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.720 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166622+0000", 2024-09-14T06:17:18.720 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040029799999999999 2024-09-14T06:17:18.720 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.720 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.721 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.721 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177008+0000", 2024-09-14T06:17:18.721 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010386772000000001 2024-09-14T06:17:18.721 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.721 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.721 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.721 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181420+0000", 2024-09-14T06:17:18.721 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044119190000000003 2024-09-14T06:17:18.721 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.721 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.721 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.721 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181443+0000", 2024-09-14T06:17:18.721 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2603999999999999e-05 2024-09-14T06:17:18.721 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.722 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.722 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.722 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181461+0000", 2024-09-14T06:17:18.722 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8405000000000001e-05 2024-09-14T06:17:18.722 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.722 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.722 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.722 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.722 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.722 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13238 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.722 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:18.722 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.564666090000003, 2024-09-14T06:17:18.722 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082014927000000001, 2024-09-14T06:17:18.722 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.723 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.723 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.723 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.723 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.723 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.723 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13238 2024-09-14T06:17:18.723 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.723 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.723 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.723 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.723 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:18.723 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.724 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.724 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.724 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.724 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:18.724 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.724 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.724 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.724 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.724 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099571+0000", 2024-09-14T06:17:18.724 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9699999999999996e-07 2024-09-14T06:17:18.724 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.724 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.725 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.725 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099577+0000", 2024-09-14T06:17:18.725 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2890000000000003e-06 2024-09-14T06:17:18.725 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.725 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.725 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.725 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099587+0000", 2024-09-14T06:17:18.725 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0002e-05 2024-09-14T06:17:18.725 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.725 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.725 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.725 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099593+0000", 2024-09-14T06:17:18.725 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4539999999999998e-06 2024-09-14T06:17:18.726 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.726 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.726 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.726 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167024+0000", 2024-09-14T06:17:18.726 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067431480000000002 2024-09-14T06:17:18.726 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.726 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.726 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.726 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167074+0000", 2024-09-14T06:17:18.726 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9996e-05 2024-09-14T06:17:18.726 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.726 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.726 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.726 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167288+0000", 2024-09-14T06:17:18.727 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021410900000000001 2024-09-14T06:17:18.727 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.727 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.727 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.727 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177997+0000", 2024-09-14T06:17:18.727 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010708756 2024-09-14T06:17:18.727 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.727 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.727 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.727 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181508+0000", 2024-09-14T06:17:18.727 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003511363 2024-09-14T06:17:18.727 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.727 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.728 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.728 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181552+0000", 2024-09-14T06:17:18.728 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3609999999999998e-05 2024-09-14T06:17:18.728 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.728 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.728 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.728 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181585+0000", 2024-09-14T06:17:18.728 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3271e-05 2024-09-14T06:17:18.728 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.728 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.728 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.728 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.728 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.729 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13239 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.729 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:18.729 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.564574272000002, 2024-09-14T06:17:18.729 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081998772999999997, 2024-09-14T06:17:18.729 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.729 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.729 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.729 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.729 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.729 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.729 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13239 2024-09-14T06:17:18.729 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.729 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.730 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.730 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.730 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:18.730 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.730 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.730 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.730 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.730 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:18.730 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.730 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.730 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.730 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.730 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099663+0000", 2024-09-14T06:17:18.730 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6100000000000001e-07 2024-09-14T06:17:18.731 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.731 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.731 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.731 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099666+0000", 2024-09-14T06:17:18.731 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.056e-06 2024-09-14T06:17:18.731 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.731 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.731 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.731 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099672+0000", 2024-09-14T06:17:18.731 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8120000000000002e-06 2024-09-14T06:17:18.731 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.731 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.731 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.731 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099676+0000", 2024-09-14T06:17:18.732 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:17:18.732 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.732 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.732 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.732 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167602+0000", 2024-09-14T06:17:18.732 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067925867000000001 2024-09-14T06:17:18.732 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.732 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.732 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.732 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167693+0000", 2024-09-14T06:17:18.732 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0736000000000007e-05 2024-09-14T06:17:18.732 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.732 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.732 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.733 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168065+0000", 2024-09-14T06:17:18.733 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037245399999999998 2024-09-14T06:17:18.733 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.733 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.733 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.733 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178003+0000", 2024-09-14T06:17:18.733 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0099372360000000003 2024-09-14T06:17:18.733 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.733 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.733 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.733 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181621+0000", 2024-09-14T06:17:18.733 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036178489999999998 2024-09-14T06:17:18.733 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.734 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.734 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.734 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181643+0000", 2024-09-14T06:17:18.734 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2155e-05 2024-09-14T06:17:18.734 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.734 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.734 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.734 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181661+0000", 2024-09-14T06:17:18.734 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8147000000000001e-05 2024-09-14T06:17:18.734 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.734 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.734 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.734 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.735 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.735 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13240 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.735 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:18.735 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.564455178999999, 2024-09-14T06:17:18.735 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081942607000000001, 2024-09-14T06:17:18.735 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.735 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.735 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.735 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.735 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.735 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.735 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13240 2024-09-14T06:17:18.735 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.735 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.736 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.736 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.736 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:18.736 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.736 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.736 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.736 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.736 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:18.736 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.736 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.736 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.736 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.736 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099782+0000", 2024-09-14T06:17:18.736 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3900000000000004e-07 2024-09-14T06:17:18.737 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.737 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.737 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.737 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099787+0000", 2024-09-14T06:17:18.737 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6189999999999999e-06 2024-09-14T06:17:18.737 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.737 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.737 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.737 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099793+0000", 2024-09-14T06:17:18.737 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.998e-06 2024-09-14T06:17:18.737 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.737 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.737 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.738 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099797+0000", 2024-09-14T06:17:18.738 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8449999999999996e-06 2024-09-14T06:17:18.738 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.738 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.738 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.738 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168494+0000", 2024-09-14T06:17:18.738 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.068696500999999993 2024-09-14T06:17:18.738 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.738 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.738 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.738 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168590+0000", 2024-09-14T06:17:18.738 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6733999999999999e-05 2024-09-14T06:17:18.738 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.738 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.739 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.739 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168973+0000", 2024-09-14T06:17:18.739 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038229399999999998 2024-09-14T06:17:18.739 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.739 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.739 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.739 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178013+0000", 2024-09-14T06:17:18.739 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0090397199999999994 2024-09-14T06:17:18.739 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.739 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.739 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.739 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181683+0000", 2024-09-14T06:17:18.739 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036705819999999999 2024-09-14T06:17:18.739 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.740 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.740 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.740 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181705+0000", 2024-09-14T06:17:18.740 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1551000000000001e-05 2024-09-14T06:17:18.740 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.740 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.740 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.740 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181724+0000", 2024-09-14T06:17:18.740 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9123999999999999e-05 2024-09-14T06:17:18.740 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.740 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.740 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.740 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.740 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.741 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13243 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.741 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:18.741 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.563921448999999, 2024-09-14T06:17:18.741 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081767815999999993, 2024-09-14T06:17:18.741 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.741 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.741 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.741 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.741 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.741 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.741 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13243 2024-09-14T06:17:18.741 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.741 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.741 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.742 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.742 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:18.742 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.742 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.742 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.742 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.742 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:18.742 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.742 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.742 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.742 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.742 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:18.742 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1e-07 2024-09-14T06:17:18.743 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.743 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.743 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.743 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100317+0000", 2024-09-14T06:17:18.743 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.092e-06 2024-09-14T06:17:18.743 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.743 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.743 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.743 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100321+0000", 2024-09-14T06:17:18.743 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7809999999999999e-06 2024-09-14T06:17:18.743 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.743 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.743 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.743 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100324+0000", 2024-09-14T06:17:18.744 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.117e-06 2024-09-14T06:17:18.744 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.744 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.744 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.744 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171113+0000", 2024-09-14T06:17:18.744 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.070788816000000004 2024-09-14T06:17:18.744 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.744 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.744 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.744 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171202+0000", 2024-09-14T06:17:18.744 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8843000000000003e-05 2024-09-14T06:17:18.744 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.744 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.745 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.745 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171584+0000", 2024-09-14T06:17:18.745 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038231099999999998 2024-09-14T06:17:18.745 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.745 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.745 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.745 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178101+0000", 2024-09-14T06:17:18.745 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0065168329999999997 2024-09-14T06:17:18.745 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.745 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.745 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.745 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182041+0000", 2024-09-14T06:17:18.745 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039401810000000001 2024-09-14T06:17:18.745 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.746 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.746 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.746 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182065+0000", 2024-09-14T06:17:18.746 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3623000000000001e-05 2024-09-14T06:17:18.746 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.746 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.746 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.746 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182083+0000", 2024-09-14T06:17:18.746 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7909000000000001e-05 2024-09-14T06:17:18.746 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.746 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.746 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.746 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.746 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.747 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13244 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.747 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:18.747 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.563906629000002, 2024-09-14T06:17:18.747 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081837800000000002, 2024-09-14T06:17:18.747 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.747 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.747 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.747 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.747 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.747 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.747 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13244 2024-09-14T06:17:18.747 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.747 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.747 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.748 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.748 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:18.748 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.748 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.748 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.748 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.748 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:18.748 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.748 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.748 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.748 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.748 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:18.748 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:17:18.749 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.749 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.749 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.749 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100337+0000", 2024-09-14T06:17:18.749 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.939e-06 2024-09-14T06:17:18.749 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.749 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.749 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.749 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100341+0000", 2024-09-14T06:17:18.749 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2320000000000003e-06 2024-09-14T06:17:18.749 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.749 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.749 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.749 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100344+0000", 2024-09-14T06:17:18.750 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0759999999999999e-06 2024-09-14T06:17:18.750 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.750 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.750 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.750 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171996+0000", 2024-09-14T06:17:18.750 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.071651919999999994 2024-09-14T06:17:18.750 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.750 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.750 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.750 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172079+0000", 2024-09-14T06:17:18.750 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2579999999999999e-05 2024-09-14T06:17:18.750 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.750 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.750 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.750 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172457+0000", 2024-09-14T06:17:18.751 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037785100000000001 2024-09-14T06:17:18.751 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.751 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.751 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.752 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178104+0000", 2024-09-14T06:17:18.752 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056471730000000001 2024-09-14T06:17:18.752 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.752 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.752 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.752 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182123+0000", 2024-09-14T06:17:18.752 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040190019999999998 2024-09-14T06:17:18.752 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.752 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.752 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.752 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182147+0000", 2024-09-14T06:17:18.752 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4241000000000001e-05 2024-09-14T06:17:18.753 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.753 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.753 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.753 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182168+0000", 2024-09-14T06:17:18.753 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0492999999999998e-05 2024-09-14T06:17:18.753 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.753 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.753 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.753 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.753 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.753 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13245 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.753 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:18.754 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.563887121999997, 2024-09-14T06:17:18.754 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081893899000000006, 2024-09-14T06:17:18.754 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.754 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.754 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.754 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.754 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.754 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.754 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13245 2024-09-14T06:17:18.754 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.754 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.754 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.754 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.755 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:18.755 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.755 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.755 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.755 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.755 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:18.755 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.755 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.755 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.755 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.755 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100350+0000", 2024-09-14T06:17:18.755 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5499999999999999e-07 2024-09-14T06:17:18.755 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.755 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.756 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.756 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100353+0000", 2024-09-14T06:17:18.756 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:17:18.756 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.756 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.756 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.756 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100357+0000", 2024-09-14T06:17:18.756 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8500000000000004e-06 2024-09-14T06:17:18.756 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.756 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.756 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.756 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100360+0000", 2024-09-14T06:17:18.756 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8700000000000001e-06 2024-09-14T06:17:18.757 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.757 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.757 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.757 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172862+0000", 2024-09-14T06:17:18.757 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.072501992000000001 2024-09-14T06:17:18.757 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.757 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.757 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.757 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172950+0000", 2024-09-14T06:17:18.757 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8988999999999998e-05 2024-09-14T06:17:18.757 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.757 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.757 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.758 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173324+0000", 2024-09-14T06:17:18.758 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037366199999999997 2024-09-14T06:17:18.758 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.758 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.758 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.758 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178107+0000", 2024-09-14T06:17:18.758 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047830520000000003 2024-09-14T06:17:18.758 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.758 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.758 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.758 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182202+0000", 2024-09-14T06:17:18.758 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040949280000000003 2024-09-14T06:17:18.759 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.759 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.759 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.759 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182226+0000", 2024-09-14T06:17:18.759 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3535000000000001e-05 2024-09-14T06:17:18.759 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.759 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.759 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.759 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182243+0000", 2024-09-14T06:17:18.759 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7445e-05 2024-09-14T06:17:18.759 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.759 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.760 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.760 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.760 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.760 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13246 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.760 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:18.760 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.563872799999999, 2024-09-14T06:17:18.760 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081962276000000001, 2024-09-14T06:17:18.760 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.760 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.760 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.760 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.760 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.760 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.761 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13246 2024-09-14T06:17:18.761 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.761 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.761 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.761 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.761 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:18.761 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.761 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.761 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.761 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.761 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:18.761 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.761 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.762 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.762 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.762 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:18.762 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0800000000000001e-07 2024-09-14T06:17:18.762 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.762 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.762 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.762 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100367+0000", 2024-09-14T06:17:18.762 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8439999999999998e-06 2024-09-14T06:17:18.762 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.762 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.763 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.763 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100372+0000", 2024-09-14T06:17:18.763 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4009999999999996e-06 2024-09-14T06:17:18.763 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.763 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.763 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.763 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100378+0000", 2024-09-14T06:17:18.763 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7180000000000001e-06 2024-09-14T06:17:18.763 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.763 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.763 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.763 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173780+0000", 2024-09-14T06:17:18.763 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.073401991 2024-09-14T06:17:18.763 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.764 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.764 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.764 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173879+0000", 2024-09-14T06:17:18.764 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9449000000000003e-05 2024-09-14T06:17:18.764 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.764 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.764 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.764 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174240+0000", 2024-09-14T06:17:18.764 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036118100000000002 2024-09-14T06:17:18.764 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.764 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.764 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.764 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178109+0000", 2024-09-14T06:17:18.765 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038690619999999999 2024-09-14T06:17:18.765 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.765 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.765 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.765 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182281+0000", 2024-09-14T06:17:18.765 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041715900000000002 2024-09-14T06:17:18.765 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.765 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.765 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.765 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182306+0000", 2024-09-14T06:17:18.765 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4477e-05 2024-09-14T06:17:18.765 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.765 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.765 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.765 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182326+0000", 2024-09-14T06:17:18.766 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0254999999999998e-05 2024-09-14T06:17:18.766 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.766 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.766 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.766 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.766 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.766 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13247 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.766 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:18.766 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.563775622999998, 2024-09-14T06:17:18.766 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966593000000004, 2024-09-14T06:17:18.766 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.766 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.767 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.767 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.767 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.767 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.767 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13247 2024-09-14T06:17:18.767 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.767 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.767 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.767 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.767 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:18.767 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.767 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.767 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.767 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.768 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:18.768 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.768 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.768 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.768 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.768 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100462+0000", 2024-09-14T06:17:18.768 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9800000000000002e-07 2024-09-14T06:17:18.768 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.768 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.768 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.768 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100466+0000", 2024-09-14T06:17:18.769 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4039999999999997e-06 2024-09-14T06:17:18.769 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.769 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.769 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.769 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100475+0000", 2024-09-14T06:17:18.769 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3680000000000008e-06 2024-09-14T06:17:18.769 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.769 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.769 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.769 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100482+0000", 2024-09-14T06:17:18.769 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8630000000000002e-06 2024-09-14T06:17:18.769 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.769 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.770 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.770 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174669+0000", 2024-09-14T06:17:18.770 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074186849999999999 2024-09-14T06:17:18.770 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.770 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.770 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.770 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174758+0000", 2024-09-14T06:17:18.770 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9021e-05 2024-09-14T06:17:18.770 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.770 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.770 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.770 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175137+0000", 2024-09-14T06:17:18.770 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378511 2024-09-14T06:17:18.771 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.771 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.771 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.771 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178158+0000", 2024-09-14T06:17:18.771 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003020956 2024-09-14T06:17:18.771 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.771 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.771 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.771 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182386+0000", 2024-09-14T06:17:18.771 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042278840000000003 2024-09-14T06:17:18.771 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.771 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.771 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.771 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182410+0000", 2024-09-14T06:17:18.772 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4026e-05 2024-09-14T06:17:18.772 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.772 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.772 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.772 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182427+0000", 2024-09-14T06:17:18.772 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7711999999999999e-05 2024-09-14T06:17:18.772 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.772 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.772 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.772 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.772 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.772 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13248 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.772 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:18.773 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.563693542000003, 2024-09-14T06:17:18.773 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966527999999997, 2024-09-14T06:17:18.773 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.773 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.773 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.773 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.773 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.773 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.773 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13248 2024-09-14T06:17:18.773 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.773 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.773 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.773 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.773 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:18.774 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.774 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.774 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.774 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.774 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:18.774 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.774 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.774 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.774 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.774 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:18.774 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:17:18.774 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.774 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.774 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.775 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100548+0000", 2024-09-14T06:17:18.775 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:17:18.775 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.775 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.775 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.775 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100557+0000", 2024-09-14T06:17:18.775 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:17:18.775 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.775 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.775 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.775 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100563+0000", 2024-09-14T06:17:18.775 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2519999999999996e-06 2024-09-14T06:17:18.776 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.776 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.776 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.776 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175572+0000", 2024-09-14T06:17:18.776 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075008459 2024-09-14T06:17:18.776 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.776 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.776 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.776 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175656+0000", 2024-09-14T06:17:18.776 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3906999999999995e-05 2024-09-14T06:17:18.776 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.776 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.776 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.776 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176010+0000", 2024-09-14T06:17:18.777 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00035447499999999998 2024-09-14T06:17:18.777 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.777 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.777 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.777 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178304+0000", 2024-09-14T06:17:18.777 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0022941379999999998 2024-09-14T06:17:18.777 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.777 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.777 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.777 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182467+0000", 2024-09-14T06:17:18.777 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041632020000000004 2024-09-14T06:17:18.777 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.778 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.778 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.778 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182491+0000", 2024-09-14T06:17:18.778 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3733000000000001e-05 2024-09-14T06:17:18.778 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.778 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.778 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.778 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182509+0000", 2024-09-14T06:17:18.778 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8113000000000001e-05 2024-09-14T06:17:18.778 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.778 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.778 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.778 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.779 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.779 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13249 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.779 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:18.779 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.563658607999997, 2024-09-14T06:17:18.779 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082037225000000005, 2024-09-14T06:17:18.779 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.779 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.779 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.779 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.779 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.779 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.779 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13249 2024-09-14T06:17:18.780 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.780 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.780 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.780 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.780 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:18.780 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.780 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.780 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.780 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.780 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:18.780 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.780 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.780 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.781 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.781 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:18.781 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4600000000000005e-07 2024-09-14T06:17:18.781 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.781 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.781 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.781 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100584+0000", 2024-09-14T06:17:18.781 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.947e-06 2024-09-14T06:17:18.781 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.781 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.781 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.781 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100593+0000", 2024-09-14T06:17:18.781 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9760000000000001e-06 2024-09-14T06:17:18.781 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.782 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.782 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.782 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100597+0000", 2024-09-14T06:17:18.782 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9689999999999996e-06 2024-09-14T06:17:18.782 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.782 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.782 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.782 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176461+0000", 2024-09-14T06:17:18.782 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075864132000000001 2024-09-14T06:17:18.782 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.782 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.782 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.782 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176516+0000", 2024-09-14T06:17:18.782 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4416999999999998e-05 2024-09-14T06:17:18.783 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.783 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.783 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.783 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176734+0000", 2024-09-14T06:17:18.783 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021859399999999999 2024-09-14T06:17:18.783 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.783 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.783 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.783 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178308+0000", 2024-09-14T06:17:18.783 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015733660000000001 2024-09-14T06:17:18.783 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.783 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.783 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.783 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182573+0000", 2024-09-14T06:17:18.783 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042649200000000002 2024-09-14T06:17:18.784 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.784 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.784 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.784 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182597+0000", 2024-09-14T06:17:18.784 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4355e-05 2024-09-14T06:17:18.784 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.784 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.784 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.784 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182615+0000", 2024-09-14T06:17:18.784 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8003e-05 2024-09-14T06:17:18.784 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.784 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.784 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.784 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.785 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.785 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13250 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:18.785 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:18.785 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.563494003000002, 2024-09-14T06:17:18.785 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081983890000000004, 2024-09-14T06:17:18.785 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:18.785 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:18.785 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:18.785 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:18.785 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:18.785 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:18.785 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13250 2024-09-14T06:17:18.785 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.785 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:18.786 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.786 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:18.786 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:18.786 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.786 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.786 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.786 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:18.786 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:18.786 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:18.786 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.786 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.786 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:18.786 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100743+0000", 2024-09-14T06:17:18.786 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:17:18.787 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.787 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.787 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:18.787 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100753+0000", 2024-09-14T06:17:18.787 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9979999999999998e-06 2024-09-14T06:17:18.787 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.787 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.787 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:18.787 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100772+0000", 2024-09-14T06:17:18.787 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8664e-05 2024-09-14T06:17:18.787 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.787 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.787 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:18.787 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100779+0000", 2024-09-14T06:17:18.787 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.046e-06 2024-09-14T06:17:18.788 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.788 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.788 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:18.788 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177049+0000", 2024-09-14T06:17:18.788 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076270819000000004 2024-09-14T06:17:18.788 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.788 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.788 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:18.788 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177140+0000", 2024-09-14T06:17:18.788 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0746000000000002e-05 2024-09-14T06:17:18.788 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.788 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.788 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:18.789 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177519+0000", 2024-09-14T06:17:18.789 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378656 2024-09-14T06:17:18.789 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.789 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.789 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:18.789 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178349+0000", 2024-09-14T06:17:18.789 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00083061599999999995 2024-09-14T06:17:18.789 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.789 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.789 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:18.789 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182679+0000", 2024-09-14T06:17:18.789 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043298379999999999 2024-09-14T06:17:18.789 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.789 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.790 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:18.790 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182705+0000", 2024-09-14T06:17:18.790 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5602999999999999e-05 2024-09-14T06:17:18.790 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:18.790 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:18.790 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:18.790 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182726+0000", 2024-09-14T06:17:18.790 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1297e-05 2024-09-14T06:17:18.790 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.790 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.790 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.790 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:18.790 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:18.790 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:18.792 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:17:18.792 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:18.790+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:18.806 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-14T06:17:18.892 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:18.889+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:18.993 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:18.989+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:18.994 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:18.994 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:18.995 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:18.995 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:19.009 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:17:19.015 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:19.016 INFO:teuthology.orchestra.run.smithi039.stdout:4134 flatten 2024-09-14T06:17:19.094 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:19.090+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:19.187 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:19.187 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:19.187 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:19.187 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:19.187 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:19.195 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:19.190+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:19.202 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:17:19.296 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:19.294+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:19.337 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:19.337 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:19.337 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:19.337 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:19.337 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.337 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:30547 3.1f 3:f84dfb53:::rbd_data.11529eb229bd.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write 16226533~50642] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T06:17:19.337 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:19.338 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 20.145320495, 2024-09-14T06:17:19.338 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0079770020000001, 2024-09-14T06:17:19.338 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.338 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.338 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:19.338 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.338 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.338 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.338 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 30547 2024-09-14T06:17:19.338 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.338 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.338 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.338 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.339 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:19.339 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.339 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.339 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.339 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.339 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:19.340 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.340 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.340 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.340 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.340 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189811+0000", 2024-09-14T06:17:19.340 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2060000000000001e-06 2024-09-14T06:17:19.340 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.340 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.340 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.340 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189871+0000", 2024-09-14T06:17:19.340 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9862999999999999e-05 2024-09-14T06:17:19.340 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.341 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.341 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.341 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189899+0000", 2024-09-14T06:17:19.341 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7849000000000001e-05 2024-09-14T06:17:19.341 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.341 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.341 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.341 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189915+0000", 2024-09-14T06:17:19.341 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:17:19.341 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.341 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.341 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.342 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190024+0000", 2024-09-14T06:17:19.342 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000109202 2024-09-14T06:17:19.342 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.342 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.342 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:17:19.342 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190041+0000", 2024-09-14T06:17:19.342 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7665000000000001e-05 2024-09-14T06:17:19.342 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.342 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.342 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.342 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188512+0000", 2024-09-14T06:17:19.342 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.99847100700000002 2024-09-14T06:17:19.342 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.342 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.343 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for missing object", 2024-09-14T06:17:19.343 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188646+0000", 2024-09-14T06:17:19.343 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000133986 2024-09-14T06:17:19.343 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.343 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.343 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.343 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193097+0000", 2024-09-14T06:17:19.343 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044507059999999996 2024-09-14T06:17:19.343 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.343 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.343 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.343 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193183+0000", 2024-09-14T06:17:19.343 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5496999999999998e-05 2024-09-14T06:17:19.344 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.344 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.344 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:17:19.344 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193307+0000", 2024-09-14T06:17:19.344 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000124611 2024-09-14T06:17:19.344 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.344 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.344 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:19.344 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.194318+0000", 2024-09-14T06:17:19.344 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0010106 2024-09-14T06:17:19.344 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.344 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.344 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:19.344 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197670+0000", 2024-09-14T06:17:19.345 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033523170000000001 2024-09-14T06:17:19.345 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.345 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.345 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:19.345 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197712+0000", 2024-09-14T06:17:19.345 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1681000000000003e-05 2024-09-14T06:17:19.345 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.345 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.345 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.345 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197786+0000", 2024-09-14T06:17:19.345 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4148999999999998e-05 2024-09-14T06:17:19.345 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.345 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.345 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.346 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.346 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.346 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32022 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:19.346 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:19.346 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.466940284, 2024-09-14T06:17:19.346 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095598104000000003, 2024-09-14T06:17:19.346 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.346 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.346 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.346 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.346 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.346 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.346 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32022 2024-09-14T06:17:19.347 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.347 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.347 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.347 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.347 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:19.347 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.347 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.347 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.347 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.347 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:19.347 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.347 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.347 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.347 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.348 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868190+0000", 2024-09-14T06:17:19.348 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6000000000000003e-07 2024-09-14T06:17:19.348 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.348 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.348 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.348 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868243+0000", 2024-09-14T06:17:19.348 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3443999999999999e-05 2024-09-14T06:17:19.348 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.348 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.348 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.348 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868260+0000", 2024-09-14T06:17:19.348 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6799e-05 2024-09-14T06:17:19.348 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.348 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.349 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.349 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868271+0000", 2024-09-14T06:17:19.349 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0523000000000001e-05 2024-09-14T06:17:19.349 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.349 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.349 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.349 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963502+0000", 2024-09-14T06:17:19.349 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095231466000000001 2024-09-14T06:17:19.349 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.349 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.349 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.349 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963636+0000", 2024-09-14T06:17:19.349 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013388699999999999 2024-09-14T06:17:19.349 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.350 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.350 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.350 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963787+0000", 2024-09-14T06:17:19.350 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000151225 2024-09-14T06:17:19.350 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.350 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.350 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.350 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.350 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.350 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32023 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5816320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:19.350 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:19.350 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.466845158, 2024-09-14T06:17:19.350 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095879753999999998, 2024-09-14T06:17:19.350 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.351 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.351 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.351 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.351 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.351 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.351 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32023 2024-09-14T06:17:19.351 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.351 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.351 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.351 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.351 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:19.351 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.351 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.351 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.352 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.352 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:19.352 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.352 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.352 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.352 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.352 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868285+0000", 2024-09-14T06:17:19.352 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6299999999999993e-07 2024-09-14T06:17:19.352 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.352 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.352 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.352 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868288+0000", 2024-09-14T06:17:19.352 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.18e-06 2024-09-14T06:17:19.353 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.353 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.353 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.353 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868298+0000", 2024-09-14T06:17:19.353 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0029e-05 2024-09-14T06:17:19.353 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.353 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.353 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.353 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868307+0000", 2024-09-14T06:17:19.353 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7229999999999993e-06 2024-09-14T06:17:19.353 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.353 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.353 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.353 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963871+0000", 2024-09-14T06:17:19.354 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095563876000000006 2024-09-14T06:17:19.354 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.354 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.354 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.354 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964015+0000", 2024-09-14T06:17:19.354 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143573 2024-09-14T06:17:19.354 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.354 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.354 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.354 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964164+0000", 2024-09-14T06:17:19.354 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014940999999999999 2024-09-14T06:17:19.354 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.354 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.355 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.355 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.355 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.355 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32024 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:19.355 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:19.355 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.466812014, 2024-09-14T06:17:19.355 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096230231999999999, 2024-09-14T06:17:19.355 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.355 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.355 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.355 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.355 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.355 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.355 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32024 2024-09-14T06:17:19.356 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.356 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.356 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.356 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.356 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:19.356 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.356 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.356 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.356 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.356 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:19.356 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.356 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.356 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.356 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.357 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868318+0000", 2024-09-14T06:17:19.357 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1699999999999997e-07 2024-09-14T06:17:19.357 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.357 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.357 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.357 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868321+0000", 2024-09-14T06:17:19.357 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6769999999999999e-06 2024-09-14T06:17:19.357 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.357 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.357 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.357 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868334+0000", 2024-09-14T06:17:19.357 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2928999999999999e-05 2024-09-14T06:17:19.357 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.357 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.358 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.358 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868341+0000", 2024-09-14T06:17:19.358 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9800000000000001e-06 2024-09-14T06:17:19.358 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.358 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.358 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.358 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964258+0000", 2024-09-14T06:17:19.358 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095917099000000006 2024-09-14T06:17:19.358 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.358 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.358 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.358 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964392+0000", 2024-09-14T06:17:19.358 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013396999999999999 2024-09-14T06:17:19.358 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.359 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.359 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.359 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964548+0000", 2024-09-14T06:17:19.359 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015585999999999999 2024-09-14T06:17:19.359 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.359 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.359 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.359 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.359 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.359 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32025 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:19.359 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:19.359 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.466779358, 2024-09-14T06:17:19.359 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096576591000000003, 2024-09-14T06:17:19.359 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.360 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.360 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.360 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.360 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.360 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.360 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32025 2024-09-14T06:17:19.360 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.360 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.360 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.360 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.360 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:19.360 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.360 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.360 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.361 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.361 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:19.361 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.361 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.361 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.361 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.361 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868351+0000", 2024-09-14T06:17:19.361 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1600000000000001e-07 2024-09-14T06:17:19.361 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.361 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.361 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.361 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868353+0000", 2024-09-14T06:17:19.361 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.627e-06 2024-09-14T06:17:19.361 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.362 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.362 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.362 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868364+0000", 2024-09-14T06:17:19.362 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0893000000000001e-05 2024-09-14T06:17:19.362 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.362 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.362 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.362 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868381+0000", 2024-09-14T06:17:19.362 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6769000000000002e-05 2024-09-14T06:17:19.362 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.362 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.362 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.362 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964632+0000", 2024-09-14T06:17:19.362 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096250589999999997 2024-09-14T06:17:19.362 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.363 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.363 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.363 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964772+0000", 2024-09-14T06:17:19.363 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014045299999999999 2024-09-14T06:17:19.363 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.363 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.363 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.363 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964927+0000", 2024-09-14T06:17:19.363 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015454299999999999 2024-09-14T06:17:19.363 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.363 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.363 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.363 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.363 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.364 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32026 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:19.364 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:19.364 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.466738406999999, 2024-09-14T06:17:19.364 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096901622000000007, 2024-09-14T06:17:19.364 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.364 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.364 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.364 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.364 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.364 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.364 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32026 2024-09-14T06:17:19.364 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.364 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.365 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.365 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.365 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:19.365 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.365 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.365 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.365 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.365 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:19.365 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.365 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.365 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.365 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.365 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868392+0000", 2024-09-14T06:17:19.365 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0699999999999996e-07 2024-09-14T06:17:19.366 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.366 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.366 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.366 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868395+0000", 2024-09-14T06:17:19.366 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9859999999999999e-06 2024-09-14T06:17:19.366 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.366 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.366 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.366 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868405+0000", 2024-09-14T06:17:19.366 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9229999999999997e-06 2024-09-14T06:17:19.366 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.366 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.366 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.367 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868412+0000", 2024-09-14T06:17:19.367 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9670000000000004e-06 2024-09-14T06:17:19.367 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.367 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.367 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.367 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965010+0000", 2024-09-14T06:17:19.367 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096598562999999998 2024-09-14T06:17:19.367 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.367 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.367 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.367 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965143+0000", 2024-09-14T06:17:19.367 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013247700000000001 2024-09-14T06:17:19.367 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.367 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.368 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.368 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965293+0000", 2024-09-14T06:17:19.368 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000149899 2024-09-14T06:17:19.368 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.368 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.368 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.368 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.368 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.368 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32027 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:19.368 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:19.368 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.466670069999999, 2024-09-14T06:17:19.368 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097211306999999997, 2024-09-14T06:17:19.368 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.368 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.369 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.369 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.369 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.369 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.369 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32027 2024-09-14T06:17:19.369 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.369 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.369 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.369 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.369 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:19.369 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.369 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.369 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.370 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.370 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:19.370 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.370 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.370 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.370 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.370 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868461+0000", 2024-09-14T06:17:19.370 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.296e-06 2024-09-14T06:17:19.371 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.371 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.371 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.371 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868465+0000", 2024-09-14T06:17:19.371 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9570000000000004e-06 2024-09-14T06:17:19.371 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.371 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.371 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.371 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868476+0000", 2024-09-14T06:17:19.371 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1365999999999999e-05 2024-09-14T06:17:19.371 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.371 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.372 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.372 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868483+0000", 2024-09-14T06:17:19.372 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2529999999999998e-06 2024-09-14T06:17:19.372 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.372 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.372 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.372 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965375+0000", 2024-09-14T06:17:19.372 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096891973000000006 2024-09-14T06:17:19.372 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.372 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.372 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.372 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965518+0000", 2024-09-14T06:17:19.372 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143017 2024-09-14T06:17:19.372 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.373 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.373 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.373 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965671+0000", 2024-09-14T06:17:19.373 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015244499999999999 2024-09-14T06:17:19.373 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.373 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.373 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.373 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.373 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.373 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32028 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:19.373 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:19.373 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.466635242000001, 2024-09-14T06:17:19.373 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097841480999999994, 2024-09-14T06:17:19.374 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.374 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.374 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.374 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.374 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.374 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.374 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32028 2024-09-14T06:17:19.374 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.374 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.374 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.374 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.374 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:19.374 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.374 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.374 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.375 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.375 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:19.375 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.375 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.375 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.375 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.375 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868495+0000", 2024-09-14T06:17:19.375 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6400000000000001e-07 2024-09-14T06:17:19.375 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.375 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.375 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.375 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868498+0000", 2024-09-14T06:17:19.376 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7760000000000002e-06 2024-09-14T06:17:19.376 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.376 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.376 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.376 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868507+0000", 2024-09-14T06:17:19.376 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1530000000000003e-06 2024-09-14T06:17:19.376 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.376 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.376 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.376 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868516+0000", 2024-09-14T06:17:19.376 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8929999999999994e-06 2024-09-14T06:17:19.376 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.376 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.376 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.377 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965753+0000", 2024-09-14T06:17:19.377 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097236970000000006 2024-09-14T06:17:19.377 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.377 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.377 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.377 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965890+0000", 2024-09-14T06:17:19.377 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013677599999999999 2024-09-14T06:17:19.377 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.377 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.377 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.377 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966336+0000", 2024-09-14T06:17:19.377 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00044614900000000001 2024-09-14T06:17:19.377 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.377 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.378 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.378 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.378 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.378 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32029 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:19.378 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:19.378 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.466604167, 2024-09-14T06:17:19.378 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098577779000000004, 2024-09-14T06:17:19.378 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.378 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.378 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.378 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.378 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.378 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.378 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32029 2024-09-14T06:17:19.379 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.379 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.379 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.379 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.379 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:19.379 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.379 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.379 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.379 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.379 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:19.379 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.379 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.379 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.380 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.380 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868526+0000", 2024-09-14T06:17:19.380 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9300000000000006e-07 2024-09-14T06:17:19.380 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.380 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.380 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.380 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868529+0000", 2024-09-14T06:17:19.380 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5969999999999999e-06 2024-09-14T06:17:19.380 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.380 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.380 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.380 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868538+0000", 2024-09-14T06:17:19.380 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1700000000000003e-06 2024-09-14T06:17:19.380 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.381 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.381 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.381 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868560+0000", 2024-09-14T06:17:19.381 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1596000000000001e-05 2024-09-14T06:17:19.381 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.381 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.381 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.381 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966423+0000", 2024-09-14T06:17:19.381 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097863396000000005 2024-09-14T06:17:19.381 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.381 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.381 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.381 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966578+0000", 2024-09-14T06:17:19.382 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000154792 2024-09-14T06:17:19.382 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.382 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.382 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.382 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967103+0000", 2024-09-14T06:17:19.382 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00052523499999999998 2024-09-14T06:17:19.382 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.382 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.382 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.382 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.382 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.382 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32030 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:19.382 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:19.382 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.466559248999999, 2024-09-14T06:17:19.383 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099324030999999993, 2024-09-14T06:17:19.383 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.383 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.383 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.383 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.383 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.383 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.383 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32030 2024-09-14T06:17:19.383 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.383 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.383 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.383 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.383 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:19.383 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.384 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:19.384 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.384 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868571+0000", 2024-09-14T06:17:19.384 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9699999999999995e-07 2024-09-14T06:17:19.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.384 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868574+0000", 2024-09-14T06:17:19.385 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9529999999999999e-06 2024-09-14T06:17:19.385 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.385 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.385 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.385 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868584+0000", 2024-09-14T06:17:19.385 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7860000000000008e-06 2024-09-14T06:17:19.385 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.385 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.385 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.385 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868591+0000", 2024-09-14T06:17:19.385 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0500000000000003e-06 2024-09-14T06:17:19.385 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.385 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.386 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.386 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967191+0000", 2024-09-14T06:17:19.386 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098600035000000003 2024-09-14T06:17:19.386 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.386 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.386 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.386 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967333+0000", 2024-09-14T06:17:19.386 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014190599999999999 2024-09-14T06:17:19.386 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.386 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.386 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.386 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967894+0000", 2024-09-14T06:17:19.386 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000561504 2024-09-14T06:17:19.386 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.387 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.387 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.387 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.387 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.387 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32031 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:19.387 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:19.387 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.466529687, 2024-09-14T06:17:19.387 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.10010279, 2024-09-14T06:17:19.387 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.387 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.387 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.387 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.387 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.388 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.388 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32031 2024-09-14T06:17:19.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.388 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.388 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.388 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.388 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:19.388 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.388 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.388 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.388 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:19.388 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.389 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.389 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.389 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:19.389 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:17:19.389 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.389 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.389 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.389 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868603+0000", 2024-09-14T06:17:19.389 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.599e-06 2024-09-14T06:17:19.389 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.389 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.389 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.389 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868612+0000", 2024-09-14T06:17:19.389 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.986e-06 2024-09-14T06:17:19.390 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.390 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.390 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868619+0000", 2024-09-14T06:17:19.390 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8900000000000001e-06 2024-09-14T06:17:19.390 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.390 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.390 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967983+0000", 2024-09-14T06:17:19.390 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099363612000000004 2024-09-14T06:17:19.390 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.390 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.390 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968140+0000", 2024-09-14T06:17:19.390 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015788600000000001 2024-09-14T06:17:19.391 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.391 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.391 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.391 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968703+0000", 2024-09-14T06:17:19.391 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00056208300000000001 2024-09-14T06:17:19.391 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.391 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.391 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.391 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.391 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.391 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32032 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6258688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:19.391 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:19.391 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.466501212000001, 2024-09-14T06:17:19.391 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100488088, 2024-09-14T06:17:19.392 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.392 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.392 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.392 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.392 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.392 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.392 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32032 2024-09-14T06:17:19.392 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.392 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.392 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.392 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.392 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:19.392 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.392 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.393 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.393 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:19.393 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.393 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.393 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868629+0000", 2024-09-14T06:17:19.393 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:17:19.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.393 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.393 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868631+0000", 2024-09-14T06:17:19.393 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4099999999999998e-06 2024-09-14T06:17:19.394 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.394 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.394 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.394 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868640+0000", 2024-09-14T06:17:19.394 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0939999999999997e-06 2024-09-14T06:17:19.394 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.394 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.394 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.394 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868647+0000", 2024-09-14T06:17:19.394 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9460000000000001e-06 2024-09-14T06:17:19.394 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.394 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.394 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.394 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968797+0000", 2024-09-14T06:17:19.395 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100149867 2024-09-14T06:17:19.395 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.395 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.395 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.395 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968937+0000", 2024-09-14T06:17:19.395 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013981399999999999 2024-09-14T06:17:19.395 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.395 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.395 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.395 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969116+0000", 2024-09-14T06:17:19.395 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017927399999999999 2024-09-14T06:17:19.395 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.395 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.395 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.396 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.396 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32044 3.1e 3:7e6619d4:::rbd_data.115274ed0aa8.0000000000000068:head [set-alloc-hint object_size 1048576 write_size 1048576,write 835584~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:19.396 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:19.396 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 15.458812129, 2024-09-14T06:17:19.396 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096039367, 2024-09-14T06:17:19.396 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.396 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.396 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:19.396 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.396 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.396 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.396 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32044 2024-09-14T06:17:19.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.397 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.397 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.397 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:19.397 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.397 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.397 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:19.397 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.397 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.397 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876320+0000", 2024-09-14T06:17:19.398 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8270000000000002e-06 2024-09-14T06:17:19.398 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.398 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.398 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.398 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876397+0000", 2024-09-14T06:17:19.398 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6985999999999998e-05 2024-09-14T06:17:19.398 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.398 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.398 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.398 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876431+0000", 2024-09-14T06:17:19.398 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3719e-05 2024-09-14T06:17:19.398 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.398 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.398 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.399 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876452+0000", 2024-09-14T06:17:19.399 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0891999999999999e-05 2024-09-14T06:17:19.399 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.399 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.399 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.399 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969186+0000", 2024-09-14T06:17:19.399 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.092734681999999999 2024-09-14T06:17:19.399 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.399 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.399 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.399 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969550+0000", 2024-09-14T06:17:19.399 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036402199999999998 2024-09-14T06:17:19.399 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.399 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.400 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:17:19.400 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969987+0000", 2024-09-14T06:17:19.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000436759 2024-09-14T06:17:19.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.400 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.400 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:19.400 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971569+0000", 2024-09-14T06:17:19.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001581821 2024-09-14T06:17:19.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.400 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.400 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:19.400 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971947+0000", 2024-09-14T06:17:19.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378446 2024-09-14T06:17:19.401 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.401 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.401 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:19.401 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972146+0000", 2024-09-14T06:17:19.401 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019819 2024-09-14T06:17:19.401 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.401 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.401 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.401 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972357+0000", 2024-09-14T06:17:19.401 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021102299999999999 2024-09-14T06:17:19.401 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.401 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.401 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.401 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.402 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.402 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40252 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 827392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:19.402 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:19.402 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 9.6181760230000002, 2024-09-14T06:17:19.402 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095633474999999996, 2024-09-14T06:17:19.402 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.402 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.402 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.402 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.402 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.402 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.402 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40252 2024-09-14T06:17:19.402 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.402 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.402 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.403 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.403 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:19.403 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.403 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.403 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.403 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:19.403 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.403 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.403 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.403 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717376+0000", 2024-09-14T06:17:19.403 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000422216 2024-09-14T06:17:19.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.404 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.404 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717386+0000", 2024-09-14T06:17:19.404 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0396e-05 2024-09-14T06:17:19.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.404 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.404 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717395+0000", 2024-09-14T06:17:19.404 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7819999999999999e-06 2024-09-14T06:17:19.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.404 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.404 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717400+0000", 2024-09-14T06:17:19.404 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1599999999999997e-06 2024-09-14T06:17:19.405 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.405 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.405 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.405 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.811922+0000", 2024-09-14T06:17:19.405 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.094522018999999999 2024-09-14T06:17:19.405 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.405 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.405 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.405 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812007+0000", 2024-09-14T06:17:19.405 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5328999999999996e-05 2024-09-14T06:17:19.405 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.405 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.405 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.405 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812587+0000", 2024-09-14T06:17:19.406 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000579573 2024-09-14T06:17:19.406 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.406 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.406 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.406 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.406 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.406 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40254 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:19.406 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:19.406 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 9.61771785, 2024-09-14T06:17:19.406 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095562731999999997, 2024-09-14T06:17:19.406 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.406 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.406 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.406 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.407 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.407 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.407 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40254 2024-09-14T06:17:19.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.407 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.407 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.407 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.407 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:19.407 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.407 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.407 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.407 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:19.407 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.408 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.408 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.408 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:19.408 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1699999999999999e-07 2024-09-14T06:17:19.408 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.408 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.408 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.408 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717414+0000", 2024-09-14T06:17:19.408 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:17:19.408 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.408 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.408 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.408 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717419+0000", 2024-09-14T06:17:19.409 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2839999999999997e-06 2024-09-14T06:17:19.409 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.409 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.409 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.409 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717424+0000", 2024-09-14T06:17:19.409 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1549999999999998e-06 2024-09-14T06:17:19.409 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.409 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.409 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.409 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812619+0000", 2024-09-14T06:17:19.409 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095194940000000006 2024-09-14T06:17:19.409 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.409 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.409 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.409 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812717+0000", 2024-09-14T06:17:19.410 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7647999999999997e-05 2024-09-14T06:17:19.410 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.410 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.410 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.410 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812974+0000", 2024-09-14T06:17:19.410 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025752099999999998 2024-09-14T06:17:19.410 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.410 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.410 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.410 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.410 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.410 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40256 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 843776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:19.410 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:19.411 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 9.6176952460000003, 2024-09-14T06:17:19.411 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095932066999999996, 2024-09-14T06:17:19.411 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.411 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.411 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.411 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.411 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.411 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.411 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40256 2024-09-14T06:17:19.411 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.411 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.411 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.411 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.411 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:19.412 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.412 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.412 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.412 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.412 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:19.412 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.412 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.412 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.412 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.412 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717628+0000", 2024-09-14T06:17:19.412 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019339499999999999 2024-09-14T06:17:19.412 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.412 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.412 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.413 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717632+0000", 2024-09-14T06:17:19.413 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2230000000000001e-06 2024-09-14T06:17:19.413 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.413 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.413 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.413 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717641+0000", 2024-09-14T06:17:19.413 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6309999999999998e-06 2024-09-14T06:17:19.413 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.413 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.413 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.413 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717656+0000", 2024-09-14T06:17:19.413 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4724e-05 2024-09-14T06:17:19.413 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.413 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.414 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.414 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813006+0000", 2024-09-14T06:17:19.414 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095349801999999997 2024-09-14T06:17:19.414 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.414 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.414 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.414 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813107+0000", 2024-09-14T06:17:19.414 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000100973 2024-09-14T06:17:19.414 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.414 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.414 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.414 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813366+0000", 2024-09-14T06:17:19.414 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025931899999999999 2024-09-14T06:17:19.414 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.414 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.415 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.415 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.415 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.415 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40258 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 851968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:19.415 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:19.415 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 9.6174591459999998, 2024-09-14T06:17:19.415 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096062543, 2024-09-14T06:17:19.415 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.415 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.415 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.415 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.415 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.415 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.415 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40258 2024-09-14T06:17:19.416 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.416 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.416 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.416 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.416 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:19.416 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.416 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.416 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.416 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.416 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:19.416 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.416 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.416 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.416 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.417 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717671+0000", 2024-09-14T06:17:19.417 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8299999999999997e-07 2024-09-14T06:17:19.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.417 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.417 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.417 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717674+0000", 2024-09-14T06:17:19.417 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7080000000000002e-06 2024-09-14T06:17:19.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.417 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.417 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.417 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717681+0000", 2024-09-14T06:17:19.417 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7889999999999995e-06 2024-09-14T06:17:19.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.417 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.418 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.418 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717687+0000", 2024-09-14T06:17:19.418 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7529999999999998e-06 2024-09-14T06:17:19.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.418 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.418 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.418 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813408+0000", 2024-09-14T06:17:19.418 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095720451999999998 2024-09-14T06:17:19.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.418 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.418 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.418 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813499+0000", 2024-09-14T06:17:19.418 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0933000000000005e-05 2024-09-14T06:17:19.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.418 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.419 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.419 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813733+0000", 2024-09-14T06:17:19.419 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00023432499999999999 2024-09-14T06:17:19.419 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.419 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.419 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.419 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.419 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40260 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 860160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:19.419 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:19.419 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 9.6174332969999998, 2024-09-14T06:17:19.419 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096337381999999999, 2024-09-14T06:17:19.419 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.419 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.420 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.420 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.420 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.420 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.420 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40260 2024-09-14T06:17:19.420 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.420 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.420 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.420 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.420 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:19.420 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.420 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.420 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.420 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.420 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:19.421 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.421 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.421 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.421 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.421 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718008+0000", 2024-09-14T06:17:19.421 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00031192699999999998 2024-09-14T06:17:19.421 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.421 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.421 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.421 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718013+0000", 2024-09-14T06:17:19.421 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1830000000000002e-06 2024-09-14T06:17:19.421 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.421 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.421 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.422 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718036+0000", 2024-09-14T06:17:19.422 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2721000000000002e-05 2024-09-14T06:17:19.422 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.422 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.422 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.422 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718053+0000", 2024-09-14T06:17:19.422 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6736e-05 2024-09-14T06:17:19.422 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.422 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.422 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.422 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813767+0000", 2024-09-14T06:17:19.422 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095714658999999994 2024-09-14T06:17:19.422 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.422 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.423 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.423 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813816+0000", 2024-09-14T06:17:19.423 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8848999999999999e-05 2024-09-14T06:17:19.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.423 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.423 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.423 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814034+0000", 2024-09-14T06:17:19.423 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000217307 2024-09-14T06:17:19.423 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.423 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.423 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.423 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.423 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40262 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 868352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:19.424 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:19.424 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 9.6170336140000003, 2024-09-14T06:17:19.424 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096354165000000006, 2024-09-14T06:17:19.424 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.424 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.424 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.424 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.424 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.424 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.424 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40262 2024-09-14T06:17:19.424 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.424 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.424 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.424 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.425 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:19.425 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.425 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.425 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.425 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:19.425 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.425 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.425 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.425 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718281+0000", 2024-09-14T06:17:19.425 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000185069 2024-09-14T06:17:19.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.425 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.426 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.426 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718286+0000", 2024-09-14T06:17:19.426 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.118e-06 2024-09-14T06:17:19.426 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.426 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.426 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.426 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718295+0000", 2024-09-14T06:17:19.426 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0429999999999996e-06 2024-09-14T06:17:19.426 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.426 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.426 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.426 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718302+0000", 2024-09-14T06:17:19.426 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2790000000000001e-06 2024-09-14T06:17:19.426 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.427 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.427 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.427 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814076+0000", 2024-09-14T06:17:19.427 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095774076999999999 2024-09-14T06:17:19.427 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.427 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.427 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.427 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814191+0000", 2024-09-14T06:17:19.427 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00011500700000000001 2024-09-14T06:17:19.427 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.427 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.427 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.427 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814450+0000", 2024-09-14T06:17:19.427 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025857199999999999 2024-09-14T06:17:19.427 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.428 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.428 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.428 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.428 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.428 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40264 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 876544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:19.428 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:19.428 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 9.6166765460000008, 2024-09-14T06:17:19.428 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096372478999999997, 2024-09-14T06:17:19.428 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.428 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.428 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:19.428 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.428 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.429 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.429 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40264 2024-09-14T06:17:19.429 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.429 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.429 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.429 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.429 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:19.429 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.429 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.429 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.429 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.429 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:19.429 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.429 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.430 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.430 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.430 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718793+0000", 2024-09-14T06:17:19.430 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00033991099999999998 2024-09-14T06:17:19.430 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.430 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.430 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.430 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718805+0000", 2024-09-14T06:17:19.430 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2514000000000001e-05 2024-09-14T06:17:19.430 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.430 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.430 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.430 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718815+0000", 2024-09-14T06:17:19.430 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9590000000000007e-06 2024-09-14T06:17:19.430 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.431 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.431 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.431 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718822+0000", 2024-09-14T06:17:19.431 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2030000000000003e-06 2024-09-14T06:17:19.431 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.431 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.431 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.431 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814490+0000", 2024-09-14T06:17:19.431 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095667346 2024-09-14T06:17:19.431 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.431 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.431 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.431 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814577+0000", 2024-09-14T06:17:19.431 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6920000000000001e-05 2024-09-14T06:17:19.432 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.432 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.432 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.432 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814825+0000", 2024-09-14T06:17:19.432 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000248626 2024-09-14T06:17:19.432 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.432 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.432 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.432 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.432 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.432 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:19.433 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:17:19.434 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:17:19.435 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:19.430+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:19.534 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:19.530+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:19.570 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:19.570 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:19.570 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:19.570 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:19.587 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:17:19.635 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:19.633+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:19.703 INFO:teuthology.orchestra.run.smithi039.stdout:4136 write 0x488c175 thru 0x488fb4f (0x39db bytes) 2024-09-14T06:17:19.707 INFO:teuthology.orchestra.run.smithi039.stdout:4137 write 0x21a3a67 thru 0x21a8e3d (0x53d7 bytes) 2024-09-14T06:17:19.713 INFO:teuthology.orchestra.run.smithi039.stdout:4138 punch from 0x55190a8 to 0x5528fbc, (0xff14 bytes) 2024-09-14T06:17:19.713 INFO:teuthology.orchestra.run.smithi039.stdout:4139 punch from 0x47a1a33 to 0x47a5c2c, (0x41f9 bytes) 2024-09-14T06:17:19.713 INFO:teuthology.orchestra.run.smithi039.stdout:4140 punch from 0x6d970f0 to 0x6d9c627, (0x5537 bytes) 2024-09-14T06:17:19.713 INFO:teuthology.orchestra.run.smithi039.stdout:4141 read 0x76c2021 thru 0x76cd2b5 (0xb295 bytes) 2024-09-14T06:17:19.713 INFO:teuthology.orchestra.run.smithi039.stdout:4142 clone 25 order 25 su 8388608 sc 6 2024-09-14T06:17:19.714 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:19.714 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:19.714 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:19.714 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:19.714 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:19.728 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:17:19.737 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:19.734+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:19.838 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:19.834+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:19.894 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:19.894 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:19.895 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:19.895 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:19.895 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.895 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:3595 3.1 3:811c1c24:::rbd_data.1152bd92e04f.000000000000006e:head [set-alloc-hint object_size 2097152 write_size 2097152,write 440183~2185] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T06:17:19.895 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:19.895 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.670068200000003, 2024-09-14T06:17:19.895 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0365806689999999, 2024-09-14T06:17:19.895 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.895 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.895 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.895 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.895 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.895 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.895 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 3595 2024-09-14T06:17:19.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.896 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.896 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.896 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.896 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:19.896 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.896 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.896 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.896 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:19.896 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.897 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.897 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.897 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222823+0000", 2024-09-14T06:17:19.897 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8219999999999998e-06 2024-09-14T06:17:19.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.897 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.897 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.897 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222835+0000", 2024-09-14T06:17:19.897 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2052000000000001e-05 2024-09-14T06:17:19.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.897 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.897 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.897 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222859+0000", 2024-09-14T06:17:19.898 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3694e-05 2024-09-14T06:17:19.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.898 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.898 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.898 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222872+0000", 2024-09-14T06:17:19.898 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3203999999999999e-05 2024-09-14T06:17:19.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.898 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.898 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.898 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223068+0000", 2024-09-14T06:17:19.898 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000195995 2024-09-14T06:17:19.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.898 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.898 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:19.899 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223312+0000", 2024-09-14T06:17:19.899 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024424199999999998 2024-09-14T06:17:19.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.899 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.899 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:19.899 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223786+0000", 2024-09-14T06:17:19.899 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00047356600000000003 2024-09-14T06:17:19.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.899 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.899 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:19.899 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.225374+0000", 2024-09-14T06:17:19.899 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015875150000000001 2024-09-14T06:17:19.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.899 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.900 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.900 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.233122+0000", 2024-09-14T06:17:19.900 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0077480700000000001 2024-09-14T06:17:19.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.900 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.900 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:17:19.900 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.233157+0000", 2024-09-14T06:17:19.900 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5113000000000001e-05 2024-09-14T06:17:19.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.900 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.900 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.900 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.252998+0000", 2024-09-14T06:17:19.900 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.019841142 2024-09-14T06:17:19.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.901 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for degraded object", 2024-09-14T06:17:19.901 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.254121+0000", 2024-09-14T06:17:19.901 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0011227979999999999 2024-09-14T06:17:19.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.901 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.901 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.259263+0000", 2024-09-14T06:17:19.901 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0051426700000000002 2024-09-14T06:17:19.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.901 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.902 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.259401+0000", 2024-09-14T06:17:19.902 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013778600000000001 2024-09-14T06:17:19.902 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.902 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.902 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.902 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.902 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11421 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.902 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:19.902 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.907173014000001, 2024-09-14T06:17:19.902 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078553170000000005, 2024-09-14T06:17:19.902 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.902 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.902 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.902 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.903 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.903 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.903 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11421 2024-09-14T06:17:19.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.903 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.903 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:19.903 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.903 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:19.904 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.904 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.904 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.904 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985718+0000", 2024-09-14T06:17:19.904 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8840000000000001e-06 2024-09-14T06:17:19.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.904 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.904 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.904 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985723+0000", 2024-09-14T06:17:19.904 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8799999999999996e-06 2024-09-14T06:17:19.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.904 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.905 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.905 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985742+0000", 2024-09-14T06:17:19.905 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8776000000000001e-05 2024-09-14T06:17:19.905 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.905 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.905 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.905 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985759+0000", 2024-09-14T06:17:19.905 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7119999999999999e-05 2024-09-14T06:17:19.905 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.905 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.905 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.905 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064259+0000", 2024-09-14T06:17:19.905 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078499418000000001 2024-09-14T06:17:19.905 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.906 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.906 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.906 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064269+0000", 2024-09-14T06:17:19.906 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:17:19.906 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.906 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.906 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.906 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.906 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.906 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11422 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.906 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:19.906 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.907103047, 2024-09-14T06:17:19.906 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078521540000000001, 2024-09-14T06:17:19.906 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.907 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.907 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.907 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.907 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.907 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.907 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11422 2024-09-14T06:17:19.907 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.907 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.907 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.907 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.907 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:19.907 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.907 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.908 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.908 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.908 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:19.908 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.908 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.908 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.908 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985787+0000", 2024-09-14T06:17:19.908 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7630000000000001e-06 2024-09-14T06:17:19.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.908 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.908 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.908 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985842+0000", 2024-09-14T06:17:19.909 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4555999999999997e-05 2024-09-14T06:17:19.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.909 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985864+0000", 2024-09-14T06:17:19.909 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2189999999999999e-05 2024-09-14T06:17:19.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.909 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985879+0000", 2024-09-14T06:17:19.909 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4382999999999999e-05 2024-09-14T06:17:19.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.910 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.910 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064298+0000", 2024-09-14T06:17:19.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078419121999999994 2024-09-14T06:17:19.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.910 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.910 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064307+0000", 2024-09-14T06:17:19.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5259999999999999e-06 2024-09-14T06:17:19.910 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.910 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.910 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.911 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.911 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11423 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.911 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:19.911 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.906985675000001, 2024-09-14T06:17:19.911 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078437175999999997, 2024-09-14T06:17:19.911 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.911 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.911 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.911 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.911 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.911 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.911 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11423 2024-09-14T06:17:19.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.912 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.912 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.912 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.912 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:19.912 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.912 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.912 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.912 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.912 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:19.912 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.912 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.912 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.912 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.913 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985905+0000", 2024-09-14T06:17:19.913 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8139999999999999e-06 2024-09-14T06:17:19.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.913 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.913 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.913 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985911+0000", 2024-09-14T06:17:19.913 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7230000000000001e-06 2024-09-14T06:17:19.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.913 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.913 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.913 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985930+0000", 2024-09-14T06:17:19.913 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9360000000000001e-05 2024-09-14T06:17:19.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985944+0000", 2024-09-14T06:17:19.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4479e-05 2024-09-14T06:17:19.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064332+0000", 2024-09-14T06:17:19.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078387686999999998 2024-09-14T06:17:19.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064340+0000", 2024-09-14T06:17:19.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1130000000000001e-06 2024-09-14T06:17:19.915 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.915 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.915 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.915 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11480 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.915 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:19.915 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.905635199000002, 2024-09-14T06:17:19.915 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077122298000000006, 2024-09-14T06:17:19.915 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.915 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.915 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.915 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.915 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.916 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.916 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11480 2024-09-14T06:17:19.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.916 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:19.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:19.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.917 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.917 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.917 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987255+0000", 2024-09-14T06:17:19.917 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:17:19.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.917 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.917 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.917 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987263+0000", 2024-09-14T06:17:19.917 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6499999999999996e-06 2024-09-14T06:17:19.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.917 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.917 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.917 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987285+0000", 2024-09-14T06:17:19.918 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1860000000000001e-05 2024-09-14T06:17:19.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.918 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.918 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.918 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987341+0000", 2024-09-14T06:17:19.918 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6648999999999999e-05 2024-09-14T06:17:19.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.918 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.918 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.918 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064367+0000", 2024-09-14T06:17:19.918 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077025622000000002 2024-09-14T06:17:19.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.918 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.919 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.919 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064376+0000", 2024-09-14T06:17:19.919 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7499999999999992e-06 2024-09-14T06:17:19.919 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.919 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.919 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.919 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.919 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11481 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.919 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:19.919 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.905499497000001, 2024-09-14T06:17:19.919 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077022057000000005, 2024-09-14T06:17:19.919 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.919 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.920 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.920 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.920 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.920 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.920 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11481 2024-09-14T06:17:19.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.920 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.920 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.920 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:19.920 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.920 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.921 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.921 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:19.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.921 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.921 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987391+0000", 2024-09-14T06:17:19.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4839999999999999e-06 2024-09-14T06:17:19.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.921 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.921 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987399+0000", 2024-09-14T06:17:19.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8660000000000006e-06 2024-09-14T06:17:19.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.922 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.922 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.922 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987419+0000", 2024-09-14T06:17:19.922 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9984e-05 2024-09-14T06:17:19.922 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.922 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.922 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.922 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987434+0000", 2024-09-14T06:17:19.922 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5396e-05 2024-09-14T06:17:19.922 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.922 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.922 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.922 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064403+0000", 2024-09-14T06:17:19.922 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076969212999999995 2024-09-14T06:17:19.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.923 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.923 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.923 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064411+0000", 2024-09-14T06:17:19.923 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1140000000000006e-06 2024-09-14T06:17:19.923 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.923 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.923 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.923 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.923 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11482 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.923 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:19.923 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.905414637, 2024-09-14T06:17:19.923 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076971172000000004, 2024-09-14T06:17:19.923 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.924 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.924 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.924 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.924 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.924 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.924 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11482 2024-09-14T06:17:19.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.924 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.924 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.924 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.924 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:19.924 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:19.925 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.925 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987476+0000", 2024-09-14T06:17:19.925 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.793e-06 2024-09-14T06:17:19.925 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987482+0000", 2024-09-14T06:17:19.925 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8270000000000003e-06 2024-09-14T06:17:19.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.926 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.926 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987502+0000", 2024-09-14T06:17:19.926 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0656e-05 2024-09-14T06:17:19.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.926 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.926 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987518+0000", 2024-09-14T06:17:19.926 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:17:19.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.926 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.927 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064438+0000", 2024-09-14T06:17:19.927 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076919623000000006 2024-09-14T06:17:19.927 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.927 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.927 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.927 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064445+0000", 2024-09-14T06:17:19.927 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.61e-06 2024-09-14T06:17:19.927 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.927 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.927 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.927 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.927 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.927 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11483 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.928 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:19.928 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.905346788000003, 2024-09-14T06:17:19.928 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076936562999999999, 2024-09-14T06:17:19.928 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.928 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.928 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.928 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.928 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.928 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.928 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11483 2024-09-14T06:17:19.928 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.928 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.928 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:19.929 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.929 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:19.929 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.929 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987544+0000", 2024-09-14T06:17:19.929 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.654e-06 2024-09-14T06:17:19.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.930 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987550+0000", 2024-09-14T06:17:19.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0299999999999999e-06 2024-09-14T06:17:19.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.930 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987570+0000", 2024-09-14T06:17:19.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0080000000000001e-05 2024-09-14T06:17:19.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.930 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987619+0000", 2024-09-14T06:17:19.931 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9228999999999999e-05 2024-09-14T06:17:19.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.931 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.931 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.931 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064471+0000", 2024-09-14T06:17:19.931 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076852086999999999 2024-09-14T06:17:19.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.931 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.931 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.931 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064478+0000", 2024-09-14T06:17:19.931 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4830000000000002e-06 2024-09-14T06:17:19.931 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.931 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.932 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.932 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.932 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11485 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.932 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:19.932 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.905248295, 2024-09-14T06:17:19.932 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076868463999999997, 2024-09-14T06:17:19.932 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.932 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.932 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.932 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.932 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.932 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.932 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11485 2024-09-14T06:17:19.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.933 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.933 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.933 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:19.933 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.933 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.933 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:19.933 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.933 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987642+0000", 2024-09-14T06:17:19.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4100000000000001e-06 2024-09-14T06:17:19.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987648+0000", 2024-09-14T06:17:19.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2500000000000003e-06 2024-09-14T06:17:19.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987683+0000", 2024-09-14T06:17:19.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5193e-05 2024-09-14T06:17:19.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.935 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.935 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987698+0000", 2024-09-14T06:17:19.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4893e-05 2024-09-14T06:17:19.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.935 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.935 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.935 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064501+0000", 2024-09-14T06:17:19.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076803263999999996 2024-09-14T06:17:19.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.935 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.935 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.935 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064509+0000", 2024-09-14T06:17:19.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4540000000000001e-06 2024-09-14T06:17:19.935 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.936 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.936 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.936 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.936 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11490 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.936 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:19.936 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.905098725999999, 2024-09-14T06:17:19.936 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076755043999999994, 2024-09-14T06:17:19.936 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.936 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.936 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.936 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.936 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.936 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.937 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11490 2024-09-14T06:17:19.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.937 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.937 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.937 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:19.937 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.937 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.937 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:19.937 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987791+0000", 2024-09-14T06:17:19.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:17:19.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987797+0000", 2024-09-14T06:17:19.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7969999999999997e-06 2024-09-14T06:17:19.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987817+0000", 2024-09-14T06:17:19.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9746e-05 2024-09-14T06:17:19.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987832+0000", 2024-09-14T06:17:19.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.452e-05 2024-09-14T06:17:19.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064535+0000", 2024-09-14T06:17:19.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076703478000000005 2024-09-14T06:17:19.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064545+0000", 2024-09-14T06:17:19.940 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9799999999999993e-06 2024-09-14T06:17:19.940 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.940 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.940 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.940 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.940 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11492 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.940 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:19.940 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.904971861, 2024-09-14T06:17:19.940 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076661759999999995, 2024-09-14T06:17:19.940 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.940 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.940 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.940 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.941 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.941 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.941 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11492 2024-09-14T06:17:19.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.941 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:19.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:19.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987918+0000", 2024-09-14T06:17:19.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3239999999999999e-06 2024-09-14T06:17:19.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987924+0000", 2024-09-14T06:17:19.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6930000000000004e-06 2024-09-14T06:17:19.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987943+0000", 2024-09-14T06:17:19.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9487000000000002e-05 2024-09-14T06:17:19.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.943 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.943 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987958+0000", 2024-09-14T06:17:19.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4453e-05 2024-09-14T06:17:19.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.943 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.943 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064571+0000", 2024-09-14T06:17:19.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076612988000000007 2024-09-14T06:17:19.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.944 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.944 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064579+0000", 2024-09-14T06:17:19.944 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8150000000000006e-06 2024-09-14T06:17:19.944 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.944 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.944 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.944 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11495 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.944 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:19.944 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.904803172000001, 2024-09-14T06:17:19.944 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076524746000000005, 2024-09-14T06:17:19.944 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.945 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.945 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.945 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.945 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.945 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.945 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11495 2024-09-14T06:17:19.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.945 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:19.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:19.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988087+0000", 2024-09-14T06:17:19.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0499999999999999e-06 2024-09-14T06:17:19.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988118+0000", 2024-09-14T06:17:19.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1900000000000003e-05 2024-09-14T06:17:19.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988138+0000", 2024-09-14T06:17:19.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.933e-05 2024-09-14T06:17:19.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988152+0000", 2024-09-14T06:17:19.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3889e-05 2024-09-14T06:17:19.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064603+0000", 2024-09-14T06:17:19.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076451082000000004 2024-09-14T06:17:19.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.948 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.948 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.948 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064610+0000", 2024-09-14T06:17:19.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4950000000000002e-06 2024-09-14T06:17:19.948 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.948 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.948 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.948 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.948 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11498 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.948 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:19.948 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.904717095999999, 2024-09-14T06:17:19.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076471883000000004, 2024-09-14T06:17:19.949 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.949 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.949 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.949 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.949 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.949 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.949 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11498 2024-09-14T06:17:19.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.949 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.949 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.949 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:19.949 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:19.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988173+0000", 2024-09-14T06:17:19.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2300000000000001e-06 2024-09-14T06:17:19.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988178+0000", 2024-09-14T06:17:19.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4229999999999996e-06 2024-09-14T06:17:19.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988235+0000", 2024-09-14T06:17:19.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6616999999999997e-05 2024-09-14T06:17:19.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988264+0000", 2024-09-14T06:17:19.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9189000000000001e-05 2024-09-14T06:17:19.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064636+0000", 2024-09-14T06:17:19.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076371573999999998 2024-09-14T06:17:19.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.952 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.952 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064643+0000", 2024-09-14T06:17:19.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8499999999999994e-06 2024-09-14T06:17:19.952 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.952 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.952 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.952 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11523 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.952 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:19.952 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.903996948, 2024-09-14T06:17:19.953 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076062454000000002, 2024-09-14T06:17:19.953 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.953 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.953 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.953 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.953 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.953 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.953 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11523 2024-09-14T06:17:19.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.953 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.953 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.953 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.953 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:19.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:19.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988894+0000", 2024-09-14T06:17:19.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.843e-06 2024-09-14T06:17:19.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988903+0000", 2024-09-14T06:17:19.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0440000000000001e-06 2024-09-14T06:17:19.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988929+0000", 2024-09-14T06:17:19.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6679e-05 2024-09-14T06:17:19.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988967+0000", 2024-09-14T06:17:19.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7453000000000002e-05 2024-09-14T06:17:19.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064668+0000", 2024-09-14T06:17:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075701389999999993 2024-09-14T06:17:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064954+0000", 2024-09-14T06:17:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028604500000000002 2024-09-14T06:17:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11530 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.903506397999998, 2024-09-14T06:17:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075620674999999998, 2024-09-14T06:17:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11530 2024-09-14T06:17:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989384+0000", 2024-09-14T06:17:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6160000000000001e-06 2024-09-14T06:17:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989390+0000", 2024-09-14T06:17:19.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7039999999999999e-06 2024-09-14T06:17:19.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989409+0000", 2024-09-14T06:17:19.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.95e-05 2024-09-14T06:17:19.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989429+0000", 2024-09-14T06:17:19.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0173000000000002e-05 2024-09-14T06:17:19.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.960 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.960 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064992+0000", 2024-09-14T06:17:19.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075562764000000004 2024-09-14T06:17:19.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.960 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.960 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065003+0000", 2024-09-14T06:17:19.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0917999999999999e-05 2024-09-14T06:17:19.960 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.960 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.960 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.961 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.961 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.961 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11532 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.961 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:19.961 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.903436001999999, 2024-09-14T06:17:19.961 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075604656000000006, 2024-09-14T06:17:19.961 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.961 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.961 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.961 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.961 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.961 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.961 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11532 2024-09-14T06:17:19.961 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.962 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.962 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.962 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.962 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:19.962 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.962 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.962 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.962 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:19.962 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.962 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.962 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.962 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989454+0000", 2024-09-14T06:17:19.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6500000000000001e-06 2024-09-14T06:17:19.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989460+0000", 2024-09-14T06:17:19.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7459999999999997e-06 2024-09-14T06:17:19.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989480+0000", 2024-09-14T06:17:19.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.948e-05 2024-09-14T06:17:19.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989497+0000", 2024-09-14T06:17:19.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7153e-05 2024-09-14T06:17:19.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065046+0000", 2024-09-14T06:17:19.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075548905999999999 2024-09-14T06:17:19.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065057+0000", 2024-09-14T06:17:19.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1721e-05 2024-09-14T06:17:19.964 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.964 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.965 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.965 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.965 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11534 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.965 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:19.965 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.903369515999998, 2024-09-14T06:17:19.965 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075574350999999998, 2024-09-14T06:17:19.965 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.965 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.965 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.965 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.965 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.965 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.966 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11534 2024-09-14T06:17:19.966 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.966 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.966 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.966 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.966 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:19.966 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.966 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.966 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.966 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.966 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:19.966 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.966 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.966 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989521+0000", 2024-09-14T06:17:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.533e-06 2024-09-14T06:17:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989527+0000", 2024-09-14T06:17:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0700000000000003e-06 2024-09-14T06:17:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989546+0000", 2024-09-14T06:17:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.948e-05 2024-09-14T06:17:19.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.968 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.968 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.968 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989561+0000", 2024-09-14T06:17:19.968 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4827000000000001e-05 2024-09-14T06:17:19.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.968 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.968 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.968 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065086+0000", 2024-09-14T06:17:19.968 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075524542 2024-09-14T06:17:19.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.968 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.968 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.968 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065094+0000", 2024-09-14T06:17:19.969 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8990000000000001e-06 2024-09-14T06:17:19.969 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.969 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.969 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.969 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11535 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.969 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:19.969 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.903302152999998, 2024-09-14T06:17:19.969 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075542466000000003, 2024-09-14T06:17:19.969 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.969 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.969 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.970 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.970 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.970 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.970 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11535 2024-09-14T06:17:19.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.970 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.970 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.970 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.970 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:19.970 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.970 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.970 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.970 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:19.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.971 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.971 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.971 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989588+0000", 2024-09-14T06:17:19.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1659999999999999e-06 2024-09-14T06:17:19.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.971 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.971 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.971 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989593+0000", 2024-09-14T06:17:19.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6230000000000002e-06 2024-09-14T06:17:19.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.971 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.971 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.972 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989613+0000", 2024-09-14T06:17:19.972 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9604000000000001e-05 2024-09-14T06:17:19.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.972 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.972 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.972 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989643+0000", 2024-09-14T06:17:19.972 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0450000000000002e-05 2024-09-14T06:17:19.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.972 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.972 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.972 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065119+0000", 2024-09-14T06:17:19.972 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075475930999999996 2024-09-14T06:17:19.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.972 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065129+0000", 2024-09-14T06:17:19.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6919999999999997e-06 2024-09-14T06:17:19.973 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.973 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.973 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.973 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.973 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11537 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.973 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:19.973 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.903224229999999, 2024-09-14T06:17:19.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075497868999999995, 2024-09-14T06:17:19.973 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.974 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.974 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.974 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.974 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.974 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.974 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11537 2024-09-14T06:17:19.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.974 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.974 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.974 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.974 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:19.974 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.974 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.975 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.975 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:19.975 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.975 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.975 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.975 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989666+0000", 2024-09-14T06:17:19.975 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.48e-06 2024-09-14T06:17:19.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.975 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.975 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.975 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989672+0000", 2024-09-14T06:17:19.975 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7960000000000001e-06 2024-09-14T06:17:19.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.975 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989690+0000", 2024-09-14T06:17:19.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8743e-05 2024-09-14T06:17:19.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.976 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989704+0000", 2024-09-14T06:17:19.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3913e-05 2024-09-14T06:17:19.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.976 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065155+0000", 2024-09-14T06:17:19.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075450351999999998 2024-09-14T06:17:19.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.977 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.977 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065162+0000", 2024-09-14T06:17:19.977 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5850000000000002e-06 2024-09-14T06:17:19.977 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.977 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.977 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.977 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11543 3.12 3.4d02a8b2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:19.977 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:19.977 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 43.902946665999998, 2024-09-14T06:17:19.977 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074335152000000002, 2024-09-14T06:17:19.978 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:19.978 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:19.978 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:19.978 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:19.978 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:19.978 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:19.978 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11543 2024-09-14T06:17:19.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.978 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:19.978 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.978 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:19.978 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:19.978 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:19.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:19.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:19.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:19.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989944+0000", 2024-09-14T06:17:19.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.694e-06 2024-09-14T06:17:19.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:19.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989950+0000", 2024-09-14T06:17:19.980 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3389999999999998e-06 2024-09-14T06:17:19.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.980 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.980 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:19.980 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989969+0000", 2024-09-14T06:17:19.980 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8963000000000001e-05 2024-09-14T06:17:19.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.980 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.980 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:19.980 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989985+0000", 2024-09-14T06:17:19.980 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5586999999999998e-05 2024-09-14T06:17:19.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.980 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.980 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:19.981 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064270+0000", 2024-09-14T06:17:19.981 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074285552000000005 2024-09-14T06:17:19.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:19.981 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:19.981 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:19.981 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064277+0000", 2024-09-14T06:17:19.981 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0169999999999999e-06 2024-09-14T06:17:19.981 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.981 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.981 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.981 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:19.981 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:19.981 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:19.983 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:17:19.983 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:19.981+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:20.008 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:17:20.084 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:20.081+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:20.152 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:20.153 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:20.153 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:20.153 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:20.165 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:20.166 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:20.167 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:17:20.182 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:20.199 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:20.199 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:20.211 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:20.209+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:20.215 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:20.215 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:20.235 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:20.236 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:20.254 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:20.255 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:20.272 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:20.273 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:20.285 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:20.281+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:20.291 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:20.292 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:20.336 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:20.336 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:20.350 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:17:20.386 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:20.382+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:20.439 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-09-14T06:17:20.488 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:20.485+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:20.525 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:20.525 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:20.542 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:17:20.589 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:20.585+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:20.676 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:20.676 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:20.689 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:20.686+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:20.693 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:17:20.745 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-14T06:17:20.790 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:20.785+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:20.881 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:20.881 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:20.891 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:20.889+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:20.902 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:17:20.995 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:20.994+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:21.068 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:21.068 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:21.084 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:17:21.093 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:21.089+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:21.116 INFO:teuthology.orchestra.run.smithi039.stdout:4143 trunc from 0xc0f4f8f to 0x9cf26d4 2024-09-14T06:17:21.164 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:21.165 INFO:teuthology.orchestra.run.smithi039.stdout:4145 write 0x42d43a1 thru 0x42d5040 (0xca0 bytes) 2024-09-14T06:17:21.167 INFO:teuthology.orchestra.run.smithi039.stdout:4149 write 0x2701e7 thru 0x27ef58 (0xed72 bytes) 2024-09-14T06:17:21.192 INFO:teuthology.orchestra.run.smithi039.stdout:4150 read 0x82f5956 thru 0x830422c (0xe8d7 bytes) 2024-09-14T06:17:21.193 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:21.190+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:21.193 INFO:teuthology.orchestra.run.smithi039.stdout:4151 read 0x7fed8f3 thru 0x7ffd747 (0xfe55 bytes) 2024-09-14T06:17:21.200 INFO:teuthology.orchestra.run.smithi039.stdout:4155 trunc from 0x9cf26d4 to 0x65d122c 2024-09-14T06:17:21.212 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:21.213 INFO:teuthology.orchestra.run.smithi039.stdout:4157 write 0x6142c5b thru 0x614e5e8 (0xb98e bytes) 2024-09-14T06:17:21.239 INFO:teuthology.orchestra.run.smithi039.stdout:4158 write 0xceb118f thru 0xceb3d3c (0x2bae bytes) 2024-09-14T06:17:21.245 INFO:teuthology.orchestra.run.smithi039.stdout:4163 write 0x6031a87 thru 0x6033ef5 (0x246f bytes) 2024-09-14T06:17:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:4164 punch from 0xc427f98 to 0xc42ef28, (0x6f90 bytes) 2024-09-14T06:17:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:4165 read 0x224309a thru 0x2248586 (0x54ed bytes) 2024-09-14T06:17:21.250 INFO:teuthology.orchestra.run.smithi039.stdout:4167 write 0x9457f5a thru 0x945f156 (0x71fd bytes) 2024-09-14T06:17:21.253 INFO:teuthology.orchestra.run.smithi039.stdout:4169 writesame 0xaff709a thru 0xafffb60 data_size 0x1(0x8ac7 bytes) 2024-09-14T06:17:21.260 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:21.261 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:21.276 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:17:21.294 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:21.289+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:21.303 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:21.303 INFO:teuthology.orchestra.run.smithi039.stdout:4171 write 0x7eafc6b thru 0x7ebf3c5 (0xf75b bytes) 2024-09-14T06:17:21.307 INFO:teuthology.orchestra.run.smithi039.stdout:4174 read 0x310b6da thru 0x311a58d (0xeeb4 bytes) 2024-09-14T06:17:21.308 INFO:teuthology.orchestra.run.smithi039.stdout:4177 read 0x4c318c6 thru 0x4c31ffa (0x735 bytes) 2024-09-14T06:17:21.309 INFO:teuthology.orchestra.run.smithi039.stdout:4180 write 0xaedbcdb thru 0xaeeab06 (0xee2c bytes) 2024-09-14T06:17:21.313 INFO:teuthology.orchestra.run.smithi039.stdout:4181 writesame 0x7ca4bb1 thru 0x7cb4198 data_size 0xc(0xf5e8 bytes) 2024-09-14T06:17:21.319 INFO:teuthology.orchestra.run.smithi039.stdout:4183 punch from 0xb348a22 to 0xb34bce5, (0x32c3 bytes) 2024-09-14T06:17:21.320 INFO:teuthology.orchestra.run.smithi039.stdout:4184 writesame 0x1a5e5b8 thru 0x1a69fb4 data_size 0x3b(0xb9fd bytes) 2024-09-14T06:17:21.323 INFO:teuthology.orchestra.run.smithi039.stdout:4185 read 0x4372776 thru 0x4382108 (0xf993 bytes) 2024-09-14T06:17:21.324 INFO:teuthology.orchestra.run.smithi039.stdout:4186 clone 26 order 25 su 524288 sc 5 2024-09-14T06:17:21.396 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:21.393+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:21.432 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:21.432 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:21.446 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:17:21.496 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:21.493+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:21.562 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone25 from 0xceb3d3d (overlap 0x65d122c) to 0x3d1714d 2024-09-14T06:17:21.597 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:21.593+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:21.630 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:21.631 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:21.647 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:17:21.698 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:21.694+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:21.738 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-14T06:17:21.773 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:21.773 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:21.791 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:17:21.799 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:21.797+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:21.900 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:21.897+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:21.960 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:21.961 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:21.977 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:17:22.024 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:22.022+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:22.101 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:22.097+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:22.104 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:22.104 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:22.121 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:17:22.170 INFO:teuthology.orchestra.run.smithi039.stdout:4188 trunc from 0xceb3d3d to 0x1d859f 2024-09-14T06:17:22.202 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:22.198+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:22.245 INFO:teuthology.orchestra.run.smithi039.stdout:4189 write 0x22a9f4e thru 0x22adfb8 (0x406b bytes) 2024-09-14T06:17:22.252 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:22.252 INFO:teuthology.orchestra.run.smithi039.stdout:4192 write 0x685a8b thru 0x694146 (0xe6bc bytes) 2024-09-14T06:17:22.270 INFO:teuthology.orchestra.run.smithi039.stdout:4193 trunc from 0x22adfb9 to 0xb75cf46 2024-09-14T06:17:22.282 INFO:teuthology.orchestra.run.smithi039.stdout:4194 punch from 0x90f345b to 0x90f90f4, (0x5c99 bytes) 2024-09-14T06:17:22.282 INFO:teuthology.orchestra.run.smithi039.stdout:4195 writesame 0x120f8f thru 0x130d0a data_size 0x2(0xfd7c bytes) 2024-09-14T06:17:22.298 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:22.298 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:22.303 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:22.301+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:22.311 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:17:22.313 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:17:22.326 INFO:teuthology.orchestra.run.smithi039.stdout:4197 read 0x7ef5e9b thru 0x7f04094 (0xe1fa bytes) 2024-09-14T06:17:22.328 INFO:teuthology.orchestra.run.smithi039.stdout:4198 read 0x10295bf thru 0x10297c1 (0x203 bytes) 2024-09-14T06:17:22.328 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:22.338 INFO:teuthology.orchestra.run.smithi039.stdout:4200 write 0xbea1911 thru 0xbea5a07 (0x40f7 bytes) 2024-09-14T06:17:22.356 INFO:teuthology.orchestra.run.smithi039.stdout:4202 trunc from 0xbea5a08 to 0x51699b8 2024-09-14T06:17:22.374 INFO:teuthology.orchestra.run.smithi039.stdout:4203 read 0x5045d2c thru 0x5052570 (0xc845 bytes) 2024-09-14T06:17:22.378 INFO:teuthology.orchestra.run.smithi039.stdout:4206 trunc from 0x51699b8 to 0x695d20f 2024-09-14T06:17:22.385 INFO:teuthology.orchestra.run.smithi039.stdout:4207 read 0x690a3ee thru 0x6912760 (0x8373 bytes) 2024-09-14T06:17:22.385 INFO:teuthology.orchestra.run.smithi039.stdout:4208 write 0x64a9ad thru 0x655f59 (0xb5ad bytes) 2024-09-14T06:17:22.393 INFO:teuthology.orchestra.run.smithi039.stdout:4209 trunc from 0x695d20f to 0x9a392fe 2024-09-14T06:17:22.401 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:22.401 INFO:teuthology.orchestra.run.smithi039.stdout:4211 read 0x1d1f010 thru 0x1d27489 (0x847a bytes) 2024-09-14T06:17:22.404 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:22.401+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:22.417 INFO:teuthology.orchestra.run.smithi039.stdout:4212 write 0xd2c3efa thru 0xd2c6af1 (0x2bf8 bytes) 2024-09-14T06:17:22.422 INFO:teuthology.orchestra.run.smithi039.stdout:4214 write 0x5a00aeb thru 0x5a1057a (0xfa90 bytes) 2024-09-14T06:17:22.428 INFO:teuthology.orchestra.run.smithi039.stdout:4215 trunc from 0xd2c6af2 to 0xaa8d397 2024-09-14T06:17:22.442 INFO:teuthology.orchestra.run.smithi039.stdout:4217 write 0x50978f4 thru 0x5099798 (0x1ea5 bytes) 2024-09-14T06:17:22.446 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:22.446 INFO:teuthology.orchestra.run.smithi039.stdout:4219 writesame 0x9ce0a0b thru 0x9ce4651 data_size 0x1(0x3c47 bytes) 2024-09-14T06:17:22.460 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:22.461 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:22.465 INFO:teuthology.orchestra.run.smithi039.stdout:4223 read 0x22dd074 thru 0x22e8726 (0xb6b3 bytes) 2024-09-14T06:17:22.466 INFO:teuthology.orchestra.run.smithi039.stdout:4226 writesame 0x9593ef7 thru 0x959f5e8 data_size 0x1(0xb6f2 bytes) 2024-09-14T06:17:22.470 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:17:22.480 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:17:22.495 INFO:teuthology.orchestra.run.smithi039.stdout:4227 write 0x34841ab thru 0x3491ca8 (0xdafe bytes) 2024-09-14T06:17:22.497 INFO:teuthology.orchestra.run.smithi039.stdout:4228 writesame 0x82c9598 thru 0x82cd9c3 data_size 0x2(0x442c bytes) 2024-09-14T06:17:22.499 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:17:22.499 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:17:22.499 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-14T06:17:22.500 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-14T06:17:22.509 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:22.510 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:22.510 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:22.506+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:22.510 INFO:teuthology.orchestra.run.smithi039.stdout:4230 writesame 0x86126e3 thru 0x86220cc data_size 0x3(0xf9ea bytes) 2024-09-14T06:17:22.528 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:22.528 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:22.538 INFO:teuthology.orchestra.run.smithi039.stdout:4231 write 0x6b1b584 thru 0x6b29db1 (0xe82e bytes) 2024-09-14T06:17:22.545 INFO:teuthology.orchestra.run.smithi039.stdout:4233 write 0x513bccb thru 0x514862b (0xc961 bytes) 2024-09-14T06:17:22.548 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:22.548 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:22.548 INFO:teuthology.orchestra.run.smithi039.stdout:4235 write 0x90ff8cc thru 0x910ee76 (0xf5ab bytes) 2024-09-14T06:17:22.551 INFO:teuthology.orchestra.run.smithi039.stdout:4236 read 0x159abe4 thru 0x15a4867 (0x9c84 bytes) 2024-09-14T06:17:22.567 INFO:teuthology.orchestra.run.smithi039.stdout:4237 write 0xdf68034 thru 0xdf6a1d6 (0x21a3 bytes) 2024-09-14T06:17:22.570 INFO:teuthology.orchestra.run.smithi039.stdout:4238 punch from 0x198c932 to 0x1997c8b, (0xb359 bytes) 2024-09-14T06:17:22.570 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:22.570 INFO:teuthology.orchestra.run.smithi039.stdout:4240 read 0x892274d thru 0x8926a74 (0x4328 bytes) 2024-09-14T06:17:22.571 INFO:teuthology.orchestra.run.smithi039.stdout:4241 read 0x2a11032 thru 0x2a1a3cf (0x939e bytes) 2024-09-14T06:17:22.571 INFO:teuthology.orchestra.run.smithi039.stdout:4242 read 0x16a6024 thru 0x16aa4c8 (0x44a5 bytes) 2024-09-14T06:17:22.572 INFO:teuthology.orchestra.run.smithi039.stdout:4243 write 0x2ef5543 thru 0x2efdb1a (0x85d8 bytes) 2024-09-14T06:17:22.576 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:22.584 INFO:teuthology.orchestra.run.smithi039.stdout:4245 write 0xe4e7805 thru 0xe4ea768 (0x2f64 bytes) 2024-09-14T06:17:22.590 INFO:teuthology.orchestra.run.smithi039.stdout:4246 punch from 0x768eb06 to 0x769715f, (0x8659 bytes) 2024-09-14T06:17:22.590 INFO:teuthology.orchestra.run.smithi039.stdout:4247 read 0x813dd33 thru 0x81433d2 (0x56a0 bytes) 2024-09-14T06:17:22.595 INFO:teuthology.orchestra.run.smithi039.stdout:4248 trunc from 0xe4ea769 to 0xb87ce07 2024-09-14T06:17:22.601 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:22.601 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:22.605 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:22.601+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:22.607 INFO:teuthology.orchestra.run.smithi039.stdout:4249 punch from 0x56c6f27 to 0x56d419d, (0xd276 bytes) 2024-09-14T06:17:22.607 INFO:teuthology.orchestra.run.smithi039.stdout:4252 read 0xb827ea3 thru 0xb82ae81 (0x2fdf bytes) 2024-09-14T06:17:22.607 INFO:teuthology.orchestra.run.smithi039.stdout:4253 read 0x8b862c3 thru 0x8b8a2d2 (0x4010 bytes) 2024-09-14T06:17:22.608 INFO:teuthology.orchestra.run.smithi039.stdout:4254 trunc from 0xb87ce07 to 0xbc52194 2024-09-14T06:17:22.616 INFO:teuthology.orchestra.run.smithi039.stdout:4256 write 0x56c96ac thru 0x56d0b37 (0x748c bytes) 2024-09-14T06:17:22.619 INFO:teuthology.orchestra.run.smithi039.stdout:4257 write 0x6ed9e1b thru 0x6ee01a8 (0x638e bytes) 2024-09-14T06:17:22.621 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:22.622 INFO:teuthology.orchestra.run.smithi039.stdout:4260 read 0x71a71a thru 0x71e2f9 (0x3be0 bytes) 2024-09-14T06:17:22.629 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:22.629 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:22.635 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:22.635 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:22.635 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:22.636 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:22.636 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.636 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11263 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 204800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:22.636 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:22.636 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.642145163999999, 2024-09-14T06:17:22.636 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049944574999999998, 2024-09-14T06:17:22.636 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.636 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.636 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:22.636 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.636 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.636 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.636 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11263 2024-09-14T06:17:22.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.637 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:22.637 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:22.637 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.638 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.638 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.638 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991553+0000", 2024-09-14T06:17:22.638 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.51e-07 2024-09-14T06:17:22.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.638 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.638 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.639 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991560+0000", 2024-09-14T06:17:22.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6719999999999998e-06 2024-09-14T06:17:22.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.639 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.639 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.639 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991567+0000", 2024-09-14T06:17:22.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1470000000000004e-06 2024-09-14T06:17:22.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.639 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.639 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.639 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991572+0000", 2024-09-14T06:17:22.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.237e-06 2024-09-14T06:17:22.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.639 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.640 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.640 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041149+0000", 2024-09-14T06:17:22.640 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049576988000000002 2024-09-14T06:17:22.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.640 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.640 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041277+0000", 2024-09-14T06:17:22.640 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000127687 2024-09-14T06:17:22.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.640 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.640 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041497+0000", 2024-09-14T06:17:22.640 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022039299999999999 2024-09-14T06:17:22.641 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.641 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.641 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.641 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11264 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 327680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:22.641 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:22.641 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.642118822999997, 2024-09-14T06:17:22.641 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050248567000000001, 2024-09-14T06:17:22.641 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.641 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.641 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:22.641 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.641 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.642 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.642 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11264 2024-09-14T06:17:22.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.642 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.642 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.642 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.642 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:22.642 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.642 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.642 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.642 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:22.642 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.643 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.643 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991579+0000", 2024-09-14T06:17:22.643 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.41e-07 2024-09-14T06:17:22.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.643 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.643 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991581+0000", 2024-09-14T06:17:22.643 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3740000000000001e-06 2024-09-14T06:17:22.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.643 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.643 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991585+0000", 2024-09-14T06:17:22.643 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9400000000000004e-06 2024-09-14T06:17:22.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.644 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.644 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.644 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991587+0000", 2024-09-14T06:17:22.644 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.694e-06 2024-09-14T06:17:22.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.644 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.644 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.644 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041527+0000", 2024-09-14T06:17:22.644 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049939349000000001 2024-09-14T06:17:22.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.644 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.645 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.645 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041598+0000", 2024-09-14T06:17:22.645 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1633999999999999e-05 2024-09-14T06:17:22.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.645 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.645 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.645 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041827+0000", 2024-09-14T06:17:22.645 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000229235 2024-09-14T06:17:22.645 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.645 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.645 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.645 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.645 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:22.645 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:22.646 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.642105483000002, 2024-09-14T06:17:22.646 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:17:22.646 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.646 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.646 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:22.646 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.646 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.646 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.646 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:17:22.646 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.646 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.646 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.646 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.647 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:22.647 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.647 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.647 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.647 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:22.647 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.647 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.647 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.647 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:22.647 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:17:22.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.647 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.647 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.648 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:17:22.648 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:17:22.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.648 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.648 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.648 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:17:22.648 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:17:22.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.648 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.648 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.648 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:17:22.648 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:17:22.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.648 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.649 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.649 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:17:22.649 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:17:22.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.649 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.649 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.649 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:17:22.649 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:17:22.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.649 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.649 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.649 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:17:22.649 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:17:22.649 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.649 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.650 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.650 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.650 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:22.650 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:22.650 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.642092411, 2024-09-14T06:17:22.650 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:17:22.650 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.650 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.650 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:22.650 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.650 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.650 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.650 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:17:22.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.651 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:22.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:22.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:17:22.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:17:22.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.652 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.652 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:17:22.652 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:17:22.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.652 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.652 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:17:22.652 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:17:22.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.652 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.652 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:17:22.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:17:22.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.653 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.653 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.653 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:17:22.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:17:22.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.653 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.653 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.653 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:17:22.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:17:22.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.653 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.653 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.653 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:17:22.654 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:17:22.654 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.654 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.654 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.654 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.654 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:22.654 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:22.654 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.642074979, 2024-09-14T06:17:22.654 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:17:22.654 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.654 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.654 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:22.654 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.654 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.655 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.655 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:17:22.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.655 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.655 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:22.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.655 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:22.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.656 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.656 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:22.656 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:17:22.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.656 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.656 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:17:22.656 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:17:22.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.657 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.657 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:17:22.657 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:17:22.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.657 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.657 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:17:22.657 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:17:22.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.657 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.657 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:17:22.658 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:17:22.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.658 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.658 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.658 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:17:22.658 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:17:22.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.658 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.658 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.658 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:17:22.658 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:17:22.659 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.659 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.659 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.659 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.659 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.659 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:22.659 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:22.659 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.642053873999998, 2024-09-14T06:17:22.659 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:17:22.659 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.659 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.659 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:22.659 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.659 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.660 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.660 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:17:22.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.660 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.660 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.660 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.660 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:22.660 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.660 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.660 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.660 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:22.660 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.661 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.661 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.661 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:22.661 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:17:22.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.661 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.661 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.661 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:17:22.661 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:17:22.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.661 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.661 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.661 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:17:22.661 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:17:22.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.662 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.662 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.662 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:17:22.662 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:17:22.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.662 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.663 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.663 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:17:22.663 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:17:22.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.664 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.664 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.664 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:17:22.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:17:22.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.664 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.664 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.664 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:17:22.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:17:22.664 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.664 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.664 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.665 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:22.665 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:22.665 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.642032766, 2024-09-14T06:17:22.665 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:17:22.665 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.665 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.665 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:22.665 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.665 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.665 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.665 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:17:22.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.666 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.666 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.666 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:22.666 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.666 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.666 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:22.666 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.666 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.666 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:22.667 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:17:22.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.667 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.667 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.667 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:17:22.667 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:17:22.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.667 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.667 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.667 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:17:22.667 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:17:22.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.667 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.668 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.668 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:17:22.668 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:17:22.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.668 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.668 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.668 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:17:22.668 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:17:22.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.668 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.668 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.668 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:17:22.668 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:17:22.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.669 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.669 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.669 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:17:22.669 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:17:22.669 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.669 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.669 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.669 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.669 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48234 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:22.669 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:22.669 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.600761474, 2024-09-14T06:17:22.669 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051730353999999999, 2024-09-14T06:17:22.670 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.670 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.670 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:22.670 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.670 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.670 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.670 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48234 2024-09-14T06:17:22.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.670 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.670 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.670 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.670 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:22.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:22.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032937+0000", 2024-09-14T06:17:22.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:17:22.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032940+0000", 2024-09-14T06:17:22.672 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0580000000000002e-06 2024-09-14T06:17:22.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.672 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.672 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032946+0000", 2024-09-14T06:17:22.672 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2960000000000004e-06 2024-09-14T06:17:22.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.672 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.672 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032951+0000", 2024-09-14T06:17:22.672 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.938e-06 2024-09-14T06:17:22.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074419+0000", 2024-09-14T06:17:22.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041467979000000002 2024-09-14T06:17:22.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074490+0000", 2024-09-14T06:17:22.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1324999999999999e-05 2024-09-14T06:17:22.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:22.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074859+0000", 2024-09-14T06:17:22.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036914899999999999 2024-09-14T06:17:22.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.674 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:22.674 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078464+0000", 2024-09-14T06:17:22.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036043170000000001 2024-09-14T06:17:22.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.674 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:22.674 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084547+0000", 2024-09-14T06:17:22.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006083097 2024-09-14T06:17:22.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.674 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:22.674 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084614+0000", 2024-09-14T06:17:22.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7509999999999996e-05 2024-09-14T06:17:22.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.675 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.675 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.675 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084667+0000", 2024-09-14T06:17:22.675 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:17:22.675 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.675 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.675 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.675 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.675 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48237 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:22.675 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:22.675 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.60068173, 2024-09-14T06:17:22.675 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051772830999999998, 2024-09-14T06:17:22.676 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.676 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.676 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:22.676 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.676 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.676 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.676 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48237 2024-09-14T06:17:22.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.676 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.676 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.676 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.676 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:22.676 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.677 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.677 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:22.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.677 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.677 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:22.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0499999999999999e-07 2024-09-14T06:17:22.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.677 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033018+0000", 2024-09-14T06:17:22.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0949999999999998e-06 2024-09-14T06:17:22.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.678 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033023+0000", 2024-09-14T06:17:22.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1620000000000001e-06 2024-09-14T06:17:22.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.678 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033026+0000", 2024-09-14T06:17:22.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0120000000000001e-06 2024-09-14T06:17:22.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075349+0000", 2024-09-14T06:17:22.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042323373999999997 2024-09-14T06:17:22.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075394+0000", 2024-09-14T06:17:22.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4861000000000003e-05 2024-09-14T06:17:22.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:22.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075578+0000", 2024-09-14T06:17:22.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00018414899999999999 2024-09-14T06:17:22.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:22.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078894+0000", 2024-09-14T06:17:22.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033158480000000001 2024-09-14T06:17:22.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:22.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084677+0000", 2024-09-14T06:17:22.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0057832120000000002 2024-09-14T06:17:22.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:22.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084752+0000", 2024-09-14T06:17:22.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5123000000000006e-05 2024-09-14T06:17:22.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.681 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.681 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084789+0000", 2024-09-14T06:17:22.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6690000000000003e-05 2024-09-14T06:17:22.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.681 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.682 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48243 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:22.682 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:22.682 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.600600605, 2024-09-14T06:17:22.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051780221000000001, 2024-09-14T06:17:22.682 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.682 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.682 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:22.682 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.682 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.682 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.682 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48243 2024-09-14T06:17:22.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.682 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:22.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:22.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:22.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:17:22.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033100+0000", 2024-09-14T06:17:22.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:17:22.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033108+0000", 2024-09-14T06:17:22.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1760000000000004e-06 2024-09-14T06:17:22.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033111+0000", 2024-09-14T06:17:22.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0790000000000001e-06 2024-09-14T06:17:22.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075903+0000", 2024-09-14T06:17:22.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042791942999999999 2024-09-14T06:17:22.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075987+0000", 2024-09-14T06:17:22.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4467000000000003e-05 2024-09-14T06:17:22.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:22.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.076259+0000", 2024-09-14T06:17:22.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027201900000000003 2024-09-14T06:17:22.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:22.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078947+0000", 2024-09-14T06:17:22.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026875879999999999 2024-09-14T06:17:22.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:22.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084797+0000", 2024-09-14T06:17:22.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0058500380000000001 2024-09-14T06:17:22.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:22.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084843+0000", 2024-09-14T06:17:22.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6298999999999998e-05 2024-09-14T06:17:22.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084877+0000", 2024-09-14T06:17:22.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4378999999999997e-05 2024-09-14T06:17:22.687 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.687 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.687 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.687 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48287 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:22.687 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:22.687 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.600560805000001, 2024-09-14T06:17:22.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051924706000000001, 2024-09-14T06:17:22.688 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.688 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.688 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:22.688 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.688 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.688 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.688 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48287 2024-09-14T06:17:22.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.688 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:22.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:22.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033138+0000", 2024-09-14T06:17:22.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6599999999999996e-07 2024-09-14T06:17:22.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033141+0000", 2024-09-14T06:17:22.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0879999999999999e-06 2024-09-14T06:17:22.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033147+0000", 2024-09-14T06:17:22.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2249999999999997e-06 2024-09-14T06:17:22.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033153+0000", 2024-09-14T06:17:22.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.066e-06 2024-09-14T06:17:22.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077790+0000", 2024-09-14T06:17:22.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.044637255000000001 2024-09-14T06:17:22.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077845+0000", 2024-09-14T06:17:22.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4410999999999998e-05 2024-09-14T06:17:22.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:22.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078062+0000", 2024-09-14T06:17:22.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021755599999999999 2024-09-14T06:17:22.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:22.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.080991+0000", 2024-09-14T06:17:22.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029282969999999998 2024-09-14T06:17:22.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:22.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084885+0000", 2024-09-14T06:17:22.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003894495 2024-09-14T06:17:22.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:22.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085020+0000", 2024-09-14T06:17:22.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013500799999999999 2024-09-14T06:17:22.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085062+0000", 2024-09-14T06:17:22.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1638999999999999e-05 2024-09-14T06:17:22.693 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.693 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.693 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.693 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48288 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:22.693 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:22.693 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.600537106000001, 2024-09-14T06:17:22.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051994018000000003, 2024-09-14T06:17:22.694 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.694 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.694 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:22.694 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.694 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.694 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.694 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48288 2024-09-14T06:17:22.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.694 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.694 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:22.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.695 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:22.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.695 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:22.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.89e-07 2024-09-14T06:17:22.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.695 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033164+0000", 2024-09-14T06:17:22.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.779e-06 2024-09-14T06:17:22.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033171+0000", 2024-09-14T06:17:22.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6279999999999999e-06 2024-09-14T06:17:22.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033176+0000", 2024-09-14T06:17:22.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1610000000000002e-06 2024-09-14T06:17:22.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078991+0000", 2024-09-14T06:17:22.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045815348999999998 2024-09-14T06:17:22.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079086+0000", 2024-09-14T06:17:22.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5282e-05 2024-09-14T06:17:22.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:22.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079397+0000", 2024-09-14T06:17:22.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031055999999999998 2024-09-14T06:17:22.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:22.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084432+0000", 2024-09-14T06:17:22.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050355599999999997 2024-09-14T06:17:22.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:22.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085073+0000", 2024-09-14T06:17:22.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064015199999999995 2024-09-14T06:17:22.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:22.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085120+0000", 2024-09-14T06:17:22.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7107000000000001e-05 2024-09-14T06:17:22.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085155+0000", 2024-09-14T06:17:22.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4950999999999998e-05 2024-09-14T06:17:22.699 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.699 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.699 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.700 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48416 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:22.700 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:22.700 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.598296694, 2024-09-14T06:17:22.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053690144000000002, 2024-09-14T06:17:22.700 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.700 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.700 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:22.700 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.700 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.700 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.700 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48416 2024-09-14T06:17:22.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.700 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:22.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:22.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035402+0000", 2024-09-14T06:17:22.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7999999999999995e-07 2024-09-14T06:17:22.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035405+0000", 2024-09-14T06:17:22.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3960000000000002e-06 2024-09-14T06:17:22.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035413+0000", 2024-09-14T06:17:22.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2209999999999997e-06 2024-09-14T06:17:22.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035420+0000", 2024-09-14T06:17:22.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1539999999999996e-06 2024-09-14T06:17:22.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081021+0000", 2024-09-14T06:17:22.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045600389999999998 2024-09-14T06:17:22.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081091+0000", 2024-09-14T06:17:22.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0325000000000002e-05 2024-09-14T06:17:22.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:22.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081434+0000", 2024-09-14T06:17:22.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000342519 2024-09-14T06:17:22.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:22.704 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085254+0000", 2024-09-14T06:17:22.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038201609999999999 2024-09-14T06:17:22.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:22.704 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088977+0000", 2024-09-14T06:17:22.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003722805 2024-09-14T06:17:22.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:22.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089048+0000", 2024-09-14T06:17:22.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1266999999999994e-05 2024-09-14T06:17:22.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089091+0000", 2024-09-14T06:17:22.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3325999999999998e-05 2024-09-14T06:17:22.705 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.705 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.705 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.705 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48417 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:22.706 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:22.706 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.598268024999999, 2024-09-14T06:17:22.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053805032000000003, 2024-09-14T06:17:22.706 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.706 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.706 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:22.706 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.706 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.706 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.706 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48417 2024-09-14T06:17:22.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.706 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:22.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:22.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:22.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9500000000000003e-07 2024-09-14T06:17:22.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035433+0000", 2024-09-14T06:17:22.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.796e-06 2024-09-14T06:17:22.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035442+0000", 2024-09-14T06:17:22.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7120000000000006e-06 2024-09-14T06:17:22.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035448+0000", 2024-09-14T06:17:22.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.015e-06 2024-09-14T06:17:22.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081855+0000", 2024-09-14T06:17:22.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.046407634000000003 2024-09-14T06:17:22.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081938+0000", 2024-09-14T06:17:22.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2687000000000003e-05 2024-09-14T06:17:22.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:22.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082313+0000", 2024-09-14T06:17:22.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000374872 2024-09-14T06:17:22.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:22.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087616+0000", 2024-09-14T06:17:22.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053032670000000004 2024-09-14T06:17:22.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:22.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089140+0000", 2024-09-14T06:17:22.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0015234350000000001 2024-09-14T06:17:22.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:22.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089184+0000", 2024-09-14T06:17:22.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4804e-05 2024-09-14T06:17:22.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089235+0000", 2024-09-14T06:17:22.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0315000000000002e-05 2024-09-14T06:17:22.711 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.711 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.711 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.711 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48418 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:22.711 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:22.712 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.598223886, 2024-09-14T06:17:22.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053936425000000003, 2024-09-14T06:17:22.712 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.712 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.712 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:22.712 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.712 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.712 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.712 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48418 2024-09-14T06:17:22.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.712 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:22.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:22.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035475+0000", 2024-09-14T06:17:22.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7000000000000003e-07 2024-09-14T06:17:22.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035479+0000", 2024-09-14T06:17:22.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2350000000000001e-06 2024-09-14T06:17:22.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035488+0000", 2024-09-14T06:17:22.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8820000000000007e-06 2024-09-14T06:17:22.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035494+0000", 2024-09-14T06:17:22.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3060000000000003e-06 2024-09-14T06:17:22.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082826+0000", 2024-09-14T06:17:22.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.047332008000000002 2024-09-14T06:17:22.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082883+0000", 2024-09-14T06:17:22.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7124999999999999e-05 2024-09-14T06:17:22.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:22.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.083385+0000", 2024-09-14T06:17:22.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050134099999999998 2024-09-14T06:17:22.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:22.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088888+0000", 2024-09-14T06:17:22.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005503127 2024-09-14T06:17:22.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:22.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089294+0000", 2024-09-14T06:17:22.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040669700000000003 2024-09-14T06:17:22.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:22.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089348+0000", 2024-09-14T06:17:22.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3729e-05 2024-09-14T06:17:22.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089410+0000", 2024-09-14T06:17:22.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2105000000000003e-05 2024-09-14T06:17:22.717 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.717 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.717 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.717 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48419 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:22.718 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:22.718 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.598146256, 2024-09-14T06:17:22.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054007545999999997, 2024-09-14T06:17:22.718 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.718 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.718 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:22.718 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.718 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.718 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.718 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48419 2024-09-14T06:17:22.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.718 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:22.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:22.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035552+0000", 2024-09-14T06:17:22.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4099999999999998e-07 2024-09-14T06:17:22.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035557+0000", 2024-09-14T06:17:22.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1819999999999997e-06 2024-09-14T06:17:22.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035566+0000", 2024-09-14T06:17:22.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1120000000000002e-06 2024-09-14T06:17:22.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035572+0000", 2024-09-14T06:17:22.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0430000000000004e-06 2024-09-14T06:17:22.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085304+0000", 2024-09-14T06:17:22.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049731707 2024-09-14T06:17:22.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085371+0000", 2024-09-14T06:17:22.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6771000000000002e-05 2024-09-14T06:17:22.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:22.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085601+0000", 2024-09-14T06:17:22.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022968000000000001 2024-09-14T06:17:22.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:22.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088893+0000", 2024-09-14T06:17:22.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032927780000000001 2024-09-14T06:17:22.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:22.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089467+0000", 2024-09-14T06:17:22.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057330399999999996 2024-09-14T06:17:22.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:22.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089520+0000", 2024-09-14T06:17:22.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3133000000000001e-05 2024-09-14T06:17:22.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089559+0000", 2024-09-14T06:17:22.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9195e-05 2024-09-14T06:17:22.723 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.723 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.723 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.723 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48420 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:22.723 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:22.723 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.598057025999999, 2024-09-14T06:17:22.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056401932000000002, 2024-09-14T06:17:22.724 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.724 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.724 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:22.724 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.724 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.724 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.724 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48420 2024-09-14T06:17:22.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.724 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:22.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:22.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035642+0000", 2024-09-14T06:17:22.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1519999999999999e-06 2024-09-14T06:17:22.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035647+0000", 2024-09-14T06:17:22.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2299999999999999e-06 2024-09-14T06:17:22.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035660+0000", 2024-09-14T06:17:22.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3166999999999999e-05 2024-09-14T06:17:22.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035666+0000", 2024-09-14T06:17:22.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1720000000000004e-06 2024-09-14T06:17:22.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086104+0000", 2024-09-14T06:17:22.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050437956999999999 2024-09-14T06:17:22.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086164+0000", 2024-09-14T06:17:22.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9592999999999997e-05 2024-09-14T06:17:22.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:22.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086402+0000", 2024-09-14T06:17:22.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000238314 2024-09-14T06:17:22.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:22.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089103+0000", 2024-09-14T06:17:22.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027011600000000002 2024-09-14T06:17:22.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:22.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091962+0000", 2024-09-14T06:17:22.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028581259999999999 2024-09-14T06:17:22.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:22.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092005+0000", 2024-09-14T06:17:22.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3111000000000001e-05 2024-09-14T06:17:22.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092043+0000", 2024-09-14T06:17:22.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7950000000000001e-05 2024-09-14T06:17:22.729 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.729 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.729 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.730 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48424 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:22.730 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:22.730 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.597793081000001, 2024-09-14T06:17:22.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056273173000000003, 2024-09-14T06:17:22.730 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.730 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.730 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:22.730 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.730 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.730 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.730 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48424 2024-09-14T06:17:22.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.730 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:22.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:22.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:22.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2900000000000003e-07 2024-09-14T06:17:22.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035909+0000", 2024-09-14T06:17:22.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6490000000000001e-06 2024-09-14T06:17:22.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035918+0000", 2024-09-14T06:17:22.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5089999999999999e-06 2024-09-14T06:17:22.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035924+0000", 2024-09-14T06:17:22.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7089999999999999e-06 2024-09-14T06:17:22.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087662+0000", 2024-09-14T06:17:22.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737621999999997 2024-09-14T06:17:22.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087744+0000", 2024-09-14T06:17:22.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2174000000000003e-05 2024-09-14T06:17:22.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:22.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088106+0000", 2024-09-14T06:17:22.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000361947 2024-09-14T06:17:22.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:22.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092053+0000", 2024-09-14T06:17:22.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0039473080000000001 2024-09-14T06:17:22.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:22.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092087+0000", 2024-09-14T06:17:22.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4153000000000003e-05 2024-09-14T06:17:22.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:22.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092138+0000", 2024-09-14T06:17:22.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0804999999999999e-05 2024-09-14T06:17:22.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092178+0000", 2024-09-14T06:17:22.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9668e-05 2024-09-14T06:17:22.735 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.735 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.735 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.736 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48508 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:22.736 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:22.736 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.592718, 2024-09-14T06:17:22.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054906017000000001, 2024-09-14T06:17:22.736 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.736 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.736 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:22.736 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.736 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.736 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.736 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48508 2024-09-14T06:17:22.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.737 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:22.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:22.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040981+0000", 2024-09-14T06:17:22.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7700000000000004e-07 2024-09-14T06:17:22.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040984+0000", 2024-09-14T06:17:22.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1480000000000002e-06 2024-09-14T06:17:22.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040991+0000", 2024-09-14T06:17:22.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4580000000000004e-06 2024-09-14T06:17:22.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040997+0000", 2024-09-14T06:17:22.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7799999999999997e-06 2024-09-14T06:17:22.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089601+0000", 2024-09-14T06:17:22.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048604264000000001 2024-09-14T06:17:22.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089708+0000", 2024-09-14T06:17:22.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010681200000000001 2024-09-14T06:17:22.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:22.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090384+0000", 2024-09-14T06:17:22.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00067557499999999998 2024-09-14T06:17:22.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:22.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093879+0000", 2024-09-14T06:17:22.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034954679999999998 2024-09-14T06:17:22.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:22.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095764+0000", 2024-09-14T06:17:22.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018848070000000001 2024-09-14T06:17:22.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:22.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095837+0000", 2024-09-14T06:17:22.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3433e-05 2024-09-14T06:17:22.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095886+0000", 2024-09-14T06:17:22.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8495000000000002e-05 2024-09-14T06:17:22.741 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.741 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.741 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.741 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48537 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:22.741 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:22.742 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.59135423, 2024-09-14T06:17:22.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053681833999999998, 2024-09-14T06:17:22.742 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:22.742 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:22.742 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:22.742 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:22.742 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:22.742 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:22.742 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48537 2024-09-14T06:17:22.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.742 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:22.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:22.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:22.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:22.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:22.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:22.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:22.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:22.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2299999999999999e-07 2024-09-14T06:17:22.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:22.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042350+0000", 2024-09-14T06:17:22.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5690000000000004e-06 2024-09-14T06:17:22.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:22.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042361+0000", 2024-09-14T06:17:22.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1337e-05 2024-09-14T06:17:22.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:22.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042369+0000", 2024-09-14T06:17:22.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8359999999999992e-06 2024-09-14T06:17:22.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:22.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090914+0000", 2024-09-14T06:17:22.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048544945999999999 2024-09-14T06:17:22.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:22.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090995+0000", 2024-09-14T06:17:22.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0548000000000002e-05 2024-09-14T06:17:22.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:22.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091508+0000", 2024-09-14T06:17:22.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000513723 2024-09-14T06:17:22.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:22.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093900+0000", 2024-09-14T06:17:22.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023920429999999999 2024-09-14T06:17:22.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:22.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095944+0000", 2024-09-14T06:17:22.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020437379999999998 2024-09-14T06:17:22.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:22.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095987+0000", 2024-09-14T06:17:22.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2521999999999997e-05 2024-09-14T06:17:22.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:22.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:22.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:22.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.096025+0000", 2024-09-14T06:17:22.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8649000000000002e-05 2024-09-14T06:17:22.747 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.747 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.747 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.747 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:22.747 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:22.747 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:22.748 INFO:teuthology.orchestra.run.smithi039.stdout:4261 write 0xeb8e0da thru 0xeb933f0 (0x5317 bytes) 2024-09-14T06:17:22.748 INFO:teuthology.orchestra.run.smithi039.stdout:4262 write 0xa948dbe thru 0xa94f110 (0x6353 bytes) 2024-09-14T06:17:22.749 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:22.749 INFO:teuthology.orchestra.run.smithi039.stdout:4264 writesame 0x984d26e thru 0x985bf26 data_size 0x1(0xecb9 bytes) 2024-09-14T06:17:22.750 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:17:22.750 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:22.750 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:22.751 INFO:teuthology.orchestra.run.smithi039.stdout:4265 write 0xe4add7a thru 0xe4b510e (0x7395 bytes) 2024-09-14T06:17:22.751 INFO:teuthology.orchestra.run.smithi039.stdout:4266 punch from 0x3237f60 to 0x324133b, (0x93db bytes) 2024-09-14T06:17:22.751 INFO:teuthology.orchestra.run.smithi039.stdout:4267 write 0x857a206 thru 0x8587e1c (0xdc17 bytes) 2024-09-14T06:17:22.751 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:22.751 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:22.752 INFO:teuthology.orchestra.run.smithi039.stdout:4269 write 0x650a816 thru 0x6517fab (0xd796 bytes) 2024-09-14T06:17:22.752 INFO:teuthology.orchestra.run.smithi039.stdout:4272 trunc from 0xeb933f1 to 0xec4589 2024-09-14T06:17:22.753 INFO:teuthology.orchestra.run.smithi039.stdout:4275 punch from 0xc7c330 to 0xc801a8, (0x3e78 bytes) 2024-09-14T06:17:22.753 INFO:teuthology.orchestra.run.smithi039.stdout:4276 write 0x5253a3d thru 0x525b694 (0x7c58 bytes) 2024-09-14T06:17:22.753 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:22.745+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:22.754 INFO:teuthology.orchestra.run.smithi039.stdout:4277 writesame 0x4925edc thru 0x49359e5 data_size 0x3(0xfb0a bytes) 2024-09-14T06:17:22.785 INFO:teuthology.orchestra.run.smithi039.stdout:4278 punch from 0x45c6461 to 0x45d073a, (0xa2d9 bytes) 2024-09-14T06:17:22.785 INFO:teuthology.orchestra.run.smithi039.stdout:4279 read 0x3f5e05c thru 0x3f6dc18 (0xfbbd bytes) 2024-09-14T06:17:22.786 INFO:teuthology.orchestra.run.smithi039.stdout:4280 writesame 0x18caa8b thru 0x18d8412 data_size 0x4(0xd988 bytes) 2024-09-14T06:17:22.816 INFO:teuthology.orchestra.run.smithi039.stdout:4281 write 0xed411b1 thru 0xed41ad2 (0x922 bytes) 2024-09-14T06:17:22.821 INFO:teuthology.orchestra.run.smithi039.stdout:4282 read 0x235c8bc thru 0x235cf10 (0x655 bytes) 2024-09-14T06:17:22.821 INFO:teuthology.orchestra.run.smithi039.stdout:4284 read 0x8362a36 thru 0x836c83b (0x9e06 bytes) 2024-09-14T06:17:22.829 INFO:teuthology.orchestra.run.smithi039.stdout:4286 trunc from 0xed41ad3 to 0xab42f49 2024-09-14T06:17:22.840 INFO:teuthology.orchestra.run.smithi039.stdout:4287 read 0x21bcf2a thru 0x21cc25d (0xf334 bytes) 2024-09-14T06:17:22.841 INFO:teuthology.orchestra.run.smithi039.stdout:4288 write 0xa0c5579 thru 0xa0cd2a7 (0x7d2f bytes) 2024-09-14T06:17:22.850 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:22.845+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:22.861 INFO:teuthology.orchestra.run.smithi039.stdout:4289 writesame 0xe3921ca thru 0xe39c31c data_size 0x1(0xa153 bytes) 2024-09-14T06:17:22.862 INFO:teuthology.orchestra.run.smithi039.stdout:4291 read 0x2ad48d5 thru 0x2ae2a8a (0xe1b6 bytes) 2024-09-14T06:17:22.862 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:22.862 INFO:teuthology.orchestra.run.smithi039.stdout:4293 read 0xb0166be thru 0xb025f7a (0xf8bd bytes) 2024-09-14T06:17:22.862 INFO:teuthology.orchestra.run.smithi039.stdout:4294 writesame 0xb1a5870 thru 0xb1a988b data_size 0x4(0x401c bytes) 2024-09-14T06:17:22.869 INFO:teuthology.orchestra.run.smithi039.stdout:4295 writesame 0x72ca314 thru 0x72d7eb1 data_size 0x1(0xdb9e bytes) 2024-09-14T06:17:22.884 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:22.884 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:22.884 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:22.884 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:22.897 INFO:teuthology.orchestra.run.smithi039.stdout:4296 read 0x457b155 thru 0x457e419 (0x32c5 bytes) 2024-09-14T06:17:22.898 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:17:22.899 INFO:teuthology.orchestra.run.smithi039.stdout:4297 read 0x4982a8f thru 0x49923a0 (0xf912 bytes) 2024-09-14T06:17:22.899 INFO:teuthology.orchestra.run.smithi039.stdout:4299 punch from 0xd20307b to 0xd2096de, (0x6663 bytes) 2024-09-14T06:17:22.899 INFO:teuthology.orchestra.run.smithi039.stdout:4301 read 0x6e305d5 thru 0x6e3af0f (0xa93b bytes) 2024-09-14T06:17:22.900 INFO:teuthology.orchestra.run.smithi039.stdout:4302 read 0x7e0d15a thru 0x7e1246e (0x5315 bytes) 2024-09-14T06:17:22.901 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:22.901 INFO:teuthology.orchestra.run.smithi039.stdout:4304 writesame 0x86daff9 thru 0x86e9f5d data_size 0x67(0xef65 bytes) 2024-09-14T06:17:22.903 INFO:teuthology.orchestra.run.smithi039.stdout:4305 write 0xac873d9 thru 0xac8d1fc (0x5e24 bytes) 2024-09-14T06:17:22.905 INFO:teuthology.orchestra.run.smithi039.stdout:4306 read 0x8cea91b thru 0x8cead92 (0x478 bytes) 2024-09-14T06:17:22.906 INFO:teuthology.orchestra.run.smithi039.stdout:4307 writesame 0x10ff7ff thru 0x11063f3 data_size 0x1(0x6bf5 bytes) 2024-09-14T06:17:22.935 INFO:teuthology.orchestra.run.smithi039.stdout:4309 trunc from 0xe39c31d to 0x5a5e540 2024-09-14T06:17:22.950 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:22.946+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:22.953 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:22.953 INFO:teuthology.orchestra.run.smithi039.stdout:4313 write 0x2df7fb2 thru 0x2e039ae (0xb9fd bytes) 2024-09-14T06:17:22.956 INFO:teuthology.orchestra.run.smithi039.stdout:4314 read 0x1eed0a9 thru 0x1eee77c (0x16d4 bytes) 2024-09-14T06:17:22.957 INFO:teuthology.orchestra.run.smithi039.stdout:4315 read 0x4c98dff thru 0x4ca174b (0x894d bytes) 2024-09-14T06:17:22.957 INFO:teuthology.orchestra.run.smithi039.stdout:4317 clone 27 order 19 su 262144 sc 12 2024-09-14T06:17:23.051 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:23.050+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:23.065 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:23.065 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:23.065 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:23.065 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:23.066 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:23.079 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:17:23.152 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:23.149+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:23.206 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:23.206 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:23.206 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:23.206 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:23.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.206 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13229 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.206 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:23.207 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.104721407, 2024-09-14T06:17:23.207 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081798868999999996, 2024-09-14T06:17:23.207 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.207 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.207 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.207 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.207 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.207 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.207 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13229 2024-09-14T06:17:23.207 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.207 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.207 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.207 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.208 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:23.208 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.208 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.208 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.208 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:23.208 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.208 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.208 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.208 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098710+0000", 2024-09-14T06:17:23.208 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8199999999999999e-07 2024-09-14T06:17:23.209 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.209 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.209 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.209 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098714+0000", 2024-09-14T06:17:23.209 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:17:23.209 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.209 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.209 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.209 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098727+0000", 2024-09-14T06:17:23.209 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2520999999999999e-05 2024-09-14T06:17:23.209 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.209 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.209 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.209 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098749+0000", 2024-09-14T06:17:23.210 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2773999999999999e-05 2024-09-14T06:17:23.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.210 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.210 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.210 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160234+0000", 2024-09-14T06:17:23.210 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.061484430999999999 2024-09-14T06:17:23.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.210 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.210 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.210 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160281+0000", 2024-09-14T06:17:23.210 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7437e-05 2024-09-14T06:17:23.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.210 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.211 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.211 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160569+0000", 2024-09-14T06:17:23.211 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028788700000000002 2024-09-14T06:17:23.211 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.211 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.211 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.211 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163121+0000", 2024-09-14T06:17:23.211 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025514090000000001 2024-09-14T06:17:23.211 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.211 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.211 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.211 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180410+0000", 2024-09-14T06:17:23.211 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.017289098999999999 2024-09-14T06:17:23.212 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.212 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.212 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.212 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180472+0000", 2024-09-14T06:17:23.212 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2547999999999998e-05 2024-09-14T06:17:23.212 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.212 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.212 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.212 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180508+0000", 2024-09-14T06:17:23.212 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5534000000000002e-05 2024-09-14T06:17:23.212 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.213 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.213 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.213 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.213 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.213 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13230 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.213 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:23.213 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.104640205000003, 2024-09-14T06:17:23.213 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081851558000000005, 2024-09-14T06:17:23.213 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.213 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.213 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.213 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.213 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.214 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.214 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13230 2024-09-14T06:17:23.214 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.214 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.214 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.214 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.214 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:23.214 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.214 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.214 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.214 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.214 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:23.214 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.214 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.214 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.215 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.215 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098791+0000", 2024-09-14T06:17:23.215 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8800000000000001e-07 2024-09-14T06:17:23.215 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.215 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.215 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.215 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098795+0000", 2024-09-14T06:17:23.215 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.391e-06 2024-09-14T06:17:23.215 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.215 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.215 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.215 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098804+0000", 2024-09-14T06:17:23.215 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3440000000000001e-06 2024-09-14T06:17:23.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.216 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.216 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.216 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098810+0000", 2024-09-14T06:17:23.216 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2339999999999999e-06 2024-09-14T06:17:23.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.216 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.216 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.216 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160930+0000", 2024-09-14T06:17:23.216 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062120018999999999 2024-09-14T06:17:23.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.216 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.216 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.216 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161017+0000", 2024-09-14T06:17:23.217 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6868999999999998e-05 2024-09-14T06:17:23.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.217 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.217 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.217 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161300+0000", 2024-09-14T06:17:23.217 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028349499999999998 2024-09-14T06:17:23.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.217 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.217 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.217 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164612+0000", 2024-09-14T06:17:23.217 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033119680000000002 2024-09-14T06:17:23.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.217 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.217 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.218 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180559+0000", 2024-09-14T06:17:23.218 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015946612999999998 2024-09-14T06:17:23.218 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.218 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.218 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.218 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180605+0000", 2024-09-14T06:17:23.218 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5890000000000003e-05 2024-09-14T06:17:23.218 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.218 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.218 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.218 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180642+0000", 2024-09-14T06:17:23.218 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:17:23.218 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.218 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.219 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.219 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.219 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.219 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13231 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.219 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:23.219 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.104377790999997, 2024-09-14T06:17:23.219 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081750685000000003, 2024-09-14T06:17:23.219 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.219 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.219 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.219 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.219 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.219 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.219 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13231 2024-09-14T06:17:23.220 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.220 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.220 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.220 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.220 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:23.220 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.220 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.220 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.220 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.220 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:23.220 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.220 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.221 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.221 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.221 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099053+0000", 2024-09-14T06:17:23.221 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:17:23.221 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.221 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.221 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.221 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099059+0000", 2024-09-14T06:17:23.221 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1789999999999999e-06 2024-09-14T06:17:23.221 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.221 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.221 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.221 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099072+0000", 2024-09-14T06:17:23.221 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3179999999999999e-05 2024-09-14T06:17:23.222 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.222 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.222 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.222 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099080+0000", 2024-09-14T06:17:23.222 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9030000000000004e-06 2024-09-14T06:17:23.222 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.222 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.222 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.222 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161681+0000", 2024-09-14T06:17:23.222 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062601641999999999 2024-09-14T06:17:23.222 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.222 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.222 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.222 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161766+0000", 2024-09-14T06:17:23.222 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4909999999999998e-05 2024-09-14T06:17:23.223 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.223 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.223 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.223 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162109+0000", 2024-09-14T06:17:23.223 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00034264899999999999 2024-09-14T06:17:23.223 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.223 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.223 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.223 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164615+0000", 2024-09-14T06:17:23.223 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025061979999999998 2024-09-14T06:17:23.223 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.223 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.223 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.223 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180718+0000", 2024-09-14T06:17:23.224 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016103240000000001 2024-09-14T06:17:23.224 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.224 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.224 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.224 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180768+0000", 2024-09-14T06:17:23.224 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9781999999999999e-05 2024-09-14T06:17:23.224 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.224 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.224 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.224 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180803+0000", 2024-09-14T06:17:23.224 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5021999999999998e-05 2024-09-14T06:17:23.224 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.224 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.225 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.225 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.225 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.225 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13232 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.225 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:23.225 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.104336017000001, 2024-09-14T06:17:23.225 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081861981, 2024-09-14T06:17:23.225 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.225 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.225 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.225 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.225 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.225 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.225 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13232 2024-09-14T06:17:23.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.226 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.226 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.226 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:23.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.226 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.226 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:23.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.226 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.226 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099095+0000", 2024-09-14T06:17:23.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:17:23.227 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.227 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.227 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.227 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099098+0000", 2024-09-14T06:17:23.227 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5420000000000001e-06 2024-09-14T06:17:23.227 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.227 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.227 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.227 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099106+0000", 2024-09-14T06:17:23.227 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.824e-06 2024-09-14T06:17:23.227 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.227 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.227 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.227 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099112+0000", 2024-09-14T06:17:23.228 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9789999999999998e-06 2024-09-14T06:17:23.228 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.228 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.228 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.228 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162576+0000", 2024-09-14T06:17:23.228 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.063463723999999999 2024-09-14T06:17:23.228 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.228 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.228 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.228 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162627+0000", 2024-09-14T06:17:23.228 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1530000000000003e-05 2024-09-14T06:17:23.228 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.228 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.229 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.229 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162854+0000", 2024-09-14T06:17:23.229 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000226159 2024-09-14T06:17:23.229 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.229 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.229 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.229 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166090+0000", 2024-09-14T06:17:23.229 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032362319999999999 2024-09-14T06:17:23.229 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.230 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.230 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.230 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180877+0000", 2024-09-14T06:17:23.230 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014786726 2024-09-14T06:17:23.230 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.230 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.230 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.230 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180920+0000", 2024-09-14T06:17:23.230 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3427000000000003e-05 2024-09-14T06:17:23.230 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.230 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.230 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.230 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180956+0000", 2024-09-14T06:17:23.230 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6195999999999997e-05 2024-09-14T06:17:23.231 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.231 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.231 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.231 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.231 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.231 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13233 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.231 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:23.231 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.104310753999997, 2024-09-14T06:17:23.231 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081924860000000002, 2024-09-14T06:17:23.231 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.231 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.231 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.231 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.232 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.232 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.232 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13233 2024-09-14T06:17:23.232 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.232 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.232 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.232 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.232 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:23.232 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.232 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.232 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.232 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.232 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:23.233 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.233 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.233 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.233 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.233 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099120+0000", 2024-09-14T06:17:23.233 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:17:23.233 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.233 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.233 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.233 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099122+0000", 2024-09-14T06:17:23.233 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:17:23.233 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.233 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.233 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.233 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099126+0000", 2024-09-14T06:17:23.234 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2540000000000003e-06 2024-09-14T06:17:23.234 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.234 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.234 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.234 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099130+0000", 2024-09-14T06:17:23.234 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4060000000000001e-06 2024-09-14T06:17:23.234 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.234 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.234 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.234 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163169+0000", 2024-09-14T06:17:23.234 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064039415000000002 2024-09-14T06:17:23.234 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.234 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.234 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.234 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163260+0000", 2024-09-14T06:17:23.235 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1038000000000005e-05 2024-09-14T06:17:23.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.235 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.235 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.235 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163640+0000", 2024-09-14T06:17:23.235 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037954699999999999 2024-09-14T06:17:23.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.235 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.235 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.235 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166093+0000", 2024-09-14T06:17:23.235 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024533229999999999 2024-09-14T06:17:23.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.235 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.235 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.236 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180993+0000", 2024-09-14T06:17:23.236 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01489997 2024-09-14T06:17:23.236 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.236 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.236 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.236 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181020+0000", 2024-09-14T06:17:23.236 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6806999999999999e-05 2024-09-14T06:17:23.236 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.236 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.236 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.236 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181044+0000", 2024-09-14T06:17:23.236 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4405000000000001e-05 2024-09-14T06:17:23.236 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.236 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.237 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.237 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.237 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.237 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13234 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.237 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:23.237 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.104268548999997, 2024-09-14T06:17:23.237 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082015597999999995, 2024-09-14T06:17:23.237 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.237 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.237 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.237 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.237 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.237 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.238 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13234 2024-09-14T06:17:23.238 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.238 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.238 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.238 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.238 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:23.238 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.238 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.238 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.238 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.238 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:23.238 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.238 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.238 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.239 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.239 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:23.239 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0999999999999998e-07 2024-09-14T06:17:23.239 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.239 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.239 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.239 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099166+0000", 2024-09-14T06:17:23.239 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:17:23.239 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.239 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.239 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.239 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099171+0000", 2024-09-14T06:17:23.239 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6949999999999997e-06 2024-09-14T06:17:23.239 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.239 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.240 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.240 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099175+0000", 2024-09-14T06:17:23.240 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9029999999999997e-06 2024-09-14T06:17:23.240 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.240 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.240 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.240 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164063+0000", 2024-09-14T06:17:23.240 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064887608999999999 2024-09-14T06:17:23.240 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.240 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.240 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.240 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164111+0000", 2024-09-14T06:17:23.240 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7648999999999997e-05 2024-09-14T06:17:23.241 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.241 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.241 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.241 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164351+0000", 2024-09-14T06:17:23.241 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024072899999999999 2024-09-14T06:17:23.241 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.241 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.241 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.241 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167561+0000", 2024-09-14T06:17:23.241 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032094179999999999 2024-09-14T06:17:23.241 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.241 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.241 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.241 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181093+0000", 2024-09-14T06:17:23.242 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013532245 2024-09-14T06:17:23.242 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.242 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.242 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.242 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181139+0000", 2024-09-14T06:17:23.242 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6483999999999997e-05 2024-09-14T06:17:23.242 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.242 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.242 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.242 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181177+0000", 2024-09-14T06:17:23.242 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7861000000000002e-05 2024-09-14T06:17:23.242 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.242 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.242 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.243 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.243 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.243 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13235 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.243 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:23.243 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.104161275000003, 2024-09-14T06:17:23.243 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081988088000000001, 2024-09-14T06:17:23.243 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.243 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.243 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.243 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.243 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.243 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.243 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13235 2024-09-14T06:17:23.243 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.244 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.244 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.244 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.244 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:23.244 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.244 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.244 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.244 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.244 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:23.244 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.244 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.244 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.244 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.245 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099270+0000", 2024-09-14T06:17:23.245 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7400000000000003e-07 2024-09-14T06:17:23.245 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.245 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.245 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.245 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099276+0000", 2024-09-14T06:17:23.245 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:17:23.245 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.245 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.245 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.245 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099284+0000", 2024-09-14T06:17:23.245 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7979999999999995e-06 2024-09-14T06:17:23.245 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.245 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.246 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.246 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099289+0000", 2024-09-14T06:17:23.246 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:17:23.246 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.246 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.246 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.246 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164664+0000", 2024-09-14T06:17:23.246 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065374901999999999 2024-09-14T06:17:23.246 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.246 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.246 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.246 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164755+0000", 2024-09-14T06:17:23.246 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0790000000000003e-05 2024-09-14T06:17:23.247 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.247 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.247 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.247 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165143+0000", 2024-09-14T06:17:23.247 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000387904 2024-09-14T06:17:23.247 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.247 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.247 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.247 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167564+0000", 2024-09-14T06:17:23.247 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024208989999999998 2024-09-14T06:17:23.247 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.247 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.247 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.247 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181212+0000", 2024-09-14T06:17:23.247 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013648664 2024-09-14T06:17:23.248 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.248 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.248 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.248 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181237+0000", 2024-09-14T06:17:23.248 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5177999999999998e-05 2024-09-14T06:17:23.248 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.248 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.248 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.248 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181257+0000", 2024-09-14T06:17:23.248 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9660000000000002e-05 2024-09-14T06:17:23.248 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.248 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.248 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.249 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.249 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.249 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13236 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.249 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:23.249 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.104054456, 2024-09-14T06:17:23.249 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082010939000000005, 2024-09-14T06:17:23.249 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.249 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.249 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.249 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.249 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.249 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.249 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13236 2024-09-14T06:17:23.250 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.250 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.250 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.250 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.250 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:23.250 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.250 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.250 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.250 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.250 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:23.250 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.250 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.250 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.250 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.251 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:23.251 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.61e-07 2024-09-14T06:17:23.251 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.251 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.251 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.251 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099382+0000", 2024-09-14T06:17:23.251 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3589999999999999e-06 2024-09-14T06:17:23.251 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.251 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.251 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.251 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099388+0000", 2024-09-14T06:17:23.251 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9619999999999998e-06 2024-09-14T06:17:23.252 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.252 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.252 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.252 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099393+0000", 2024-09-14T06:17:23.252 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3410000000000002e-06 2024-09-14T06:17:23.252 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.252 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.252 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.252 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165553+0000", 2024-09-14T06:17:23.252 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066159604999999996 2024-09-14T06:17:23.252 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.252 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.252 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.253 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165603+0000", 2024-09-14T06:17:23.253 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0427999999999999e-05 2024-09-14T06:17:23.253 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.253 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.253 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.253 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165827+0000", 2024-09-14T06:17:23.253 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022358299999999999 2024-09-14T06:17:23.253 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.253 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.253 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.253 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177005+0000", 2024-09-14T06:17:23.253 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011178044 2024-09-14T06:17:23.253 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.253 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.254 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.254 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181302+0000", 2024-09-14T06:17:23.254 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042970669999999999 2024-09-14T06:17:23.254 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.254 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.254 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.254 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181349+0000", 2024-09-14T06:17:23.254 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7287999999999999e-05 2024-09-14T06:17:23.254 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.254 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.254 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.254 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181387+0000", 2024-09-14T06:17:23.254 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7601000000000001e-05 2024-09-14T06:17:23.254 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.255 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.255 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.255 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.255 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.255 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13237 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.255 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:23.255 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.103896309, 2024-09-14T06:17:23.255 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081927299999999995, 2024-09-14T06:17:23.255 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.255 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.255 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.255 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.255 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.255 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.256 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13237 2024-09-14T06:17:23.256 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.256 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.256 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.256 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.256 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:23.256 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.256 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.256 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.256 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.256 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:23.256 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.256 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.256 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.257 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.257 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099535+0000", 2024-09-14T06:17:23.257 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:17:23.257 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.257 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.257 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.257 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099538+0000", 2024-09-14T06:17:23.257 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:17:23.257 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.257 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.257 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.257 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099545+0000", 2024-09-14T06:17:23.257 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:17:23.257 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.258 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.258 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.258 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099551+0000", 2024-09-14T06:17:23.258 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1770000000000003e-06 2024-09-14T06:17:23.258 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.258 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.258 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.258 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166131+0000", 2024-09-14T06:17:23.258 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066580499000000001 2024-09-14T06:17:23.258 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.258 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.258 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.259 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166221+0000", 2024-09-14T06:17:23.259 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0000000000000006e-05 2024-09-14T06:17:23.259 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.259 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.259 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.259 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166622+0000", 2024-09-14T06:17:23.259 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040029799999999999 2024-09-14T06:17:23.259 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.259 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.259 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.259 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177008+0000", 2024-09-14T06:17:23.259 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010386772000000001 2024-09-14T06:17:23.259 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.259 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.260 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.260 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181420+0000", 2024-09-14T06:17:23.260 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044119190000000003 2024-09-14T06:17:23.260 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.260 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.260 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.260 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181443+0000", 2024-09-14T06:17:23.260 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2603999999999999e-05 2024-09-14T06:17:23.260 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.260 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.261 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.261 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181461+0000", 2024-09-14T06:17:23.261 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8405000000000001e-05 2024-09-14T06:17:23.261 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.261 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.261 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.261 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.261 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.261 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13238 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.261 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:23.261 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.103860001000001, 2024-09-14T06:17:23.262 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082014927000000001, 2024-09-14T06:17:23.262 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.262 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.262 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.262 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.262 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.262 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.262 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13238 2024-09-14T06:17:23.262 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.262 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.262 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.262 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.263 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:23.263 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.263 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.263 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.263 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.263 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:23.263 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.263 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.263 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.263 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.263 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099571+0000", 2024-09-14T06:17:23.263 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9699999999999996e-07 2024-09-14T06:17:23.263 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.264 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.264 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.264 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099577+0000", 2024-09-14T06:17:23.264 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2890000000000003e-06 2024-09-14T06:17:23.264 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.264 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.264 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.264 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099587+0000", 2024-09-14T06:17:23.264 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0002e-05 2024-09-14T06:17:23.264 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.264 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.264 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.264 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099593+0000", 2024-09-14T06:17:23.264 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4539999999999998e-06 2024-09-14T06:17:23.265 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.265 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.265 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.265 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167024+0000", 2024-09-14T06:17:23.265 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067431480000000002 2024-09-14T06:17:23.265 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.265 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.265 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.265 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167074+0000", 2024-09-14T06:17:23.265 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9996e-05 2024-09-14T06:17:23.265 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.265 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.265 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.266 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167288+0000", 2024-09-14T06:17:23.266 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021410900000000001 2024-09-14T06:17:23.266 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.266 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.266 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.266 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177997+0000", 2024-09-14T06:17:23.266 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010708756 2024-09-14T06:17:23.266 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.266 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.266 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.266 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181508+0000", 2024-09-14T06:17:23.266 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003511363 2024-09-14T06:17:23.266 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.266 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.267 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.267 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181552+0000", 2024-09-14T06:17:23.267 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3609999999999998e-05 2024-09-14T06:17:23.267 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.267 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.267 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.267 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181585+0000", 2024-09-14T06:17:23.267 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3271e-05 2024-09-14T06:17:23.267 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.267 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.267 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.267 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.268 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.268 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13239 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.268 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:23.268 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.103768183, 2024-09-14T06:17:23.268 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081998772999999997, 2024-09-14T06:17:23.268 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.268 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.268 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.268 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.268 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.268 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.268 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13239 2024-09-14T06:17:23.268 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.269 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.269 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.269 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.269 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:23.269 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.269 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.269 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.269 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.269 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:23.269 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.269 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.269 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.269 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.270 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099663+0000", 2024-09-14T06:17:23.270 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6100000000000001e-07 2024-09-14T06:17:23.270 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.270 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.270 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.270 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099666+0000", 2024-09-14T06:17:23.270 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.056e-06 2024-09-14T06:17:23.270 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.270 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.270 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.270 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099672+0000", 2024-09-14T06:17:23.270 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8120000000000002e-06 2024-09-14T06:17:23.270 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.270 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.271 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.271 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099676+0000", 2024-09-14T06:17:23.271 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:17:23.271 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.271 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.271 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.271 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167602+0000", 2024-09-14T06:17:23.271 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067925867000000001 2024-09-14T06:17:23.271 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.271 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.271 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.271 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167693+0000", 2024-09-14T06:17:23.271 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0736000000000007e-05 2024-09-14T06:17:23.271 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.271 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.272 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.272 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168065+0000", 2024-09-14T06:17:23.272 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037245399999999998 2024-09-14T06:17:23.272 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.272 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.272 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.272 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178003+0000", 2024-09-14T06:17:23.272 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0099372360000000003 2024-09-14T06:17:23.272 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.272 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.272 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.272 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181621+0000", 2024-09-14T06:17:23.272 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036178489999999998 2024-09-14T06:17:23.272 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.272 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.273 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.273 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181643+0000", 2024-09-14T06:17:23.273 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2155e-05 2024-09-14T06:17:23.273 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.273 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.273 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.273 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181661+0000", 2024-09-14T06:17:23.273 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8147000000000001e-05 2024-09-14T06:17:23.273 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.273 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.273 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.273 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.273 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.273 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13240 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.273 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.103649089999998, 2024-09-14T06:17:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081942607000000001, 2024-09-14T06:17:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13240 2024-09-14T06:17:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.274 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.275 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.275 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.275 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:23.275 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.275 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.275 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.275 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.275 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099782+0000", 2024-09-14T06:17:23.275 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3900000000000004e-07 2024-09-14T06:17:23.275 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.275 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.275 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.275 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099787+0000", 2024-09-14T06:17:23.275 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6189999999999999e-06 2024-09-14T06:17:23.275 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.276 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.276 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.276 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099793+0000", 2024-09-14T06:17:23.276 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.998e-06 2024-09-14T06:17:23.276 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.276 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.276 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.276 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099797+0000", 2024-09-14T06:17:23.276 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8449999999999996e-06 2024-09-14T06:17:23.276 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.276 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.276 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.276 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168494+0000", 2024-09-14T06:17:23.276 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.068696500999999993 2024-09-14T06:17:23.277 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.277 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.277 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.277 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168590+0000", 2024-09-14T06:17:23.277 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6733999999999999e-05 2024-09-14T06:17:23.277 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.277 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.277 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.277 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168973+0000", 2024-09-14T06:17:23.277 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038229399999999998 2024-09-14T06:17:23.277 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.277 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.277 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.277 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178013+0000", 2024-09-14T06:17:23.278 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0090397199999999994 2024-09-14T06:17:23.278 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.278 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.278 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.278 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181683+0000", 2024-09-14T06:17:23.278 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036705819999999999 2024-09-14T06:17:23.278 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.278 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.278 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.278 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181705+0000", 2024-09-14T06:17:23.278 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1551000000000001e-05 2024-09-14T06:17:23.278 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.278 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.279 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.279 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181724+0000", 2024-09-14T06:17:23.279 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9123999999999999e-05 2024-09-14T06:17:23.279 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.279 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.279 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.279 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.279 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.279 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13243 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.279 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:23.279 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.103115359999997, 2024-09-14T06:17:23.279 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081767815999999993, 2024-09-14T06:17:23.279 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.280 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.280 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.280 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.280 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.280 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.280 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13243 2024-09-14T06:17:23.280 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.280 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.280 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.280 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.280 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:23.280 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.280 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.280 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.281 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.281 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:23.281 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.281 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.281 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.281 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.281 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:23.281 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1e-07 2024-09-14T06:17:23.281 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.281 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.281 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.281 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100317+0000", 2024-09-14T06:17:23.281 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.092e-06 2024-09-14T06:17:23.282 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.282 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.282 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.282 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100321+0000", 2024-09-14T06:17:23.282 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7809999999999999e-06 2024-09-14T06:17:23.282 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.282 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.282 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.282 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100324+0000", 2024-09-14T06:17:23.282 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.117e-06 2024-09-14T06:17:23.282 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.282 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.282 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.282 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171113+0000", 2024-09-14T06:17:23.283 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.070788816000000004 2024-09-14T06:17:23.283 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.283 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.283 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.283 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171202+0000", 2024-09-14T06:17:23.283 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8843000000000003e-05 2024-09-14T06:17:23.283 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.283 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.283 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.283 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171584+0000", 2024-09-14T06:17:23.283 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038231099999999998 2024-09-14T06:17:23.283 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.283 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.283 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178101+0000", 2024-09-14T06:17:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0065168329999999997 2024-09-14T06:17:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182041+0000", 2024-09-14T06:17:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039401810000000001 2024-09-14T06:17:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182065+0000", 2024-09-14T06:17:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3623000000000001e-05 2024-09-14T06:17:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.284 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182083+0000", 2024-09-14T06:17:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7909000000000001e-05 2024-09-14T06:17:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13244 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.10310054, 2024-09-14T06:17:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081837800000000002, 2024-09-14T06:17:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.285 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13244 2024-09-14T06:17:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.286 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:17:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.287 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100337+0000", 2024-09-14T06:17:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.939e-06 2024-09-14T06:17:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100341+0000", 2024-09-14T06:17:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2320000000000003e-06 2024-09-14T06:17:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100344+0000", 2024-09-14T06:17:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0759999999999999e-06 2024-09-14T06:17:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.288 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171996+0000", 2024-09-14T06:17:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.071651919999999994 2024-09-14T06:17:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172079+0000", 2024-09-14T06:17:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2579999999999999e-05 2024-09-14T06:17:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172457+0000", 2024-09-14T06:17:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037785100000000001 2024-09-14T06:17:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.289 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178104+0000", 2024-09-14T06:17:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056471730000000001 2024-09-14T06:17:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182123+0000", 2024-09-14T06:17:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040190019999999998 2024-09-14T06:17:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182147+0000", 2024-09-14T06:17:23.290 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4241000000000001e-05 2024-09-14T06:17:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182168+0000", 2024-09-14T06:17:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0492999999999998e-05 2024-09-14T06:17:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13245 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:23.291 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.103081033000002, 2024-09-14T06:17:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081893899000000006, 2024-09-14T06:17:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13245 2024-09-14T06:17:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.292 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100350+0000", 2024-09-14T06:17:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5499999999999999e-07 2024-09-14T06:17:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.293 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100353+0000", 2024-09-14T06:17:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:17:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100357+0000", 2024-09-14T06:17:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8500000000000004e-06 2024-09-14T06:17:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100360+0000", 2024-09-14T06:17:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8700000000000001e-06 2024-09-14T06:17:23.294 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172862+0000", 2024-09-14T06:17:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.072501992000000001 2024-09-14T06:17:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172950+0000", 2024-09-14T06:17:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8988999999999998e-05 2024-09-14T06:17:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173324+0000", 2024-09-14T06:17:23.295 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037366199999999997 2024-09-14T06:17:23.296 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.296 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.296 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.296 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178107+0000", 2024-09-14T06:17:23.296 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047830520000000003 2024-09-14T06:17:23.296 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.296 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.296 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.296 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182202+0000", 2024-09-14T06:17:23.296 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040949280000000003 2024-09-14T06:17:23.296 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.296 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.296 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.297 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182226+0000", 2024-09-14T06:17:23.297 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3535000000000001e-05 2024-09-14T06:17:23.297 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.297 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.297 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.297 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182243+0000", 2024-09-14T06:17:23.297 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7445e-05 2024-09-14T06:17:23.297 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.297 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.297 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.297 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.297 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.297 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13246 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.298 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:23.298 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.103066710999997, 2024-09-14T06:17:23.298 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081962276000000001, 2024-09-14T06:17:23.298 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.298 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.298 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.298 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.298 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.298 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.298 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13246 2024-09-14T06:17:23.298 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.298 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.298 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.298 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.298 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:23.299 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.299 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.299 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.299 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.299 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:23.299 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.299 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.299 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.299 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.299 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:23.299 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0800000000000001e-07 2024-09-14T06:17:23.299 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.299 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.300 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.300 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100367+0000", 2024-09-14T06:17:23.300 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8439999999999998e-06 2024-09-14T06:17:23.300 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.300 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.300 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.300 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100372+0000", 2024-09-14T06:17:23.300 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4009999999999996e-06 2024-09-14T06:17:23.300 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.300 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.300 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.300 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100378+0000", 2024-09-14T06:17:23.300 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7180000000000001e-06 2024-09-14T06:17:23.301 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.301 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.301 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.301 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173780+0000", 2024-09-14T06:17:23.301 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.073401991 2024-09-14T06:17:23.301 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.301 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.301 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.301 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173879+0000", 2024-09-14T06:17:23.301 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9449000000000003e-05 2024-09-14T06:17:23.301 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.301 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.301 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.301 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174240+0000", 2024-09-14T06:17:23.302 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036118100000000002 2024-09-14T06:17:23.302 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.302 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.302 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.302 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178109+0000", 2024-09-14T06:17:23.302 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038690619999999999 2024-09-14T06:17:23.302 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.302 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.302 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.302 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182281+0000", 2024-09-14T06:17:23.302 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041715900000000002 2024-09-14T06:17:23.302 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.302 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.302 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.302 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182306+0000", 2024-09-14T06:17:23.303 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4477e-05 2024-09-14T06:17:23.303 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.303 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.303 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.303 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182326+0000", 2024-09-14T06:17:23.303 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0254999999999998e-05 2024-09-14T06:17:23.303 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.303 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.303 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.303 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.303 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.303 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13247 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.303 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:23.304 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.102969534000003, 2024-09-14T06:17:23.304 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966593000000004, 2024-09-14T06:17:23.304 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.304 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.304 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.304 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.304 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.304 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.304 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13247 2024-09-14T06:17:23.304 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.304 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.304 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.305 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.305 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:23.305 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.305 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.305 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.305 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.305 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:23.305 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.305 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.305 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.305 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.305 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100462+0000", 2024-09-14T06:17:23.305 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9800000000000002e-07 2024-09-14T06:17:23.305 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.306 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.306 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.306 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100466+0000", 2024-09-14T06:17:23.306 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4039999999999997e-06 2024-09-14T06:17:23.306 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.306 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.306 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.306 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100475+0000", 2024-09-14T06:17:23.306 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3680000000000008e-06 2024-09-14T06:17:23.306 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.306 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.306 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.306 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100482+0000", 2024-09-14T06:17:23.306 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8630000000000002e-06 2024-09-14T06:17:23.307 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.307 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.307 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.307 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174669+0000", 2024-09-14T06:17:23.307 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074186849999999999 2024-09-14T06:17:23.307 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.307 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.307 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.307 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174758+0000", 2024-09-14T06:17:23.307 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9021e-05 2024-09-14T06:17:23.307 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.307 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.307 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.307 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175137+0000", 2024-09-14T06:17:23.308 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378511 2024-09-14T06:17:23.308 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.308 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.308 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.308 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178158+0000", 2024-09-14T06:17:23.308 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003020956 2024-09-14T06:17:23.308 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.308 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.308 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.308 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182386+0000", 2024-09-14T06:17:23.308 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042278840000000003 2024-09-14T06:17:23.308 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.308 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.308 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.309 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182410+0000", 2024-09-14T06:17:23.309 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4026e-05 2024-09-14T06:17:23.309 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.309 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.309 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.309 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182427+0000", 2024-09-14T06:17:23.309 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7711999999999999e-05 2024-09-14T06:17:23.309 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.309 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.309 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.309 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.309 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.309 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13248 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.309 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:23.309 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.102887453000001, 2024-09-14T06:17:23.310 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966527999999997, 2024-09-14T06:17:23.310 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.310 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.310 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.310 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.310 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.310 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.310 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13248 2024-09-14T06:17:23.310 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.310 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.310 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.310 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.310 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:23.311 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.311 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.311 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.311 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.311 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:23.311 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.311 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.311 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.311 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.311 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:23.311 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:17:23.311 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.311 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.311 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.312 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100548+0000", 2024-09-14T06:17:23.312 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:17:23.312 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.312 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.312 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.312 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100557+0000", 2024-09-14T06:17:23.312 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:17:23.312 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.312 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.312 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.312 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100563+0000", 2024-09-14T06:17:23.312 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2519999999999996e-06 2024-09-14T06:17:23.312 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.312 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.313 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.313 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175572+0000", 2024-09-14T06:17:23.313 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075008459 2024-09-14T06:17:23.313 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.313 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.313 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.313 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175656+0000", 2024-09-14T06:17:23.313 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3906999999999995e-05 2024-09-14T06:17:23.313 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.313 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.313 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.313 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176010+0000", 2024-09-14T06:17:23.313 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00035447499999999998 2024-09-14T06:17:23.313 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.313 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.314 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.314 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178304+0000", 2024-09-14T06:17:23.314 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0022941379999999998 2024-09-14T06:17:23.314 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.314 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.314 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.314 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182467+0000", 2024-09-14T06:17:23.314 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041632020000000004 2024-09-14T06:17:23.314 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.314 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.314 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.314 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182491+0000", 2024-09-14T06:17:23.314 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3733000000000001e-05 2024-09-14T06:17:23.314 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.314 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.315 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.315 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182509+0000", 2024-09-14T06:17:23.315 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8113000000000001e-05 2024-09-14T06:17:23.315 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.315 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.315 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.315 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.315 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.315 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13249 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.315 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:23.315 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.102852519000002, 2024-09-14T06:17:23.315 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082037225000000005, 2024-09-14T06:17:23.315 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.315 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.315 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.316 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.316 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.316 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.316 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13249 2024-09-14T06:17:23.316 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.316 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.316 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.316 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.316 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:23.316 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.316 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.316 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.316 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.316 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:23.316 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.317 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.317 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.317 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.317 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:23.317 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4600000000000005e-07 2024-09-14T06:17:23.317 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.317 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.317 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.317 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100584+0000", 2024-09-14T06:17:23.317 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.947e-06 2024-09-14T06:17:23.317 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.317 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.317 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.317 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100593+0000", 2024-09-14T06:17:23.318 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9760000000000001e-06 2024-09-14T06:17:23.318 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.318 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.318 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.318 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100597+0000", 2024-09-14T06:17:23.318 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9689999999999996e-06 2024-09-14T06:17:23.318 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.318 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.318 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.318 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176461+0000", 2024-09-14T06:17:23.318 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075864132000000001 2024-09-14T06:17:23.318 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.318 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.318 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.318 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176516+0000", 2024-09-14T06:17:23.319 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4416999999999998e-05 2024-09-14T06:17:23.319 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.319 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.319 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.319 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176734+0000", 2024-09-14T06:17:23.319 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021859399999999999 2024-09-14T06:17:23.319 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.319 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.319 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.319 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178308+0000", 2024-09-14T06:17:23.319 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015733660000000001 2024-09-14T06:17:23.319 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.319 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.319 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.319 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182573+0000", 2024-09-14T06:17:23.319 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042649200000000002 2024-09-14T06:17:23.320 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.320 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.320 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.320 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182597+0000", 2024-09-14T06:17:23.320 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4355e-05 2024-09-14T06:17:23.320 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.320 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.320 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.320 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182615+0000", 2024-09-14T06:17:23.320 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8003e-05 2024-09-14T06:17:23.320 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.320 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.320 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.321 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.321 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.321 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13250 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:23.321 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:23.321 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 47.102687914000001, 2024-09-14T06:17:23.321 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081983890000000004, 2024-09-14T06:17:23.321 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.321 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.321 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.321 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.321 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.321 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.322 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13250 2024-09-14T06:17:23.322 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.322 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.322 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.322 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.322 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:23.322 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.322 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.322 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.322 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.322 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:23.322 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.322 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.322 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.322 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.323 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100743+0000", 2024-09-14T06:17:23.323 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:17:23.323 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.323 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.323 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.323 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100753+0000", 2024-09-14T06:17:23.323 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9979999999999998e-06 2024-09-14T06:17:23.323 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.323 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.323 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.323 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100772+0000", 2024-09-14T06:17:23.323 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8664e-05 2024-09-14T06:17:23.323 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.324 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.324 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.324 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100779+0000", 2024-09-14T06:17:23.324 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.046e-06 2024-09-14T06:17:23.324 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.324 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.324 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.324 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177049+0000", 2024-09-14T06:17:23.324 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076270819000000004 2024-09-14T06:17:23.324 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.324 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.324 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.324 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177140+0000", 2024-09-14T06:17:23.324 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0746000000000002e-05 2024-09-14T06:17:23.324 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.325 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.325 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:23.325 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177519+0000", 2024-09-14T06:17:23.325 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378656 2024-09-14T06:17:23.325 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.325 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.325 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.325 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178349+0000", 2024-09-14T06:17:23.325 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00083061599999999995 2024-09-14T06:17:23.325 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.325 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.325 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.325 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182679+0000", 2024-09-14T06:17:23.325 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043298379999999999 2024-09-14T06:17:23.326 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.326 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.326 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.326 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182705+0000", 2024-09-14T06:17:23.326 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5602999999999999e-05 2024-09-14T06:17:23.326 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.326 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.326 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.326 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182726+0000", 2024-09-14T06:17:23.326 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1297e-05 2024-09-14T06:17:23.326 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.326 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.326 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.326 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.326 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.327 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:23.328 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:17:23.329 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:23.326+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:23.429 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:23.426+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:23.469 INFO:teuthology.orchestra.run.smithi039.stdout:moved 0 / 82 (0%) 2024-09-14T06:17:23.469 INFO:teuthology.orchestra.run.smithi039.stdout:avg 10.25 2024-09-14T06:17:23.469 INFO:teuthology.orchestra.run.smithi039.stdout:stddev 2.94746 -> 2.94746 (expected baseline 2.99479) 2024-09-14T06:17:23.469 INFO:teuthology.orchestra.run.smithi039.stdout:min osd.3 with 4 -> 4 pgs (0.390244 -> 0.390244 * mean) 2024-09-14T06:17:23.469 INFO:teuthology.orchestra.run.smithi039.stdout:max osd.0 with 13 -> 13 pgs (1.26829 -> 1.26829 * mean) 2024-09-14T06:17:23.469 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:23.469 INFO:teuthology.orchestra.run.smithi039.stdout:oload 110 2024-09-14T06:17:23.469 INFO:teuthology.orchestra.run.smithi039.stdout:max_change 0.05 2024-09-14T06:17:23.469 INFO:teuthology.orchestra.run.smithi039.stdout:max_change_osds 4 2024-09-14T06:17:23.469 INFO:teuthology.orchestra.run.smithi039.stdout:average_utilization 0.0005 2024-09-14T06:17:23.469 INFO:teuthology.orchestra.run.smithi039.stdout:overload_utilization 0.0005 2024-09-14T06:17:23.470 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 weight 1.0000 -> 0.9500 2024-09-14T06:17:23.470 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 weight 0.1129 -> 0.1130 2024-09-14T06:17:23.489 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-09-14T06:17:23.499 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:23.499 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:23.499 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:23.500 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:23.505 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-14T06:17:23.506 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-14T06:17:23.513 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:17:23.559 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:23.557+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:23.589 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone26 from 0x5a5e540 (overlap 0x1d859f) to 0x0 2024-09-14T06:17:23.630 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:23.625+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:23.642 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:23.642 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:23.642 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:23.642 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:23.642 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:23.656 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:17:23.690 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-14T06:17:23.743 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:23.741+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:23.825 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:23.825 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:23.826 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:23.826 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:23.826 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.826 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:30547 3.1f 3:f84dfb53:::rbd_data.11529eb229bd.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write 16226533~50642] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T06:17:23.826 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:23.826 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.633905210999998, 2024-09-14T06:17:23.826 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0079770020000001, 2024-09-14T06:17:23.826 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.826 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.826 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.826 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.826 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.827 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.827 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 30547 2024-09-14T06:17:23.827 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.827 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.827 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.827 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.827 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:23.827 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.827 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.827 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.827 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.827 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:23.827 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.828 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.828 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.828 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.828 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189811+0000", 2024-09-14T06:17:23.828 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2060000000000001e-06 2024-09-14T06:17:23.828 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.828 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.828 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.828 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189871+0000", 2024-09-14T06:17:23.828 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9862999999999999e-05 2024-09-14T06:17:23.828 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.828 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.828 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.829 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189899+0000", 2024-09-14T06:17:23.829 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7849000000000001e-05 2024-09-14T06:17:23.829 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.829 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.829 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.829 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189915+0000", 2024-09-14T06:17:23.829 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:17:23.829 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.829 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.829 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.829 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190024+0000", 2024-09-14T06:17:23.829 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000109202 2024-09-14T06:17:23.829 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.829 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.830 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:17:23.830 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190041+0000", 2024-09-14T06:17:23.830 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7665000000000001e-05 2024-09-14T06:17:23.830 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.830 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.830 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.830 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188512+0000", 2024-09-14T06:17:23.830 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.99847100700000002 2024-09-14T06:17:23.830 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.830 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.830 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for missing object", 2024-09-14T06:17:23.830 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188646+0000", 2024-09-14T06:17:23.830 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000133986 2024-09-14T06:17:23.830 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.830 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.831 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.831 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193097+0000", 2024-09-14T06:17:23.831 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044507059999999996 2024-09-14T06:17:23.831 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.831 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.831 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.831 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193183+0000", 2024-09-14T06:17:23.831 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5496999999999998e-05 2024-09-14T06:17:23.831 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.831 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.831 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:17:23.831 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193307+0000", 2024-09-14T06:17:23.831 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000124611 2024-09-14T06:17:23.831 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.832 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.832 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.832 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.194318+0000", 2024-09-14T06:17:23.832 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0010106 2024-09-14T06:17:23.832 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.832 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.832 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.832 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197670+0000", 2024-09-14T06:17:23.832 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033523170000000001 2024-09-14T06:17:23.832 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.832 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.832 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.832 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197712+0000", 2024-09-14T06:17:23.832 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1681000000000003e-05 2024-09-14T06:17:23.832 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.833 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.833 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.833 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197786+0000", 2024-09-14T06:17:23.833 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4148999999999998e-05 2024-09-14T06:17:23.833 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.833 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.833 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.833 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.833 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.833 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32022 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:23.833 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:23.833 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.955525000000002, 2024-09-14T06:17:23.833 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095598104000000003, 2024-09-14T06:17:23.833 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.833 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.834 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.834 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.834 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.834 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.834 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32022 2024-09-14T06:17:23.834 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.834 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.834 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.834 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.834 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:23.834 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.834 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.834 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.834 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.834 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:23.835 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.835 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.835 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.835 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.835 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868190+0000", 2024-09-14T06:17:23.835 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6000000000000003e-07 2024-09-14T06:17:23.835 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.835 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.835 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.835 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868243+0000", 2024-09-14T06:17:23.835 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3443999999999999e-05 2024-09-14T06:17:23.835 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.835 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.835 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.836 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868260+0000", 2024-09-14T06:17:23.836 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6799e-05 2024-09-14T06:17:23.836 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.836 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.836 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.836 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868271+0000", 2024-09-14T06:17:23.836 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0523000000000001e-05 2024-09-14T06:17:23.836 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.836 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.836 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.836 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963502+0000", 2024-09-14T06:17:23.836 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095231466000000001 2024-09-14T06:17:23.836 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.836 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.836 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.837 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963636+0000", 2024-09-14T06:17:23.837 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013388699999999999 2024-09-14T06:17:23.837 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.837 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.837 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.837 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963787+0000", 2024-09-14T06:17:23.837 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000151225 2024-09-14T06:17:23.837 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.837 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.837 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.837 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.837 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.837 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32023 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5816320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:23.838 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:23.838 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.955429874, 2024-09-14T06:17:23.838 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095879753999999998, 2024-09-14T06:17:23.838 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.838 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.838 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.838 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.838 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.838 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.838 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32023 2024-09-14T06:17:23.838 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.838 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.838 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.838 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.838 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:23.838 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.839 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.839 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.839 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.839 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:23.839 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.839 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.839 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.839 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.839 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868285+0000", 2024-09-14T06:17:23.839 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6299999999999993e-07 2024-09-14T06:17:23.839 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.839 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.839 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.839 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868288+0000", 2024-09-14T06:17:23.839 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.18e-06 2024-09-14T06:17:23.840 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.840 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.840 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.840 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868298+0000", 2024-09-14T06:17:23.840 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0029e-05 2024-09-14T06:17:23.840 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.840 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.840 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.840 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868307+0000", 2024-09-14T06:17:23.840 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7229999999999993e-06 2024-09-14T06:17:23.840 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.840 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.840 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.840 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963871+0000", 2024-09-14T06:17:23.840 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095563876000000006 2024-09-14T06:17:23.841 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.841 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.841 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.841 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964015+0000", 2024-09-14T06:17:23.841 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143573 2024-09-14T06:17:23.841 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.841 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.841 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.841 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964164+0000", 2024-09-14T06:17:23.841 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014940999999999999 2024-09-14T06:17:23.841 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.841 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.841 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.841 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.841 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.842 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32024 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:23.842 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:23.842 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.95539673, 2024-09-14T06:17:23.842 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096230231999999999, 2024-09-14T06:17:23.842 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.842 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.842 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.842 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.842 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.842 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.842 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32024 2024-09-14T06:17:23.842 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.842 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.842 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.842 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.843 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:23.843 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.843 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.843 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.843 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.843 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:23.843 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.843 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.843 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.843 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.843 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868318+0000", 2024-09-14T06:17:23.843 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1699999999999997e-07 2024-09-14T06:17:23.843 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.843 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.843 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.844 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868321+0000", 2024-09-14T06:17:23.844 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6769999999999999e-06 2024-09-14T06:17:23.844 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.844 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.844 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.844 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868334+0000", 2024-09-14T06:17:23.844 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2928999999999999e-05 2024-09-14T06:17:23.844 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.844 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.844 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.844 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868341+0000", 2024-09-14T06:17:23.844 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9800000000000001e-06 2024-09-14T06:17:23.844 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.844 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.845 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.845 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964258+0000", 2024-09-14T06:17:23.845 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095917099000000006 2024-09-14T06:17:23.845 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.845 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.845 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.845 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964392+0000", 2024-09-14T06:17:23.845 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013396999999999999 2024-09-14T06:17:23.845 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.845 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.845 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.845 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964548+0000", 2024-09-14T06:17:23.845 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015585999999999999 2024-09-14T06:17:23.846 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.846 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.846 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.846 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.846 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.846 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32025 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:23.846 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:23.846 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.955364073999998, 2024-09-14T06:17:23.846 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096576591000000003, 2024-09-14T06:17:23.846 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.846 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.846 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.846 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.846 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.846 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.847 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32025 2024-09-14T06:17:23.847 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.847 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.847 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.847 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.847 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:23.847 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.847 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.847 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.847 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.847 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:23.847 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.847 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.847 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.847 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.848 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868351+0000", 2024-09-14T06:17:23.848 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1600000000000001e-07 2024-09-14T06:17:23.848 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.848 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.848 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.848 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868353+0000", 2024-09-14T06:17:23.848 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.627e-06 2024-09-14T06:17:23.848 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.848 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.848 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.848 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868364+0000", 2024-09-14T06:17:23.848 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0893000000000001e-05 2024-09-14T06:17:23.849 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.849 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.849 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.849 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868381+0000", 2024-09-14T06:17:23.849 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6769000000000002e-05 2024-09-14T06:17:23.849 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.849 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.849 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.849 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964632+0000", 2024-09-14T06:17:23.849 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096250589999999997 2024-09-14T06:17:23.849 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.849 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.849 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.849 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964772+0000", 2024-09-14T06:17:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014045299999999999 2024-09-14T06:17:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964927+0000", 2024-09-14T06:17:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015454299999999999 2024-09-14T06:17:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32026 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.955323122999999, 2024-09-14T06:17:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096901622000000007, 2024-09-14T06:17:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.851 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.851 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.851 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.851 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.851 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.851 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32026 2024-09-14T06:17:23.851 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.851 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.851 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.851 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.851 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:23.851 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.851 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.852 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.852 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.852 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:23.852 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.852 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.852 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.852 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.852 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868392+0000", 2024-09-14T06:17:23.852 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0699999999999996e-07 2024-09-14T06:17:23.852 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.852 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.852 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.852 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868395+0000", 2024-09-14T06:17:23.852 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9859999999999999e-06 2024-09-14T06:17:23.853 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.853 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.853 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.853 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868405+0000", 2024-09-14T06:17:23.853 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9229999999999997e-06 2024-09-14T06:17:23.853 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.853 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.853 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.853 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868412+0000", 2024-09-14T06:17:23.853 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9670000000000004e-06 2024-09-14T06:17:23.853 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.853 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.853 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965010+0000", 2024-09-14T06:17:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096598562999999998 2024-09-14T06:17:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965143+0000", 2024-09-14T06:17:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013247700000000001 2024-09-14T06:17:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965293+0000", 2024-09-14T06:17:23.854 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000149899 2024-09-14T06:17:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32027 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.955254786000001, 2024-09-14T06:17:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097211306999999997, 2024-09-14T06:17:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.855 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.856 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.856 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32027 2024-09-14T06:17:23.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.856 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.856 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.856 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.856 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:23.856 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.856 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.856 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.856 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:23.856 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.856 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.857 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.857 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868461+0000", 2024-09-14T06:17:23.857 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.296e-06 2024-09-14T06:17:23.857 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.857 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.857 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.857 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868465+0000", 2024-09-14T06:17:23.857 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9570000000000004e-06 2024-09-14T06:17:23.857 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.857 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.857 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.857 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868476+0000", 2024-09-14T06:17:23.857 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1365999999999999e-05 2024-09-14T06:17:23.858 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.858 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.858 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.858 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868483+0000", 2024-09-14T06:17:23.858 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2529999999999998e-06 2024-09-14T06:17:23.858 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.858 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.858 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.858 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965375+0000", 2024-09-14T06:17:23.858 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096891973000000006 2024-09-14T06:17:23.858 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.858 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.858 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.858 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965518+0000", 2024-09-14T06:17:23.859 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143017 2024-09-14T06:17:23.859 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.859 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.859 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.859 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965671+0000", 2024-09-14T06:17:23.859 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015244499999999999 2024-09-14T06:17:23.859 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.859 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.859 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.859 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.859 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.859 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32028 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:23.859 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:23.859 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.955219958000001, 2024-09-14T06:17:23.860 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097841480999999994, 2024-09-14T06:17:23.860 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.860 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.860 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.860 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.860 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.860 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.860 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32028 2024-09-14T06:17:23.860 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.860 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.860 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.860 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.860 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:23.861 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.861 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.861 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.861 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.861 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:23.861 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.861 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.861 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.861 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.861 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868495+0000", 2024-09-14T06:17:23.861 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6400000000000001e-07 2024-09-14T06:17:23.861 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.861 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.861 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.862 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868498+0000", 2024-09-14T06:17:23.862 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7760000000000002e-06 2024-09-14T06:17:23.862 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.862 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.862 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.862 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868507+0000", 2024-09-14T06:17:23.862 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1530000000000003e-06 2024-09-14T06:17:23.862 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.862 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.862 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.862 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868516+0000", 2024-09-14T06:17:23.862 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8929999999999994e-06 2024-09-14T06:17:23.862 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.862 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.863 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.863 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965753+0000", 2024-09-14T06:17:23.863 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097236970000000006 2024-09-14T06:17:23.863 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.863 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.863 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.863 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965890+0000", 2024-09-14T06:17:23.863 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013677599999999999 2024-09-14T06:17:23.863 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.863 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.863 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.863 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966336+0000", 2024-09-14T06:17:23.863 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00044614900000000001 2024-09-14T06:17:23.863 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.863 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.864 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.864 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.864 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.864 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32029 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:23.864 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:23.864 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.955188883000002, 2024-09-14T06:17:23.864 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098577779000000004, 2024-09-14T06:17:23.864 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.864 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.864 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.864 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.864 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.864 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.864 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32029 2024-09-14T06:17:23.865 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.865 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.865 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.865 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.865 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:23.865 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.865 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.865 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.865 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.865 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:23.865 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.865 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.865 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.865 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.865 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868526+0000", 2024-09-14T06:17:23.866 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9300000000000006e-07 2024-09-14T06:17:23.866 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.866 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.866 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.866 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868529+0000", 2024-09-14T06:17:23.866 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5969999999999999e-06 2024-09-14T06:17:23.866 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.866 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.866 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.866 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868538+0000", 2024-09-14T06:17:23.866 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1700000000000003e-06 2024-09-14T06:17:23.866 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.866 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.866 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.866 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868560+0000", 2024-09-14T06:17:23.867 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1596000000000001e-05 2024-09-14T06:17:23.867 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.867 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.867 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.867 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966423+0000", 2024-09-14T06:17:23.867 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097863396000000005 2024-09-14T06:17:23.867 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.867 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.867 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.867 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966578+0000", 2024-09-14T06:17:23.867 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000154792 2024-09-14T06:17:23.867 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.867 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.867 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.867 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967103+0000", 2024-09-14T06:17:23.867 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00052523499999999998 2024-09-14T06:17:23.868 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.868 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.868 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.868 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.868 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.868 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32030 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:23.868 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:23.868 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.955143965000001, 2024-09-14T06:17:23.868 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099324030999999993, 2024-09-14T06:17:23.868 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.868 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.868 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.868 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.869 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.869 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.869 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32030 2024-09-14T06:17:23.869 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.869 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.869 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.869 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.869 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:23.869 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.869 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.869 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.869 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.869 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:23.870 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.870 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.870 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868571+0000", 2024-09-14T06:17:23.870 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9699999999999995e-07 2024-09-14T06:17:23.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.870 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.870 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868574+0000", 2024-09-14T06:17:23.870 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9529999999999999e-06 2024-09-14T06:17:23.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.870 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.870 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868584+0000", 2024-09-14T06:17:23.871 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7860000000000008e-06 2024-09-14T06:17:23.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.871 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.871 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.871 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868591+0000", 2024-09-14T06:17:23.871 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0500000000000003e-06 2024-09-14T06:17:23.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.871 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.871 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.871 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967191+0000", 2024-09-14T06:17:23.871 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098600035000000003 2024-09-14T06:17:23.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.871 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.871 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.871 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967333+0000", 2024-09-14T06:17:23.872 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014190599999999999 2024-09-14T06:17:23.872 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.872 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.872 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.872 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967894+0000", 2024-09-14T06:17:23.872 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000561504 2024-09-14T06:17:23.872 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.872 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.872 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.872 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.872 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.872 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32031 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:23.872 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:23.872 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.955114403, 2024-09-14T06:17:23.872 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.10010279, 2024-09-14T06:17:23.873 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.873 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.873 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.873 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.873 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.873 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.873 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32031 2024-09-14T06:17:23.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.873 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.873 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.873 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.873 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:23.873 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.873 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.874 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.874 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:23.874 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.874 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.874 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:23.874 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:17:23.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.874 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.874 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868603+0000", 2024-09-14T06:17:23.874 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.599e-06 2024-09-14T06:17:23.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.875 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.875 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868612+0000", 2024-09-14T06:17:23.875 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.986e-06 2024-09-14T06:17:23.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.875 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.875 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.875 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868619+0000", 2024-09-14T06:17:23.875 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8900000000000001e-06 2024-09-14T06:17:23.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.875 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.875 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.875 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967983+0000", 2024-09-14T06:17:23.875 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099363612000000004 2024-09-14T06:17:23.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.875 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.876 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.876 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968140+0000", 2024-09-14T06:17:23.876 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015788600000000001 2024-09-14T06:17:23.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.876 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.876 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.876 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968703+0000", 2024-09-14T06:17:23.876 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00056208300000000001 2024-09-14T06:17:23.876 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.876 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.876 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.876 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.876 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32032 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6258688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:23.877 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:23.877 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.955085927999999, 2024-09-14T06:17:23.877 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100488088, 2024-09-14T06:17:23.877 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.877 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.877 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.877 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.877 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.877 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.877 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32032 2024-09-14T06:17:23.877 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.877 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.877 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.878 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.878 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:23.878 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.878 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.878 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.878 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:23.878 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.878 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.878 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.878 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868629+0000", 2024-09-14T06:17:23.878 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:17:23.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.878 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.879 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.879 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868631+0000", 2024-09-14T06:17:23.879 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4099999999999998e-06 2024-09-14T06:17:23.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.879 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.879 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868640+0000", 2024-09-14T06:17:23.879 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0939999999999997e-06 2024-09-14T06:17:23.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.879 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.879 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868647+0000", 2024-09-14T06:17:23.879 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9460000000000001e-06 2024-09-14T06:17:23.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.879 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968797+0000", 2024-09-14T06:17:23.880 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100149867 2024-09-14T06:17:23.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.880 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.880 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968937+0000", 2024-09-14T06:17:23.880 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013981399999999999 2024-09-14T06:17:23.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.880 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.880 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969116+0000", 2024-09-14T06:17:23.880 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017927399999999999 2024-09-14T06:17:23.880 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.880 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.881 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.881 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.881 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32044 3.1e 3:7e6619d4:::rbd_data.115274ed0aa8.0000000000000068:head [set-alloc-hint object_size 1048576 write_size 1048576,write 835584~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:23.881 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:23.881 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 19.947396845, 2024-09-14T06:17:23.881 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096039367, 2024-09-14T06:17:23.881 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.881 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.881 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:23.881 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.881 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.881 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.881 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32044 2024-09-14T06:17:23.882 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.882 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.882 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.882 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.882 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:23.882 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.882 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.882 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.882 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.882 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:23.882 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.882 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.882 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.882 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.882 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876320+0000", 2024-09-14T06:17:23.882 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8270000000000002e-06 2024-09-14T06:17:23.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.883 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.883 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.883 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876397+0000", 2024-09-14T06:17:23.883 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6985999999999998e-05 2024-09-14T06:17:23.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.883 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.883 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.883 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876431+0000", 2024-09-14T06:17:23.883 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3719e-05 2024-09-14T06:17:23.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.883 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.883 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.883 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876452+0000", 2024-09-14T06:17:23.883 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0891999999999999e-05 2024-09-14T06:17:23.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.884 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.884 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.884 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969186+0000", 2024-09-14T06:17:23.884 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.092734681999999999 2024-09-14T06:17:23.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.884 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.884 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.884 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969550+0000", 2024-09-14T06:17:23.884 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036402199999999998 2024-09-14T06:17:23.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.884 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.884 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:17:23.884 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969987+0000", 2024-09-14T06:17:23.884 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000436759 2024-09-14T06:17:23.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:23.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971569+0000", 2024-09-14T06:17:23.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001581821 2024-09-14T06:17:23.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:23.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971947+0000", 2024-09-14T06:17:23.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378446 2024-09-14T06:17:23.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:23.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972146+0000", 2024-09-14T06:17:23.886 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019819 2024-09-14T06:17:23.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.886 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.886 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972357+0000", 2024-09-14T06:17:23.886 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021102299999999999 2024-09-14T06:17:23.886 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.886 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.886 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.886 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40252 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 827392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:23.886 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:23.886 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 14.106760739, 2024-09-14T06:17:23.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095633474999999996, 2024-09-14T06:17:23.887 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.887 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.887 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.887 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.887 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.887 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.887 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40252 2024-09-14T06:17:23.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.887 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.887 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.887 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.887 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:23.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.888 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.888 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.888 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:23.888 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.888 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.888 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.888 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.888 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717376+0000", 2024-09-14T06:17:23.888 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000422216 2024-09-14T06:17:23.888 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.888 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.888 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.888 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717386+0000", 2024-09-14T06:17:23.889 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0396e-05 2024-09-14T06:17:23.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.889 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.889 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.889 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717395+0000", 2024-09-14T06:17:23.889 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7819999999999999e-06 2024-09-14T06:17:23.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.889 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.889 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.889 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717400+0000", 2024-09-14T06:17:23.889 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1599999999999997e-06 2024-09-14T06:17:23.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.889 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.889 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.890 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.811922+0000", 2024-09-14T06:17:23.890 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.094522018999999999 2024-09-14T06:17:23.890 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.890 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.890 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.890 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812007+0000", 2024-09-14T06:17:23.890 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5328999999999996e-05 2024-09-14T06:17:23.890 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.890 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.890 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.890 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812587+0000", 2024-09-14T06:17:23.890 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000579573 2024-09-14T06:17:23.890 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.890 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.891 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.891 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.891 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40254 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:23.891 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:23.891 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 14.106302566, 2024-09-14T06:17:23.891 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095562731999999997, 2024-09-14T06:17:23.891 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.891 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.891 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.891 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.891 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.891 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.891 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40254 2024-09-14T06:17:23.892 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.892 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.892 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.892 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.892 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:23.892 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.892 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.892 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.892 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.892 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:23.892 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.892 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.892 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.892 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.892 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:23.893 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1699999999999999e-07 2024-09-14T06:17:23.893 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.893 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.893 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.893 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717414+0000", 2024-09-14T06:17:23.893 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:17:23.893 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.893 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.893 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.893 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717419+0000", 2024-09-14T06:17:23.893 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2839999999999997e-06 2024-09-14T06:17:23.893 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.893 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.893 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.894 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717424+0000", 2024-09-14T06:17:23.894 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1549999999999998e-06 2024-09-14T06:17:23.894 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.894 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.894 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.894 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812619+0000", 2024-09-14T06:17:23.894 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095194940000000006 2024-09-14T06:17:23.894 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.894 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.894 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.894 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812717+0000", 2024-09-14T06:17:23.894 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7647999999999997e-05 2024-09-14T06:17:23.894 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.894 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.894 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.894 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812974+0000", 2024-09-14T06:17:23.895 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025752099999999998 2024-09-14T06:17:23.895 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.895 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.895 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.895 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.895 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.895 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40256 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 843776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:23.895 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:23.895 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 14.106279962, 2024-09-14T06:17:23.895 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095932066999999996, 2024-09-14T06:17:23.895 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.895 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.895 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.895 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.896 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.896 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.896 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40256 2024-09-14T06:17:23.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.896 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.896 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.896 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.896 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:23.896 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.896 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.896 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.896 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:23.896 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.897 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.897 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.897 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717628+0000", 2024-09-14T06:17:23.897 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019339499999999999 2024-09-14T06:17:23.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.897 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.897 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.897 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717632+0000", 2024-09-14T06:17:23.897 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2230000000000001e-06 2024-09-14T06:17:23.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.897 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.897 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.897 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717641+0000", 2024-09-14T06:17:23.897 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6309999999999998e-06 2024-09-14T06:17:23.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.898 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.898 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.898 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717656+0000", 2024-09-14T06:17:23.898 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4724e-05 2024-09-14T06:17:23.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.898 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.898 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.898 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813006+0000", 2024-09-14T06:17:23.898 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095349801999999997 2024-09-14T06:17:23.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.898 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.898 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.898 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813107+0000", 2024-09-14T06:17:23.898 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000100973 2024-09-14T06:17:23.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.899 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.899 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.899 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813366+0000", 2024-09-14T06:17:23.899 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025931899999999999 2024-09-14T06:17:23.899 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.899 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.899 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.899 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.899 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40258 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 851968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:23.899 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:23.899 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 14.106043862, 2024-09-14T06:17:23.899 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096062543, 2024-09-14T06:17:23.899 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.899 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.900 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.900 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.900 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.900 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.900 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40258 2024-09-14T06:17:23.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.900 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.900 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.900 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.900 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:23.900 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.900 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.900 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.901 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:23.901 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.901 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.901 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717671+0000", 2024-09-14T06:17:23.901 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8299999999999997e-07 2024-09-14T06:17:23.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.901 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.901 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717674+0000", 2024-09-14T06:17:23.901 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7080000000000002e-06 2024-09-14T06:17:23.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.902 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.902 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717681+0000", 2024-09-14T06:17:23.902 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7889999999999995e-06 2024-09-14T06:17:23.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.902 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.902 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.902 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717687+0000", 2024-09-14T06:17:23.902 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7529999999999998e-06 2024-09-14T06:17:23.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.902 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.902 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.902 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813408+0000", 2024-09-14T06:17:23.902 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095720451999999998 2024-09-14T06:17:23.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.903 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813499+0000", 2024-09-14T06:17:23.903 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0933000000000005e-05 2024-09-14T06:17:23.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.903 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813733+0000", 2024-09-14T06:17:23.903 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00023432499999999999 2024-09-14T06:17:23.903 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.903 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.903 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.904 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40260 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 860160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:23.904 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:23.904 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 14.106018013, 2024-09-14T06:17:23.904 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096337381999999999, 2024-09-14T06:17:23.904 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.904 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.904 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.904 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.904 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.904 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.904 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40260 2024-09-14T06:17:23.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.904 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.904 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.905 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.905 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:23.905 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.905 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.905 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.905 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.905 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:23.905 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.905 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.905 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.905 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.905 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718008+0000", 2024-09-14T06:17:23.905 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00031192699999999998 2024-09-14T06:17:23.905 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.906 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.906 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.906 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718013+0000", 2024-09-14T06:17:23.906 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1830000000000002e-06 2024-09-14T06:17:23.906 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.906 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.906 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.906 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718036+0000", 2024-09-14T06:17:23.906 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2721000000000002e-05 2024-09-14T06:17:23.906 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.906 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.906 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.906 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718053+0000", 2024-09-14T06:17:23.906 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6736e-05 2024-09-14T06:17:23.906 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.906 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.907 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.907 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813767+0000", 2024-09-14T06:17:23.907 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095714658999999994 2024-09-14T06:17:23.907 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.907 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.907 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.907 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813816+0000", 2024-09-14T06:17:23.907 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8848999999999999e-05 2024-09-14T06:17:23.907 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.907 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.907 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.907 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814034+0000", 2024-09-14T06:17:23.907 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000217307 2024-09-14T06:17:23.908 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.908 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.908 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.908 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.908 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40262 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 868352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:23.908 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:23.908 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 14.10561833, 2024-09-14T06:17:23.908 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096354165000000006, 2024-09-14T06:17:23.908 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.908 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.908 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.908 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.909 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.909 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.909 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40262 2024-09-14T06:17:23.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.909 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.909 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:23.909 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.909 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:23.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.910 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.910 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718281+0000", 2024-09-14T06:17:23.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000185069 2024-09-14T06:17:23.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.910 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.910 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718286+0000", 2024-09-14T06:17:23.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.118e-06 2024-09-14T06:17:23.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.910 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.911 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718295+0000", 2024-09-14T06:17:23.911 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0429999999999996e-06 2024-09-14T06:17:23.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.911 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.911 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.911 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718302+0000", 2024-09-14T06:17:23.911 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2790000000000001e-06 2024-09-14T06:17:23.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.911 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.911 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.911 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814076+0000", 2024-09-14T06:17:23.911 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095774076999999999 2024-09-14T06:17:23.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.911 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.911 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.911 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814191+0000", 2024-09-14T06:17:23.912 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00011500700000000001 2024-09-14T06:17:23.912 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.912 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.912 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.912 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814450+0000", 2024-09-14T06:17:23.912 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025857199999999999 2024-09-14T06:17:23.912 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.912 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.912 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.912 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.912 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.912 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40264 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 876544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:23.912 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:23.912 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 14.105261262000001, 2024-09-14T06:17:23.912 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096372478999999997, 2024-09-14T06:17:23.913 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:23.913 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:23.913 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:23.913 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:23.913 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:23.913 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:23.913 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40264 2024-09-14T06:17:23.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.913 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:23.913 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.913 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:23.913 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:23.913 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.913 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:23.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:23.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:23.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:23.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718793+0000", 2024-09-14T06:17:23.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00033991099999999998 2024-09-14T06:17:23.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:23.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718805+0000", 2024-09-14T06:17:23.915 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2514000000000001e-05 2024-09-14T06:17:23.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.915 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:23.915 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718815+0000", 2024-09-14T06:17:23.915 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9590000000000007e-06 2024-09-14T06:17:23.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.915 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:23.915 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718822+0000", 2024-09-14T06:17:23.915 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2030000000000003e-06 2024-09-14T06:17:23.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:23.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814490+0000", 2024-09-14T06:17:23.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095667346 2024-09-14T06:17:23.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:23.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814577+0000", 2024-09-14T06:17:23.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6920000000000001e-05 2024-09-14T06:17:23.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:23.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:23.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:23.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814825+0000", 2024-09-14T06:17:23.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000248626 2024-09-14T06:17:23.916 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.917 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.917 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.917 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:23.917 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:23.917 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:23.919 INFO:teuthology.orchestra.run.smithi039.stdout:4318 trunc from 0x5a5e540 to 0x47a73cb 2024-09-14T06:17:23.919 INFO:teuthology.orchestra.run.smithi039.stdout:4320 writesame 0x1ea530e thru 0x1eb2dbf data_size 0x5d(0xdab2 bytes) 2024-09-14T06:17:23.919 INFO:teuthology.orchestra.run.smithi039.stdout:4321 read 0x2ef0ede thru 0x2efeabf (0xdbe2 bytes) 2024-09-14T06:17:23.919 INFO:teuthology.orchestra.run.smithi039.stdout:4322 punch from 0x3d86fb to 0x3e5689, (0xcf8e bytes) 2024-09-14T06:17:23.919 INFO:teuthology.orchestra.run.smithi039.stdout:4323 read 0x1630913 thru 0x1634c28 (0x4316 bytes) 2024-09-14T06:17:23.919 INFO:teuthology.orchestra.run.smithi039.stdout:4325 read 0x2357ddc thru 0x235afea (0x320f bytes) 2024-09-14T06:17:23.919 INFO:teuthology.orchestra.run.smithi039.stdout:4326 write 0xa768ef7 thru 0xa76f207 (0x6311 bytes) 2024-09-14T06:17:23.920 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:17:23.920 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:23.913+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:23.921 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-14T06:17:23.898+0000 7f4add550640 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-14T06:17:23.921 INFO:teuthology.orchestra.run.smithi039.stdout:moved 3 / 82 (3.65854%) 2024-09-14T06:17:23.921 INFO:teuthology.orchestra.run.smithi039.stdout:avg 10.25 2024-09-14T06:17:23.921 INFO:teuthology.orchestra.run.smithi039.stdout:stddev 2.94746 -> 2.90474 (expected baseline 2.99479) 2024-09-14T06:17:23.921 INFO:teuthology.orchestra.run.smithi039.stdout:min osd.3 with 4 -> 4 pgs (0.390244 -> 0.390244 * mean) 2024-09-14T06:17:23.921 INFO:teuthology.orchestra.run.smithi039.stdout:max osd.0 with 13 -> 13 pgs (1.26829 -> 1.26829 * mean) 2024-09-14T06:17:23.921 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:23.921 INFO:teuthology.orchestra.run.smithi039.stdout:oload 110 2024-09-14T06:17:23.921 INFO:teuthology.orchestra.run.smithi039.stdout:max_change 1 2024-09-14T06:17:23.922 INFO:teuthology.orchestra.run.smithi039.stdout:max_change_osds 4 2024-09-14T06:17:23.922 INFO:teuthology.orchestra.run.smithi039.stdout:average_utilization 0.0005 2024-09-14T06:17:23.922 INFO:teuthology.orchestra.run.smithi039.stdout:overload_utilization 0.0005 2024-09-14T06:17:23.922 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 weight 0.9500 -> 0.7443 2024-09-14T06:17:23.922 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 weight 0.1130 -> 0.1141 2024-09-14T06:17:23.922 INFO:teuthology.orchestra.run.smithi039.stderr:no change 2024-09-14T06:17:23.923 INFO:teuthology.orchestra.run.smithi039.stdout:4327 write 0xb62ec3c thru 0xb63b776 (0xcb3b bytes) 2024-09-14T06:17:23.923 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:23.923 INFO:teuthology.orchestra.run.smithi039.stdout:4330 write 0x612c335 thru 0x61394cf (0xd19b bytes) 2024-09-14T06:17:23.923 INFO:teuthology.orchestra.run.smithi039.stdout:4331 read 0x24bfa6e thru 0x24c93cd (0x9960 bytes) 2024-09-14T06:17:23.923 INFO:teuthology.orchestra.run.smithi039.stdout:4332 write 0x4e96cc0 thru 0x4ea55ef (0xe930 bytes) 2024-09-14T06:17:23.924 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-14T06:17:23.924 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-14T06:17:23.940 INFO:teuthology.orchestra.run.smithi039.stdout:4333 trunc from 0xb63b777 to 0x3965032 2024-09-14T06:17:23.988 INFO:teuthology.orchestra.run.smithi039.stdout:4335 read 0x843e4a thru 0x846107 (0x22be bytes) 2024-09-14T06:17:23.989 INFO:teuthology.orchestra.run.smithi039.stdout:4336 read 0xd833cf thru 0xd8d5e9 (0xa21b bytes) 2024-09-14T06:17:24.005 INFO:teuthology.orchestra.run.smithi039.stdout:4338 write 0x52cdd8b thru 0x52d2e59 (0x50cf bytes) 2024-09-14T06:17:24.010 INFO:teuthology.orchestra.run.smithi039.stdout:4339 read 0x3f3b6fd thru 0x3f3ecf8 (0x35fc bytes) 2024-09-14T06:17:24.018 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:24.014+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:24.021 INFO:teuthology.orchestra.run.smithi039.stdout:4341 writesame 0x58c4722 thru 0x58cd828 data_size 0x1(0x9107 bytes) 2024-09-14T06:17:24.021 INFO:teuthology.orchestra.run.smithi039.stdout:4342 punch from 0x5784691 to 0x5793689, (0xeff8 bytes) 2024-09-14T06:17:24.021 INFO:teuthology.orchestra.run.smithi039.stdout:4343 write 0x4088c19 thru 0x408f54c (0x6934 bytes) 2024-09-14T06:17:24.025 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:24.050 INFO:teuthology.orchestra.run.smithi039.stdout:4346 write 0xe574ec8 thru 0xe5838c1 (0xe9fa bytes) 2024-09-14T06:17:24.082 INFO:teuthology.orchestra.run.smithi039.stdout:4347 trunc from 0xe5838c2 to 0x30ea61d 2024-09-14T06:17:24.100 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:24.101 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:24.101 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:24.101 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:24.114 INFO:teuthology.orchestra.run.smithi039.stdout:4349 read 0x1f69a75 thru 0x1f6c66d (0x2bf9 bytes) 2024-09-14T06:17:24.115 INFO:teuthology.orchestra.run.smithi039.stdout:4350 read 0x2b0699d thru 0x2b10e9f (0xa503 bytes) 2024-09-14T06:17:24.118 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:17:24.119 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:24.114+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:24.137 INFO:teuthology.orchestra.run.smithi039.stdout:4353 writesame 0xcdefc19 thru 0xcdf695d data_size 0x1(0x6d45 bytes) 2024-09-14T06:17:24.137 INFO:teuthology.orchestra.run.smithi039.stdout:4354 write 0x207017c thru 0x207f11c (0xefa1 bytes) 2024-09-14T06:17:24.139 INFO:teuthology.orchestra.run.smithi039.stdout:4355 read 0x9172eb7 thru 0x9182852 (0xf99c bytes) 2024-09-14T06:17:24.147 INFO:teuthology.orchestra.run.smithi039.stdout:4356 write 0xe296b75 thru 0xe29fee2 (0x936e bytes) 2024-09-14T06:17:24.152 INFO:teuthology.orchestra.run.smithi039.stdout:4360 write 0xb9199e3 thru 0xb91a4b5 (0xad3 bytes) 2024-09-14T06:17:24.155 INFO:teuthology.orchestra.run.smithi039.stdout:4361 write 0xc9bf7c9 thru 0xc9c0f78 (0x17b0 bytes) 2024-09-14T06:17:24.161 INFO:teuthology.orchestra.run.smithi039.stdout:4362 punch from 0x590a779 to 0x590d349, (0x2bd0 bytes) 2024-09-14T06:17:24.161 INFO:teuthology.orchestra.run.smithi039.stdout:4364 writesame 0x97ed185 thru 0x97ef2d4 data_size 0x29(0x2150 bytes) 2024-09-14T06:17:24.165 INFO:teuthology.orchestra.run.smithi039.stdout:4365 write 0xb9868a0 thru 0xb9935aa (0xcd0b bytes) 2024-09-14T06:17:24.169 INFO:teuthology.orchestra.run.smithi039.stdout:4366 punch from 0xdf87038 to 0xdf9279d, (0xb765 bytes) 2024-09-14T06:17:24.169 INFO:teuthology.orchestra.run.smithi039.stdout:4367 clone 28 order 24 su 262144 sc 5 2024-09-14T06:17:24.221 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:24.218+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:24.238 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:24.238 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:24.238 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:24.238 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:24.238 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:24.252 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:17:24.322 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:24.317+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:24.378 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:24.378 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:24.378 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:24.378 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:24.378 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.378 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:3595 3.1 3:811c1c24:::rbd_data.1152bd92e04f.000000000000006e:head [set-alloc-hint object_size 2097152 write_size 2097152,write 440183~2185] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T06:17:24.378 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:24.378 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 58.152986083000002, 2024-09-14T06:17:24.378 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0365806689999999, 2024-09-14T06:17:24.378 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.378 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.378 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.378 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.379 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.379 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.379 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 3595 2024-09-14T06:17:24.379 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.379 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.379 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.379 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.379 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:24.379 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.379 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.379 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.379 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.379 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:24.379 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.379 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.380 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.380 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.380 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222823+0000", 2024-09-14T06:17:24.380 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8219999999999998e-06 2024-09-14T06:17:24.380 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.380 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.380 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.380 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222835+0000", 2024-09-14T06:17:24.380 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2052000000000001e-05 2024-09-14T06:17:24.380 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.380 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.380 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.380 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222859+0000", 2024-09-14T06:17:24.380 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3694e-05 2024-09-14T06:17:24.380 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.381 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.381 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.381 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222872+0000", 2024-09-14T06:17:24.381 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3203999999999999e-05 2024-09-14T06:17:24.381 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.381 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.381 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.381 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223068+0000", 2024-09-14T06:17:24.381 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000195995 2024-09-14T06:17:24.381 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.381 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.381 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:24.381 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223312+0000", 2024-09-14T06:17:24.381 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024424199999999998 2024-09-14T06:17:24.382 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.382 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.382 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:24.382 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223786+0000", 2024-09-14T06:17:24.382 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00047356600000000003 2024-09-14T06:17:24.382 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.382 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.382 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:24.382 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.225374+0000", 2024-09-14T06:17:24.382 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015875150000000001 2024-09-14T06:17:24.382 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.382 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.382 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.383 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.233122+0000", 2024-09-14T06:17:24.383 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0077480700000000001 2024-09-14T06:17:24.383 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.383 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.383 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:17:24.383 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.233157+0000", 2024-09-14T06:17:24.383 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5113000000000001e-05 2024-09-14T06:17:24.383 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.383 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.383 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.384 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.252998+0000", 2024-09-14T06:17:24.384 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.019841142 2024-09-14T06:17:24.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for degraded object", 2024-09-14T06:17:24.384 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.254121+0000", 2024-09-14T06:17:24.384 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0011227979999999999 2024-09-14T06:17:24.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.384 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.259263+0000", 2024-09-14T06:17:24.384 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0051426700000000002 2024-09-14T06:17:24.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.385 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.259401+0000", 2024-09-14T06:17:24.385 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013778600000000001 2024-09-14T06:17:24.385 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.385 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.385 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.385 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.385 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.385 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11421 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.385 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:24.385 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.390090897, 2024-09-14T06:17:24.385 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078553170000000005, 2024-09-14T06:17:24.385 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.385 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.386 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.386 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.386 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.386 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.386 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11421 2024-09-14T06:17:24.386 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.386 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.386 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.386 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.386 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:24.386 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.386 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.386 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.387 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.387 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:24.387 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.387 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.387 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.387 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.387 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985718+0000", 2024-09-14T06:17:24.387 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8840000000000001e-06 2024-09-14T06:17:24.387 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.387 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.387 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.387 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985723+0000", 2024-09-14T06:17:24.387 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8799999999999996e-06 2024-09-14T06:17:24.387 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.387 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.387 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.388 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985742+0000", 2024-09-14T06:17:24.388 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8776000000000001e-05 2024-09-14T06:17:24.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.388 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.388 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.388 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985759+0000", 2024-09-14T06:17:24.388 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7119999999999999e-05 2024-09-14T06:17:24.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.388 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.388 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.388 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064259+0000", 2024-09-14T06:17:24.388 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078499418000000001 2024-09-14T06:17:24.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.388 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.388 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.389 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064269+0000", 2024-09-14T06:17:24.389 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:17:24.389 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.389 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.389 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.389 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.389 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.389 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11422 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.389 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:24.389 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.390020929999999, 2024-09-14T06:17:24.389 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078521540000000001, 2024-09-14T06:17:24.389 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.389 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.389 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.389 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.390 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.390 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.390 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11422 2024-09-14T06:17:24.390 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.390 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.390 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.390 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:24.390 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.391 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.391 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.391 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.391 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:24.391 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.391 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.391 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.391 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.391 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985787+0000", 2024-09-14T06:17:24.391 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7630000000000001e-06 2024-09-14T06:17:24.391 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.391 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.391 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.391 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985842+0000", 2024-09-14T06:17:24.392 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4555999999999997e-05 2024-09-14T06:17:24.392 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.392 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.392 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.392 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985864+0000", 2024-09-14T06:17:24.392 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2189999999999999e-05 2024-09-14T06:17:24.392 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.392 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.392 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.392 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985879+0000", 2024-09-14T06:17:24.392 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4382999999999999e-05 2024-09-14T06:17:24.392 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.392 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.392 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.393 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064298+0000", 2024-09-14T06:17:24.393 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078419121999999994 2024-09-14T06:17:24.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.393 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.393 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064307+0000", 2024-09-14T06:17:24.393 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5259999999999999e-06 2024-09-14T06:17:24.393 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.393 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.393 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.393 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11423 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.394 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:24.394 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.389903558, 2024-09-14T06:17:24.394 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078437175999999997, 2024-09-14T06:17:24.394 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.394 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.394 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.394 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.394 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.394 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.394 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11423 2024-09-14T06:17:24.394 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.394 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.394 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.394 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.394 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:24.395 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.395 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.395 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.395 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.395 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:24.395 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.395 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.395 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.395 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.395 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985905+0000", 2024-09-14T06:17:24.395 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8139999999999999e-06 2024-09-14T06:17:24.395 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.395 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.395 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.395 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985911+0000", 2024-09-14T06:17:24.396 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7230000000000001e-06 2024-09-14T06:17:24.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.396 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.396 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.396 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985930+0000", 2024-09-14T06:17:24.396 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9360000000000001e-05 2024-09-14T06:17:24.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.396 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.396 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.396 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985944+0000", 2024-09-14T06:17:24.396 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4479e-05 2024-09-14T06:17:24.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.396 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.396 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.396 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064332+0000", 2024-09-14T06:17:24.397 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078387686999999998 2024-09-14T06:17:24.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.397 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.397 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064340+0000", 2024-09-14T06:17:24.397 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1130000000000001e-06 2024-09-14T06:17:24.397 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.397 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.397 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.397 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11480 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.397 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:24.397 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.388553082000001, 2024-09-14T06:17:24.397 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077122298000000006, 2024-09-14T06:17:24.398 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.398 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.398 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.398 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.398 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.398 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.398 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11480 2024-09-14T06:17:24.398 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.398 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.398 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.398 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.398 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:24.398 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.398 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.398 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.399 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.399 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:24.399 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.399 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.399 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.399 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.399 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987255+0000", 2024-09-14T06:17:24.399 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:17:24.399 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.399 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.399 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.399 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987263+0000", 2024-09-14T06:17:24.399 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6499999999999996e-06 2024-09-14T06:17:24.399 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.399 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.400 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.400 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987285+0000", 2024-09-14T06:17:24.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1860000000000001e-05 2024-09-14T06:17:24.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.400 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.400 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.400 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987341+0000", 2024-09-14T06:17:24.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6648999999999999e-05 2024-09-14T06:17:24.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.400 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.400 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.400 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064367+0000", 2024-09-14T06:17:24.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077025622000000002 2024-09-14T06:17:24.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.401 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.401 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.401 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064376+0000", 2024-09-14T06:17:24.401 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7499999999999992e-06 2024-09-14T06:17:24.401 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.401 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.401 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.401 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.401 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.401 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11481 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.401 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:24.401 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.38841738, 2024-09-14T06:17:24.401 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077022057000000005, 2024-09-14T06:17:24.402 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.402 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.402 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.402 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.402 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.402 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.402 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11481 2024-09-14T06:17:24.402 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.402 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.402 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.402 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.402 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:24.402 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.402 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.402 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.403 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.403 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:24.403 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.403 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.403 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.403 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987391+0000", 2024-09-14T06:17:24.403 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4839999999999999e-06 2024-09-14T06:17:24.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.403 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.403 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.403 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987399+0000", 2024-09-14T06:17:24.403 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8660000000000006e-06 2024-09-14T06:17:24.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.404 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.404 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987419+0000", 2024-09-14T06:17:24.404 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9984e-05 2024-09-14T06:17:24.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.404 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.404 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987434+0000", 2024-09-14T06:17:24.404 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5396e-05 2024-09-14T06:17:24.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.404 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.404 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064403+0000", 2024-09-14T06:17:24.404 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076969212999999995 2024-09-14T06:17:24.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.405 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.405 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.405 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064411+0000", 2024-09-14T06:17:24.405 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1140000000000006e-06 2024-09-14T06:17:24.405 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.405 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.405 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.405 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.405 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.405 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11482 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.405 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:24.405 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.388332519999999, 2024-09-14T06:17:24.405 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076971172000000004, 2024-09-14T06:17:24.405 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.406 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.406 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.406 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.406 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.406 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.406 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11482 2024-09-14T06:17:24.406 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.406 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.406 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.406 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.406 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:24.406 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.406 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.406 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.406 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.406 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:24.407 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.407 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.407 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.407 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987476+0000", 2024-09-14T06:17:24.407 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.793e-06 2024-09-14T06:17:24.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.407 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.407 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.407 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987482+0000", 2024-09-14T06:17:24.407 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8270000000000003e-06 2024-09-14T06:17:24.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.407 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.407 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.408 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987502+0000", 2024-09-14T06:17:24.408 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0656e-05 2024-09-14T06:17:24.408 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.408 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.408 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.408 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987518+0000", 2024-09-14T06:17:24.408 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:17:24.408 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.408 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.408 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.408 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064438+0000", 2024-09-14T06:17:24.408 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076919623000000006 2024-09-14T06:17:24.408 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.409 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.409 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.409 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064445+0000", 2024-09-14T06:17:24.409 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.61e-06 2024-09-14T06:17:24.409 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.409 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.409 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.409 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.409 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.409 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11483 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.409 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:24.409 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.388264671000002, 2024-09-14T06:17:24.409 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076936562999999999, 2024-09-14T06:17:24.409 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.409 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.410 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.410 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.410 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.410 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.410 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11483 2024-09-14T06:17:24.410 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.410 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.410 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.410 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.410 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:24.410 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.410 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.410 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.410 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.411 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:24.411 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.411 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.411 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.411 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.411 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987544+0000", 2024-09-14T06:17:24.411 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.654e-06 2024-09-14T06:17:24.411 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.411 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.411 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.411 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987550+0000", 2024-09-14T06:17:24.411 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0299999999999999e-06 2024-09-14T06:17:24.412 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.412 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.412 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.412 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987570+0000", 2024-09-14T06:17:24.412 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0080000000000001e-05 2024-09-14T06:17:24.412 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.412 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.412 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.412 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987619+0000", 2024-09-14T06:17:24.412 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9228999999999999e-05 2024-09-14T06:17:24.412 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.412 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.412 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.412 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064471+0000", 2024-09-14T06:17:24.413 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076852086999999999 2024-09-14T06:17:24.413 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.413 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.413 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.413 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064478+0000", 2024-09-14T06:17:24.413 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4830000000000002e-06 2024-09-14T06:17:24.413 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.413 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.413 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.413 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.413 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.413 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11485 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.413 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:24.413 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.388166177999999, 2024-09-14T06:17:24.414 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076868463999999997, 2024-09-14T06:17:24.414 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.414 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.414 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.414 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.414 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.414 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.414 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11485 2024-09-14T06:17:24.414 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.414 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.414 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.414 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.414 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:24.414 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987642+0000", 2024-09-14T06:17:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4100000000000001e-06 2024-09-14T06:17:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.415 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987648+0000", 2024-09-14T06:17:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2500000000000003e-06 2024-09-14T06:17:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987683+0000", 2024-09-14T06:17:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5193e-05 2024-09-14T06:17:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987698+0000", 2024-09-14T06:17:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4893e-05 2024-09-14T06:17:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.416 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064501+0000", 2024-09-14T06:17:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076803263999999996 2024-09-14T06:17:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064509+0000", 2024-09-14T06:17:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4540000000000001e-06 2024-09-14T06:17:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11490 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.417 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.388016608999997, 2024-09-14T06:17:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076755043999999994, 2024-09-14T06:17:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11490 2024-09-14T06:17:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.418 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987791+0000", 2024-09-14T06:17:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:17:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.419 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987797+0000", 2024-09-14T06:17:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7969999999999997e-06 2024-09-14T06:17:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987817+0000", 2024-09-14T06:17:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9746e-05 2024-09-14T06:17:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987832+0000", 2024-09-14T06:17:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.452e-05 2024-09-14T06:17:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.420 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064535+0000", 2024-09-14T06:17:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076703478000000005 2024-09-14T06:17:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064545+0000", 2024-09-14T06:17:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9799999999999993e-06 2024-09-14T06:17:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.421 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11492 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.387889743999999, 2024-09-14T06:17:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076661759999999995, 2024-09-14T06:17:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11492 2024-09-14T06:17:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.422 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987918+0000", 2024-09-14T06:17:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3239999999999999e-06 2024-09-14T06:17:24.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987924+0000", 2024-09-14T06:17:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6930000000000004e-06 2024-09-14T06:17:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987943+0000", 2024-09-14T06:17:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9487000000000002e-05 2024-09-14T06:17:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987958+0000", 2024-09-14T06:17:24.424 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4453e-05 2024-09-14T06:17:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064571+0000", 2024-09-14T06:17:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076612988000000007 2024-09-14T06:17:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064579+0000", 2024-09-14T06:17:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8150000000000006e-06 2024-09-14T06:17:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11495 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.387721055, 2024-09-14T06:17:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076524746000000005, 2024-09-14T06:17:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11495 2024-09-14T06:17:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.426 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988087+0000", 2024-09-14T06:17:24.427 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0499999999999999e-06 2024-09-14T06:17:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988118+0000", 2024-09-14T06:17:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1900000000000003e-05 2024-09-14T06:17:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988138+0000", 2024-09-14T06:17:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.933e-05 2024-09-14T06:17:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.428 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988152+0000", 2024-09-14T06:17:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3889e-05 2024-09-14T06:17:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064603+0000", 2024-09-14T06:17:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076451082000000004 2024-09-14T06:17:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064610+0000", 2024-09-14T06:17:24.429 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4950000000000002e-06 2024-09-14T06:17:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11498 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.387634978999998, 2024-09-14T06:17:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076471883000000004, 2024-09-14T06:17:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.430 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11498 2024-09-14T06:17:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.431 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988173+0000", 2024-09-14T06:17:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2300000000000001e-06 2024-09-14T06:17:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988178+0000", 2024-09-14T06:17:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4229999999999996e-06 2024-09-14T06:17:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988235+0000", 2024-09-14T06:17:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6616999999999997e-05 2024-09-14T06:17:24.432 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988264+0000", 2024-09-14T06:17:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9189000000000001e-05 2024-09-14T06:17:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064636+0000", 2024-09-14T06:17:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076371573999999998 2024-09-14T06:17:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.433 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064643+0000", 2024-09-14T06:17:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8499999999999994e-06 2024-09-14T06:17:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11523 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.386914830999999, 2024-09-14T06:17:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076062454000000002, 2024-09-14T06:17:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.434 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.435 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.435 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.435 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11523 2024-09-14T06:17:24.435 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.435 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.435 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.435 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.435 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:24.435 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.435 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.435 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.435 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.435 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:24.436 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.436 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.436 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.436 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.436 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988894+0000", 2024-09-14T06:17:24.436 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.843e-06 2024-09-14T06:17:24.436 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.436 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.436 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.436 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988903+0000", 2024-09-14T06:17:24.436 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0440000000000001e-06 2024-09-14T06:17:24.436 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.436 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.436 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.437 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988929+0000", 2024-09-14T06:17:24.437 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6679e-05 2024-09-14T06:17:24.437 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.437 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.437 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.437 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988967+0000", 2024-09-14T06:17:24.437 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7453000000000002e-05 2024-09-14T06:17:24.437 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.437 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.437 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.437 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064668+0000", 2024-09-14T06:17:24.437 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075701389999999993 2024-09-14T06:17:24.437 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.438 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.438 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.438 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064954+0000", 2024-09-14T06:17:24.438 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028604500000000002 2024-09-14T06:17:24.438 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.438 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.438 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.438 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.438 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.438 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11530 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.438 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:24.438 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.386424280999996, 2024-09-14T06:17:24.438 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075620674999999998, 2024-09-14T06:17:24.439 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.439 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.439 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.439 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.439 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.439 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.439 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11530 2024-09-14T06:17:24.439 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.439 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.439 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.439 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.439 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:24.439 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.439 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.440 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.440 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.440 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:24.440 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.440 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.440 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.440 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.440 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989384+0000", 2024-09-14T06:17:24.440 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6160000000000001e-06 2024-09-14T06:17:24.440 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.440 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.440 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.440 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989390+0000", 2024-09-14T06:17:24.441 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7039999999999999e-06 2024-09-14T06:17:24.441 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.441 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.441 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.441 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989409+0000", 2024-09-14T06:17:24.441 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.95e-05 2024-09-14T06:17:24.441 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.441 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.441 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.441 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989429+0000", 2024-09-14T06:17:24.441 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0173000000000002e-05 2024-09-14T06:17:24.441 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.441 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.441 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.442 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064992+0000", 2024-09-14T06:17:24.442 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075562764000000004 2024-09-14T06:17:24.442 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.442 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.442 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.442 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065003+0000", 2024-09-14T06:17:24.442 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0917999999999999e-05 2024-09-14T06:17:24.442 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.442 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.442 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.442 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.442 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.442 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11532 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.443 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:24.443 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.386353884999998, 2024-09-14T06:17:24.443 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075604656000000006, 2024-09-14T06:17:24.443 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.443 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.443 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.443 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.443 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.443 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.443 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11532 2024-09-14T06:17:24.443 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.443 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.443 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.444 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.444 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:24.444 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.444 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.444 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.444 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.444 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:24.444 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.444 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.444 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.444 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.444 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989454+0000", 2024-09-14T06:17:24.444 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6500000000000001e-06 2024-09-14T06:17:24.444 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.445 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.445 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989460+0000", 2024-09-14T06:17:24.445 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7459999999999997e-06 2024-09-14T06:17:24.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.445 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.445 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989480+0000", 2024-09-14T06:17:24.445 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.948e-05 2024-09-14T06:17:24.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.445 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.445 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989497+0000", 2024-09-14T06:17:24.445 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7153e-05 2024-09-14T06:17:24.446 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.446 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.446 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.446 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065046+0000", 2024-09-14T06:17:24.446 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075548905999999999 2024-09-14T06:17:24.446 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.446 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.446 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.446 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065057+0000", 2024-09-14T06:17:24.446 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1721e-05 2024-09-14T06:17:24.446 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.446 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.446 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.447 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.447 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.447 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11534 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.447 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:24.447 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.386287398999997, 2024-09-14T06:17:24.447 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075574350999999998, 2024-09-14T06:17:24.447 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.447 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.447 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.447 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.447 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.447 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.447 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11534 2024-09-14T06:17:24.447 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.447 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.447 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.448 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.448 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:24.448 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.448 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.448 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.448 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:24.448 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.448 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.448 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.448 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989521+0000", 2024-09-14T06:17:24.448 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.533e-06 2024-09-14T06:17:24.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.449 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.449 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989527+0000", 2024-09-14T06:17:24.449 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0700000000000003e-06 2024-09-14T06:17:24.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.449 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.449 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989546+0000", 2024-09-14T06:17:24.449 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.948e-05 2024-09-14T06:17:24.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.449 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.449 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989561+0000", 2024-09-14T06:17:24.449 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4827000000000001e-05 2024-09-14T06:17:24.450 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.450 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.450 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.450 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065086+0000", 2024-09-14T06:17:24.450 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075524542 2024-09-14T06:17:24.450 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.450 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.450 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.450 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065094+0000", 2024-09-14T06:17:24.450 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8990000000000001e-06 2024-09-14T06:17:24.450 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.450 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.450 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.450 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.451 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.451 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11535 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.451 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:24.451 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.386220035999997, 2024-09-14T06:17:24.451 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075542466000000003, 2024-09-14T06:17:24.451 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.451 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.451 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.451 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.451 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.451 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.451 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11535 2024-09-14T06:17:24.451 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.451 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.452 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.452 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.452 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:24.452 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.452 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.452 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.452 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.452 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:24.452 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.452 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.452 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.452 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.452 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989588+0000", 2024-09-14T06:17:24.452 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1659999999999999e-06 2024-09-14T06:17:24.453 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.453 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.453 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.453 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989593+0000", 2024-09-14T06:17:24.453 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6230000000000002e-06 2024-09-14T06:17:24.453 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.453 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.453 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.453 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989613+0000", 2024-09-14T06:17:24.453 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9604000000000001e-05 2024-09-14T06:17:24.453 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.453 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.453 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.454 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989643+0000", 2024-09-14T06:17:24.454 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0450000000000002e-05 2024-09-14T06:17:24.454 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.454 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.454 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.454 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065119+0000", 2024-09-14T06:17:24.454 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075475930999999996 2024-09-14T06:17:24.454 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.454 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.454 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.454 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065129+0000", 2024-09-14T06:17:24.454 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6919999999999997e-06 2024-09-14T06:17:24.454 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.454 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.455 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.455 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.455 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.455 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11537 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.455 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:24.455 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.386142112999998, 2024-09-14T06:17:24.455 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075497868999999995, 2024-09-14T06:17:24.455 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.455 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.455 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.455 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.455 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.455 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.455 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11537 2024-09-14T06:17:24.455 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.456 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.456 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.456 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.456 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:24.456 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.456 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.456 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.456 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.456 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:24.456 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.456 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.456 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.456 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.456 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989666+0000", 2024-09-14T06:17:24.457 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.48e-06 2024-09-14T06:17:24.457 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.457 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.457 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.457 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989672+0000", 2024-09-14T06:17:24.457 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7960000000000001e-06 2024-09-14T06:17:24.457 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.457 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.457 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.457 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989690+0000", 2024-09-14T06:17:24.457 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8743e-05 2024-09-14T06:17:24.457 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.457 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.457 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989704+0000", 2024-09-14T06:17:24.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3913e-05 2024-09-14T06:17:24.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.458 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.458 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065155+0000", 2024-09-14T06:17:24.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075450351999999998 2024-09-14T06:17:24.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.458 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.458 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065162+0000", 2024-09-14T06:17:24.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5850000000000002e-06 2024-09-14T06:17:24.458 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.458 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.458 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.459 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.459 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.459 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11543 3.12 3.4d02a8b2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:24.459 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:24.459 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 48.385864548999997, 2024-09-14T06:17:24.459 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074335152000000002, 2024-09-14T06:17:24.459 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:24.459 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:24.459 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:24.459 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:24.459 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:24.459 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:24.459 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11543 2024-09-14T06:17:24.459 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.460 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:24.460 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.460 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:24.460 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:24.460 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.460 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.460 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.460 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:24.460 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:24.460 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:24.460 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.460 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.460 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:24.460 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989944+0000", 2024-09-14T06:17:24.460 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.694e-06 2024-09-14T06:17:24.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:24.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989950+0000", 2024-09-14T06:17:24.461 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3389999999999998e-06 2024-09-14T06:17:24.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:24.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989969+0000", 2024-09-14T06:17:24.461 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8963000000000001e-05 2024-09-14T06:17:24.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:24.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989985+0000", 2024-09-14T06:17:24.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5586999999999998e-05 2024-09-14T06:17:24.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.462 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:24.462 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064270+0000", 2024-09-14T06:17:24.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074285552000000005 2024-09-14T06:17:24.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:24.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:24.462 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:24.462 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064277+0000", 2024-09-14T06:17:24.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0169999999999999e-06 2024-09-14T06:17:24.462 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.462 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.462 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.462 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:24.463 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:24.463 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:24.464 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:17:24.464 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:24.462+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:24.502 INFO:teuthology.orchestra.run.smithi039.stdout:moved 0 / 82 (0%) 2024-09-14T06:17:24.502 INFO:teuthology.orchestra.run.smithi039.stdout:avg 10.25 2024-09-14T06:17:24.503 INFO:teuthology.orchestra.run.smithi039.stdout:stddev 2.94746 -> 2.94746 (expected baseline 2.99479) 2024-09-14T06:17:24.503 INFO:teuthology.orchestra.run.smithi039.stdout:min osd.3 with 4 -> 4 pgs (0.390244 -> 0.390244 * mean) 2024-09-14T06:17:24.503 INFO:teuthology.orchestra.run.smithi039.stdout:max osd.0 with 13 -> 13 pgs (1.26829 -> 1.26829 * mean) 2024-09-14T06:17:24.503 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:24.503 INFO:teuthology.orchestra.run.smithi039.stdout:oload 1000 2024-09-14T06:17:24.503 INFO:teuthology.orchestra.run.smithi039.stdout:max_change 3 2024-09-14T06:17:24.503 INFO:teuthology.orchestra.run.smithi039.stdout:max_change_osds 4 2024-09-14T06:17:24.503 INFO:teuthology.orchestra.run.smithi039.stdout:average_utilization 0.0005 2024-09-14T06:17:24.503 INFO:teuthology.orchestra.run.smithi039.stdout:overload_utilization 0.0049 2024-09-14T06:17:24.503 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 weight 0.1130 -> 0.1141 2024-09-14T06:17:24.531 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-14T06:17:24.531 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-14T06:17:24.565 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:24.561+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:24.625 INFO:teuthology.orchestra.run.smithi039.stdout:truncating image image_client.0-clone27 from 0xe29fee3 (overlap 0x30ea61d) to 0xeacb6f 2024-09-14T06:17:24.652 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:24.652 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:24.652 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:24.652 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:24.666 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:24.662+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:24.677 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:17:24.753 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:17:24.767 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:24.765+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:24.776 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-14T06:17:24.786 INFO:teuthology.orchestra.run.smithi039.stdout:4368 write 0x23d86b thru 0x24b8be (0xe054 bytes) 2024-09-14T06:17:24.845 INFO:teuthology.orchestra.run.smithi039.stdout:4369 read 0xc381776 thru 0xc38ca1a (0xb2a5 bytes) 2024-09-14T06:17:24.914 INFO:teuthology.orchestra.run.smithi039.stdout:4370 read 0x66032e0 thru 0x6609baa (0x68cb bytes) 2024-09-14T06:17:24.914 INFO:teuthology.orchestra.run.smithi039.stdout:4371 write 0x43a4269 thru 0x43a4815 (0x5ad bytes) 2024-09-14T06:17:24.916 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:24.916 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:24.916 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:24.916 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:24.916 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:24.917 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:17:24.918 INFO:teuthology.orchestra.run.smithi039.stdout:4372 punch from 0xb7bd63e to 0xb7c6e61, (0x9823 bytes) 2024-09-14T06:17:24.918 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:24.918 INFO:teuthology.orchestra.run.smithi039.stdout:4374 read 0x716464 thru 0x71b6c6 (0x5263 bytes) 2024-09-14T06:17:24.918 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:24.918 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:24.918 INFO:teuthology.orchestra.run.smithi039.stdout:4378 write 0x369dec8 thru 0x36ad652 (0xf78b bytes) 2024-09-14T06:17:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:4380 write 0x14e3917 thru 0x14ee42e (0xab18 bytes) 2024-09-14T06:17:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:4381 writesame 0xb5c6f38 thru 0xb5cc72f data_size 0x14(0x57f8 bytes) 2024-09-14T06:17:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:4382 writesame 0x7fc979e thru 0x7fd0107 data_size 0x2(0x696a bytes) 2024-09-14T06:17:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:4384 read 0x1cbdae9 thru 0x1cc5caf (0x81c7 bytes) 2024-09-14T06:17:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:4385 punch from 0x1eab32b to 0x1eb29fe, (0x76d3 bytes) 2024-09-14T06:17:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:4386 writesame 0x1691406 thru 0x169a13e data_size 0x27(0x8d39 bytes) 2024-09-14T06:17:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:4387 punch from 0xa5857c1 to 0xa58bc3b, (0x647a bytes) 2024-09-14T06:17:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:4388 punch from 0x621c829 to 0x622848e, (0xbc65 bytes) 2024-09-14T06:17:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:4389 write 0x975d92e thru 0x97629e5 (0x50b8 bytes) 2024-09-14T06:17:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:4390 read 0xe1c17e3 thru 0xe1c277d (0xf9b bytes) 2024-09-14T06:17:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:4393 writesame 0xad11434 thru 0xad21314 data_size 0x1(0xfee1 bytes) 2024-09-14T06:17:24.920 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:24.920 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:24.920 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:24.914+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:24.948 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:24.949 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:24.950 INFO:teuthology.orchestra.run.smithi039.stdout:4394 punch from 0x5c5a2ed to 0x5c5c223, (0x1f36 bytes) 2024-09-14T06:17:24.950 INFO:teuthology.orchestra.run.smithi039.stdout:4395 punch from 0xa4b4fe1 to 0xa4bbaad, (0x6acc bytes) 2024-09-14T06:17:24.950 INFO:teuthology.orchestra.run.smithi039.stdout:4398 write 0x7e47f0 thru 0x7ec39c (0x7bad bytes) 2024-09-14T06:17:24.966 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-14T06:17:24.962+0000 7f4add550640 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-14T06:17:24.966 INFO:teuthology.orchestra.run.smithi039.stdout:moved 0 / 82 (0%) 2024-09-14T06:17:24.966 INFO:teuthology.orchestra.run.smithi039.stdout:avg 10.25 2024-09-14T06:17:24.966 INFO:teuthology.orchestra.run.smithi039.stdout:stddev 2.94746 -> 2.94746 (expected baseline 2.99479) 2024-09-14T06:17:24.966 INFO:teuthology.orchestra.run.smithi039.stdout:min osd.3 with 4 -> 4 pgs (0.390244 -> 0.390244 * mean) 2024-09-14T06:17:24.967 INFO:teuthology.orchestra.run.smithi039.stdout:max osd.0 with 13 -> 13 pgs (1.26829 -> 1.26829 * mean) 2024-09-14T06:17:24.967 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:24.967 INFO:teuthology.orchestra.run.smithi039.stdout:oload 1000 2024-09-14T06:17:24.967 INFO:teuthology.orchestra.run.smithi039.stdout:max_change 1 2024-09-14T06:17:24.967 INFO:teuthology.orchestra.run.smithi039.stdout:max_change_osds 4 2024-09-14T06:17:24.967 INFO:teuthology.orchestra.run.smithi039.stdout:average_utilization 0.0005 2024-09-14T06:17:24.967 INFO:teuthology.orchestra.run.smithi039.stdout:overload_utilization 0.0049 2024-09-14T06:17:24.967 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 weight 0.1141 -> 0.1151 2024-09-14T06:17:24.967 INFO:teuthology.orchestra.run.smithi039.stderr:no change 2024-09-14T06:17:24.976 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:24.977 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:24.981 INFO:teuthology.orchestra.run.smithi039.stdout:4399 read 0xe0dcc49 thru 0xe0e46e9 (0x7aa1 bytes) 2024-09-14T06:17:24.982 INFO:teuthology.orchestra.run.smithi039.stdout:4400 read 0xd3717c1 thru 0xd37eaf2 (0xd332 bytes) 2024-09-14T06:17:24.982 INFO:teuthology.orchestra.run.smithi039.stdout:4401 read 0x3eb675c thru 0x3ec1f56 (0xb7fb bytes) 2024-09-14T06:17:24.983 INFO:teuthology.orchestra.run.smithi039.stdout:4402 read 0xd7718d5 thru 0xd778007 (0x6733 bytes) 2024-09-14T06:17:24.983 INFO:teuthology.orchestra.run.smithi039.stdout:4403 writesame 0x5d1d054 thru 0x5d2c3c0 data_size 0x1(0xf36d bytes) 2024-09-14T06:17:24.995 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-14T06:17:24.995 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-14T06:17:24.996 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:24.997 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:25.016 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:25.013+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:25.026 INFO:teuthology.orchestra.run.smithi039.stdout:4404 writesame 0x96ecd9 thru 0x971534 data_size 0x52(0x285c bytes) 2024-09-14T06:17:25.027 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:25.028 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:25.055 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:25.056 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:25.076 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:25.076 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:25.092 INFO:teuthology.orchestra.run.smithi039.stdout:4405 trunc from 0xe29fee3 to 0xd0a64b 2024-09-14T06:17:25.099 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:25.099 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:25.100 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:25.100 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:25.117 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:25.113+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:25.118 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:17:25.128 INFO:teuthology.orchestra.run.smithi039.stdout:4406 trunc from 0xd0a64b to 0x6322453 2024-09-14T06:17:25.134 INFO:teuthology.orchestra.run.smithi039.stdout:4407 write 0x2dc1efc thru 0x2dc8692 (0x6797 bytes) 2024-09-14T06:17:25.137 INFO:teuthology.orchestra.run.smithi039.stdout:4408 punch from 0x140d89a to 0x141a8ea, (0xd050 bytes) 2024-09-14T06:17:25.137 INFO:teuthology.orchestra.run.smithi039.stdout:4409 write 0xe00ef6 thru 0xe044aa (0x35b5 bytes) 2024-09-14T06:17:25.139 INFO:teuthology.orchestra.run.smithi039.stdout:4410 read 0xa011f0 thru 0xa04d69 (0x3b7a bytes) 2024-09-14T06:17:25.140 INFO:teuthology.orchestra.run.smithi039.stdout:4411 write 0x12f674 thru 0x136fb2 (0x793f bytes) 2024-09-14T06:17:25.142 INFO:teuthology.orchestra.run.smithi039.stdout:4412 flatten 2024-09-14T06:17:25.170 INFO:teuthology.orchestra.run.smithi039.stdout:4414 write 0xd833c5f thru 0xd840a00 (0xcda2 bytes) 2024-09-14T06:17:25.174 INFO:teuthology.orchestra.run.smithi039.stdout:4416 write 0xb4160e9 thru 0xb41b525 (0x543d bytes) 2024-09-14T06:17:25.176 INFO:teuthology.orchestra.run.smithi039.stdout:4417 read 0xc5e674d thru 0xc5ef989 (0x923d bytes) 2024-09-14T06:17:25.177 INFO:teuthology.orchestra.run.smithi039.stdout:4418 clone 29 order 21 su 131072 sc 3 2024-09-14T06:17:25.218 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:25.214+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:25.244 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:25.244 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:25.259 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:17:25.319 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-14T06:17:25.314+0000 7f4add550640 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-14T06:17:25.319 INFO:teuthology.orchestra.run.smithi039.stdout:moved 0 / 82 (0%) 2024-09-14T06:17:25.319 INFO:teuthology.orchestra.run.smithi039.stdout:avg 10.25 2024-09-14T06:17:25.319 INFO:teuthology.orchestra.run.smithi039.stdout:stddev 2.94746 -> 2.94746 (expected baseline 2.99479) 2024-09-14T06:17:25.320 INFO:teuthology.orchestra.run.smithi039.stdout:min osd.3 with 4 -> 4 pgs (0.390244 -> 0.390244 * mean) 2024-09-14T06:17:25.320 INFO:teuthology.orchestra.run.smithi039.stdout:max osd.0 with 13 -> 13 pgs (1.26829 -> 1.26829 * mean) 2024-09-14T06:17:25.320 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:25.320 INFO:teuthology.orchestra.run.smithi039.stdout:oload 1000 2024-09-14T06:17:25.320 INFO:teuthology.orchestra.run.smithi039.stdout:max_change 1 2024-09-14T06:17:25.321 INFO:teuthology.orchestra.run.smithi039.stdout:max_change_osds 4 2024-09-14T06:17:25.321 INFO:teuthology.orchestra.run.smithi039.stdout:average_utilization 0.0005 2024-09-14T06:17:25.321 INFO:teuthology.orchestra.run.smithi039.stdout:overload_utilization 0.0049 2024-09-14T06:17:25.321 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 weight 0.1141 -> 0.1151 2024-09-14T06:17:25.321 INFO:teuthology.orchestra.run.smithi039.stderr:no change 2024-09-14T06:17:25.322 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:25.317+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:25.420 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:25.417+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:25.431 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:25.431 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:25.446 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:17:25.519 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:17:25.520 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:25.518+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:25.576 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:25.576 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:25.594 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:17:25.622 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:25.621+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:25.723 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:25.721+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:25.731 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:25.731 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:25.749 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:17:25.819 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-14T06:17:25.824 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:25.821+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:25.882 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:25.882 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:25.899 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:17:25.919 INFO:teuthology.orchestra.run.smithi039.stdout:4422 read 0x561f4ee thru 0x562682b (0x733e bytes) 2024-09-14T06:17:25.924 INFO:teuthology.orchestra.run.smithi039.stdout:4423 writesame 0x22fc139 thru 0x2308f34 data_size 0x2(0xcdfc bytes) 2024-09-14T06:17:25.924 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:25.922+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:25.940 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:25.940 INFO:teuthology.orchestra.run.smithi039.stdout:4426 punch from 0x7eb9f5c to 0x7ec59b8, (0xba5c bytes) 2024-09-14T06:17:25.940 INFO:teuthology.orchestra.run.smithi039.stdout:4427 read 0x92d3be1 thru 0x92d59ff (0x1e1f bytes) 2024-09-14T06:17:25.940 INFO:teuthology.orchestra.run.smithi039.stdout:4428 punch from 0x88d0cb7 to 0x88d7b8e, (0x6ed7 bytes) 2024-09-14T06:17:25.940 INFO:teuthology.orchestra.run.smithi039.stdout:4429 write 0x4e7ce86 thru 0x4e80f81 (0x40fc bytes) 2024-09-14T06:17:25.960 INFO:teuthology.orchestra.run.smithi039.stdout:4430 trunc from 0xd840a01 to 0x542f960 2024-09-14T06:17:25.983 INFO:teuthology.orchestra.run.smithi039.stdout:4432 trunc from 0x542f960 to 0xa2c1f07 2024-09-14T06:17:25.991 INFO:teuthology.orchestra.run.smithi039.stdout:4434 trunc from 0xa2c1f07 to 0xb2bef06 2024-09-14T06:17:26.000 INFO:teuthology.orchestra.run.smithi039.stdout:4435 trunc from 0xb2bef06 to 0x919307e 2024-09-14T06:17:26.007 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:26.010 INFO:teuthology.orchestra.run.smithi039.stdout:4439 trunc from 0x919307e to 0x7045abb 2024-09-14T06:17:26.018 INFO:teuthology.orchestra.run.smithi039.stdout:4440 punch from 0x4f340af to 0x4f42c95, (0xebe6 bytes) 2024-09-14T06:17:26.026 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:26.025+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:26.031 INFO:teuthology.orchestra.run.smithi039.stdout:4441 write 0x9e1853f thru 0x9e1fdaa (0x786c bytes) 2024-09-14T06:17:26.036 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:26.037 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:26.043 INFO:teuthology.orchestra.run.smithi039.stdout:4442 trunc from 0x9e1fdab to 0xc9804e2 2024-09-14T06:17:26.053 INFO:teuthology.orchestra.run.smithi039.stdout:4443 punch from 0x2e210 to 0x2e847, (0x637 bytes) 2024-09-14T06:17:26.053 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:26.056 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:17:26.062 INFO:teuthology.orchestra.run.smithi039.stdout:4445 writesame 0xd9bb117 thru 0xd9bc81c data_size 0x7(0x1706 bytes) 2024-09-14T06:17:26.062 INFO:teuthology.orchestra.run.smithi039.stdout:4446 read 0x2fc03ec thru 0x2fc2eaa (0x2abf bytes) 2024-09-14T06:17:26.082 INFO:teuthology.orchestra.run.smithi039.stdout:4447 trunc from 0xd9bc81d to 0x22b8ca5 2024-09-14T06:17:26.104 INFO:teuthology.orchestra.run.smithi039.stdout:4448 punch from 0x18f520 to 0x19aba6, (0xb686 bytes) 2024-09-14T06:17:26.126 INFO:teuthology.orchestra.run.smithi039.stdout:4449 write 0xcf8d466 thru 0xcf98674 (0xb20f bytes) 2024-09-14T06:17:26.127 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:26.121+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:26.131 INFO:teuthology.orchestra.run.smithi039.stdout:4450 write 0x64ee701 thru 0x64f8cf3 (0xa5f3 bytes) 2024-09-14T06:17:26.136 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:26.136 INFO:teuthology.orchestra.run.smithi039.stdout:4454 write 0x79361d4 thru 0x7943a6b (0xd898 bytes) 2024-09-14T06:17:26.143 INFO:teuthology.orchestra.run.smithi039.stdout:4455 write 0x8fba528 thru 0x8fc3de1 (0x98ba bytes) 2024-09-14T06:17:26.150 INFO:teuthology.orchestra.run.smithi039.stdout:4456 read 0x535833f thru 0x5365d4d (0xda0f bytes) 2024-09-14T06:17:26.150 INFO:teuthology.orchestra.run.smithi039.stdout:4457 writesame 0xb6f0424 thru 0xb6ff77c data_size 0x1(0xf359 bytes) 2024-09-14T06:17:26.193 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:26.193 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:26.199 INFO:teuthology.orchestra.run.smithi039.stdout:4460 write 0x7f65bdf thru 0x7f73aae (0xded0 bytes) 2024-09-14T06:17:26.206 INFO:teuthology.orchestra.run.smithi039.stdout:4461 read 0x7097667 thru 0x7098ff1 (0x198b bytes) 2024-09-14T06:17:26.206 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:26.206 INFO:teuthology.orchestra.run.smithi039.stdout:4463 punch from 0x93b3f63 to 0x93c1f3a, (0xdfd7 bytes) 2024-09-14T06:17:26.206 INFO:teuthology.orchestra.run.smithi039.stdout:4464 punch from 0x1b4897 to 0x1b5532, (0xc9b bytes) 2024-09-14T06:17:26.206 INFO:teuthology.orchestra.run.smithi039.stdout:4465 read 0xb192665 thru 0xb193363 (0xcff bytes) 2024-09-14T06:17:26.206 INFO:teuthology.orchestra.run.smithi039.stdout:4466 punch from 0x5585ff6 to 0x558e5c6, (0x85d0 bytes) 2024-09-14T06:17:26.210 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:17:26.218 INFO:teuthology.orchestra.run.smithi039.stdout:4467 writesame 0xe0b5c67 thru 0xe0b5de8 data_size 0x1(0x182 bytes) 2024-09-14T06:17:26.219 INFO:teuthology.orchestra.run.smithi039.stdout:4468 read 0x75a0ad7 thru 0x75ac06b (0xb595 bytes) 2024-09-14T06:17:26.219 INFO:teuthology.orchestra.run.smithi039.stdout:4470 read 0x32389d2 thru 0x3242e6c (0xa49b bytes) 2024-09-14T06:17:26.219 INFO:teuthology.orchestra.run.smithi039.stdout:4472 read 0x9805a98 thru 0x980f529 (0x9a92 bytes) 2024-09-14T06:17:26.219 INFO:teuthology.orchestra.run.smithi039.stdout:4473 read 0xd12c49b thru 0xd1353e4 (0x8f4a bytes) 2024-09-14T06:17:26.219 INFO:teuthology.orchestra.run.smithi039.stdout:4474 write 0x5245449 thru 0x52525ee (0xd1a6 bytes) 2024-09-14T06:17:26.224 INFO:teuthology.orchestra.run.smithi039.stdout:4475 trunc from 0xe0b5de9 to 0xe410d67 2024-09-14T06:17:26.227 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:26.225+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:26.232 INFO:teuthology.orchestra.run.smithi039.stdout:4477 write 0xd6aa43a thru 0xd6b9d1c (0xf8e3 bytes) 2024-09-14T06:17:26.236 INFO:teuthology.orchestra.run.smithi039.stdout:4478 read 0xb02c082 thru 0xb03affd (0xef7c bytes) 2024-09-14T06:17:26.237 INFO:teuthology.orchestra.run.smithi039.stdout:4479 read 0x2b9b36f thru 0x2ba18d8 (0x656a bytes) 2024-09-14T06:17:26.237 INFO:teuthology.orchestra.run.smithi039.stdout:4481 read 0x59b4eca thru 0x59c0ad5 (0xbc0c bytes) 2024-09-14T06:17:26.237 INFO:teuthology.orchestra.run.smithi039.stdout:4487 read 0x38251d5 thru 0x3831157 (0xbf83 bytes) 2024-09-14T06:17:26.237 INFO:teuthology.orchestra.run.smithi039.stdout:4488 write 0x3a12a2 thru 0x3a28e6 (0x1645 bytes) 2024-09-14T06:17:26.256 INFO:teuthology.orchestra.run.smithi039.stdout:4489 read 0x73950dd thru 0x7395c9c (0xbc0 bytes) 2024-09-14T06:17:26.256 INFO:teuthology.orchestra.run.smithi039.stdout:4490 punch from 0x6236a24 to 0x6236ba2, (0x17e bytes) 2024-09-14T06:17:26.256 INFO:teuthology.orchestra.run.smithi039.stdout:4491 write 0x9b3aa08 thru 0x9b47a00 (0xcff9 bytes) 2024-09-14T06:17:26.263 INFO:teuthology.orchestra.run.smithi039.stdout:4492 write 0x1ab41ba thru 0x1ac14a7 (0xd2ee bytes) 2024-09-14T06:17:26.273 INFO:teuthology.orchestra.run.smithi039.stdout:4493 read 0x86a3d99 thru 0x86a5dff (0x2067 bytes) 2024-09-14T06:17:26.273 INFO:teuthology.orchestra.run.smithi039.stdout:4494 read 0x9d5f96d thru 0x9d6f25f (0xf8f3 bytes) 2024-09-14T06:17:26.274 INFO:teuthology.orchestra.run.smithi039.stdout:4495 write 0xd122d8f thru 0xd1297e3 (0x6a55 bytes) 2024-09-14T06:17:26.277 INFO:teuthology.orchestra.run.smithi039.stdout:4496 read 0xfa9622 thru 0xfaebaa (0x5589 bytes) 2024-09-14T06:17:26.279 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:26.279 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:26.279 INFO:teuthology.orchestra.run.smithi039.stdout:4500 read 0x6404861 thru 0x640c5d8 (0x7d78 bytes) 2024-09-14T06:17:26.279 INFO:teuthology.orchestra.run.smithi039.stdout:4501 punch from 0x1f25f03 to 0x1f2c4e3, (0x65e0 bytes) 2024-09-14T06:17:26.279 INFO:teuthology.orchestra.run.smithi039.stdout:4502 read 0xa4410ab thru 0xa444179 (0x30cf bytes) 2024-09-14T06:17:26.279 INFO:teuthology.orchestra.run.smithi039.stdout:4503 writesame 0xabfb929 thru 0xac069dd data_size 0x11(0xb0b5 bytes) 2024-09-14T06:17:26.287 INFO:teuthology.orchestra.run.smithi039.stdout:4504 writesame 0x2728506 thru 0x272b5b5 data_size 0x4c(0x30b0 bytes) 2024-09-14T06:17:26.292 INFO:teuthology.orchestra.run.smithi039.stdout:4505 writesame 0x298ee4e thru 0x2998d8f data_size 0x87(0x9f42 bytes) 2024-09-14T06:17:26.327 INFO:teuthology.orchestra.run.smithi039.stdout:4509 trunc from 0xe410d67 to 0x102564f 2024-09-14T06:17:26.327 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:26.325+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:26.361 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:26.361 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:26.364 INFO:teuthology.orchestra.run.smithi039.stdout:4510 write 0x41246 thru 0x41cc1 (0xa7c bytes) 2024-09-14T06:17:26.367 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:26.367 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:26.367 INFO:teuthology.orchestra.run.smithi039.stdout:4514 read 0xf3e112 thru 0xf4558b (0x747a bytes) 2024-09-14T06:17:26.380 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:17:26.381 INFO:teuthology.orchestra.run.smithi039.stdout:4515 writesame 0x3470ee8 thru 0x3471bdc data_size 0x1(0xcf5 bytes) 2024-09-14T06:17:26.400 INFO:teuthology.orchestra.run.smithi039.stdout:4519 write 0x9714258 thru 0x971a113 (0x5ebc bytes) 2024-09-14T06:17:26.405 INFO:teuthology.orchestra.run.smithi039.stdout:4520 write 0x236bcce thru 0x23763bb (0xa6ee bytes) 2024-09-14T06:17:26.409 INFO:teuthology.orchestra.run.smithi039.stdout:4521 punch from 0x65791cf to 0x6582d8a, (0x9bbb bytes) 2024-09-14T06:17:26.421 INFO:teuthology.orchestra.run.smithi039.stdout:4522 writesame 0xbc0432b thru 0xbc0ddd0 data_size 0x6b(0x9aa6 bytes) 2024-09-14T06:17:26.421 INFO:teuthology.orchestra.run.smithi039.stdout:4523 punch from 0x18ccd99 to 0x18dab37, (0xdd9e bytes) 2024-09-14T06:17:26.428 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:26.425+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:26.428 INFO:teuthology.orchestra.run.smithi039.stdout:4524 trunc from 0xbc0ddd1 to 0x8af3cd3 2024-09-14T06:17:26.441 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:26.441 INFO:teuthology.orchestra.run.smithi039.stdout:4528 writesame 0x2cffd74 thru 0x2d0806f data_size 0x4(0x82fc bytes) 2024-09-14T06:17:26.457 INFO:teuthology.orchestra.run.smithi039.stdout:4530 read 0x3643801 thru 0x3651d96 (0xe596 bytes) 2024-09-14T06:17:26.457 INFO:teuthology.orchestra.run.smithi039.stdout:4531 writesame 0xa516b0 thru 0xa5a7ae data_size 0x1(0x90ff bytes) 2024-09-14T06:17:26.515 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:26.515 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:26.527 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-09-14T06:17:26.529 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:26.525+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:26.534 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:17:26.534 INFO:teuthology.orchestra.run.smithi039.stdout:4532 read 0x72d7aed thru 0x72dc1e9 (0x46fd bytes) 2024-09-14T06:17:26.534 INFO:teuthology.orchestra.run.smithi039.stdout:4533 punch from 0x4e165ba to 0x4e2382e, (0xd274 bytes) 2024-09-14T06:17:26.547 INFO:teuthology.orchestra.run.smithi039.stdout:4535 trunc from 0x8af3cd3 to 0xe9f22a 2024-09-14T06:17:26.587 INFO:teuthology.orchestra.run.smithi039.stdout:4536 writesame 0x6f311de thru 0x6f377e6 data_size 0x1(0x6609 bytes) 2024-09-14T06:17:26.588 INFO:teuthology.orchestra.run.smithi039.stdout:4537 write 0x353e3c8 thru 0x3541878 (0x34b1 bytes) 2024-09-14T06:17:26.593 INFO:teuthology.orchestra.run.smithi039.stdout:4538 read 0x2a27ae4 thru 0x2a3504a (0xd567 bytes) 2024-09-14T06:17:26.593 INFO:teuthology.orchestra.run.smithi039.stdout:4539 read 0x4d83d19 thru 0x4d84367 (0x64f bytes) 2024-09-14T06:17:26.593 INFO:teuthology.orchestra.run.smithi039.stdout:4540 read 0x2ffb989 thru 0x300b0a8 (0xf720 bytes) 2024-09-14T06:17:26.594 INFO:teuthology.orchestra.run.smithi039.stdout:4542 writesame 0x58486fd thru 0x584fac1 data_size 0x25(0x73c5 bytes) 2024-09-14T06:17:26.599 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:26.599 INFO:teuthology.orchestra.run.smithi039.stdout:4544 read 0x294fa4 thru 0x299000 (0x405d bytes) 2024-09-14T06:17:26.600 INFO:teuthology.orchestra.run.smithi039.stdout:4545 write 0x156cbba thru 0x1572af6 (0x5f3d bytes) 2024-09-14T06:17:26.604 INFO:teuthology.orchestra.run.smithi039.stdout:4546 writesame 0x1ae8f30 thru 0x1aed99c data_size 0x1d(0x4a6d bytes) 2024-09-14T06:17:26.610 INFO:teuthology.orchestra.run.smithi039.stdout:4547 punch from 0x63a250a to 0x63a97df, (0x72d5 bytes) 2024-09-14T06:17:26.626 INFO:teuthology.orchestra.run.smithi039.stdout:4548 write 0xda3ec4b thru 0xda4a944 (0xbcfa bytes) 2024-09-14T06:17:26.630 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:26.626+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:26.632 INFO:teuthology.orchestra.run.smithi039.stdout:4549 write 0xbda4d4e thru 0xbdb2222 (0xd4d5 bytes) 2024-09-14T06:17:26.637 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:26.637 INFO:teuthology.orchestra.run.smithi039.stdout:4551 write 0x88a5184 thru 0x88a777c (0x25f9 bytes) 2024-09-14T06:17:26.643 INFO:teuthology.orchestra.run.smithi039.stdout:4552 punch from 0x9eeb51c to 0x9eefa79, (0x455d bytes) 2024-09-14T06:17:26.643 INFO:teuthology.orchestra.run.smithi039.stdout:4553 writesame 0x21efdf0 thru 0x21f022a data_size 0x3(0x43b bytes) 2024-09-14T06:17:26.658 INFO:teuthology.orchestra.run.smithi039.stdout:4554 trunc from 0xda4a945 to 0x7034890 2024-09-14T06:17:26.672 INFO:teuthology.orchestra.run.smithi039.stdout:4556 write 0x5863cd0 thru 0x5866e84 (0x31b5 bytes) 2024-09-14T06:17:26.679 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:26.680 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:26.696 INFO:teuthology.orchestra.run.smithi039.stdout:4558 writesame 0xda29189 thru 0xda2d410 data_size 0x4(0x4288 bytes) 2024-09-14T06:17:26.696 INFO:teuthology.orchestra.run.smithi039.stdout:4559 write 0x94d7f27 thru 0x94d973d (0x1817 bytes) 2024-09-14T06:17:26.700 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:17:26.702 INFO:teuthology.orchestra.run.smithi039.stdout:4560 write 0x6b3ae88 thru 0x6b3d3dd (0x2556 bytes) 2024-09-14T06:17:26.712 INFO:teuthology.orchestra.run.smithi039.stdout:4561 write 0xda6615c thru 0xda6eed0 (0x8d75 bytes) 2024-09-14T06:17:26.717 INFO:teuthology.orchestra.run.smithi039.stdout:4562 write 0xcc1dd1 thru 0xcc23f4 (0x624 bytes) 2024-09-14T06:17:26.721 INFO:teuthology.orchestra.run.smithi039.stdout:4563 writesame 0x7ba7da4 thru 0x7ba890d data_size 0x3(0xb6a bytes) 2024-09-14T06:17:26.724 INFO:teuthology.orchestra.run.smithi039.stdout:4564 writesame 0x35abe31 thru 0x35ba1be data_size 0x92(0xe38e bytes) 2024-09-14T06:17:26.731 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:26.726+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:26.748 INFO:teuthology.orchestra.run.smithi039.stdout:4565 trunc from 0xda6eed1 to 0x892ebb 2024-09-14T06:17:26.799 INFO:teuthology.orchestra.run.smithi039.stdout:4566 writesame 0x8cea807 thru 0x8cf6b4c data_size 0x5(0xc346 bytes) 2024-09-14T06:17:26.800 INFO:teuthology.orchestra.run.smithi039.stdout:4567 punch from 0x66168d9 to 0x6618038, (0x175f bytes) 2024-09-14T06:17:26.800 INFO:teuthology.orchestra.run.smithi039.stdout:4568 punch from 0x2f65e1d to 0x2f7405e, (0xe241 bytes) 2024-09-14T06:17:26.800 INFO:teuthology.orchestra.run.smithi039.stdout:4569 read 0x7ecd5f8 thru 0x7ed9ddd (0xc7e6 bytes) 2024-09-14T06:17:26.808 INFO:teuthology.orchestra.run.smithi039.stdout:4570 write 0xa25124d thru 0xa254a29 (0x37dd bytes) 2024-09-14T06:17:26.813 INFO:teuthology.orchestra.run.smithi039.stdout:4572 read 0x58fff69 thru 0x590355a (0x35f2 bytes) 2024-09-14T06:17:26.813 INFO:teuthology.orchestra.run.smithi039.stdout:4576 writesame 0x26222c7 thru 0x262d222 data_size 0x9(0xaf5c bytes) 2024-09-14T06:17:26.823 INFO:teuthology.orchestra.run.smithi039.stdout:4578 punch from 0x4dacb37 to 0x4db7c0d, (0xb0d6 bytes) 2024-09-14T06:17:26.823 INFO:teuthology.orchestra.run.smithi039.stdout:4581 punch from 0x3fdfb25 to 0x3fe03ad, (0x888 bytes) 2024-09-14T06:17:26.823 INFO:teuthology.orchestra.run.smithi039.stdout:4582 punch from 0x344146e to 0x344c284, (0xae16 bytes) 2024-09-14T06:17:26.823 INFO:teuthology.orchestra.run.smithi039.stdout:4586 read 0x5d111c2 thru 0x5d19b28 (0x8967 bytes) 2024-09-14T06:17:26.824 INFO:teuthology.orchestra.run.smithi039.stdout:4587 punch from 0x5af1006 to 0x5af3172, (0x216c bytes) 2024-09-14T06:17:26.824 INFO:teuthology.orchestra.run.smithi039.stdout:4588 read 0x42440ae thru 0x4252524 (0xe477 bytes) 2024-09-14T06:17:26.824 INFO:teuthology.orchestra.run.smithi039.stdout:4589 write 0x6b1e3ba thru 0x6b290af (0xacf6 bytes) 2024-09-14T06:17:26.830 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:26.830 INFO:teuthology.orchestra.run.smithi039.stdout:4596 write 0xb7b5a8 thru 0xb8a1fc (0xec55 bytes) 2024-09-14T06:17:26.831 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:26.830+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:26.833 INFO:teuthology.orchestra.run.smithi039.stdout:4597 writesame 0x4eacd3 thru 0x4edfa7 data_size 0x4d(0x32d5 bytes) 2024-09-14T06:17:26.849 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:26.849 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:26.862 INFO:teuthology.orchestra.run.smithi039.stdout:4601 write 0xb11f23e thru 0xb1225e6 (0x33a9 bytes) 2024-09-14T06:17:26.869 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:17:26.877 INFO:teuthology.orchestra.run.smithi039.stdout:4602 trunc from 0xb1225e7 to 0x6ecf71d 2024-09-14T06:17:26.893 INFO:teuthology.orchestra.run.smithi039.stdout:4603 read 0x6585c08 thru 0x658af91 (0x538a bytes) 2024-09-14T06:17:26.905 INFO:teuthology.orchestra.run.smithi039.stdout:4604 writesame 0xa45b7b7 thru 0xa464339 data_size 0x5(0x8b83 bytes) 2024-09-14T06:17:26.905 INFO:teuthology.orchestra.run.smithi039.stdout:4605 writesame 0x589e097 thru 0x58a4be4 data_size 0xa(0x6b4e bytes) 2024-09-14T06:17:26.912 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:26.912 INFO:teuthology.orchestra.run.smithi039.stdout:4607 punch from 0x35d903b to 0x35dabd0, (0x1b95 bytes) 2024-09-14T06:17:26.913 INFO:teuthology.orchestra.run.smithi039.stdout:4608 read 0x9954714 thru 0x99645b7 (0xfea4 bytes) 2024-09-14T06:17:26.913 INFO:teuthology.orchestra.run.smithi039.stdout:4609 writesame 0x795db21 thru 0x796ce84 data_size 0x4a(0xf364 bytes) 2024-09-14T06:17:26.918 INFO:teuthology.orchestra.run.smithi039.stdout:4611 read 0xd1f171 thru 0xd2ac21 (0xbab1 bytes) 2024-09-14T06:17:26.918 INFO:teuthology.orchestra.run.smithi039.stdout:4612 punch from 0x963ecea to 0x964788b, (0x8ba1 bytes) 2024-09-14T06:17:26.918 INFO:teuthology.orchestra.run.smithi039.stdout:4613 punch from 0x5034124 to 0x5041ea0, (0xdd7c bytes) 2024-09-14T06:17:26.918 INFO:teuthology.orchestra.run.smithi039.stdout:4614 trunc from 0xa46433a to 0xa085dbe 2024-09-14T06:17:26.933 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:26.929+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:26.934 INFO:teuthology.orchestra.run.smithi039.stdout:4615 write 0xb80fe80 thru 0xb81a769 (0xa8ea bytes) 2024-09-14T06:17:26.938 INFO:teuthology.orchestra.run.smithi039.stdout:4616 write 0x801afeb thru 0x8022954 (0x796a bytes) 2024-09-14T06:17:26.944 INFO:teuthology.orchestra.run.smithi039.stdout:truncating to largest ever: 0xed919ab 2024-09-14T06:17:26.949 INFO:teuthology.orchestra.run.smithi039.stdout:4617 trunc from 0xb81a76a to 0xed919ab 2024-09-14T06:17:26.956 INFO:teuthology.orchestra.run.smithi039.stdout:4619 write 0x495906 thru 0x49e942 (0x903d bytes) 2024-09-14T06:17:26.958 INFO:teuthology.orchestra.run.smithi039.stdout:4621 write 0xdaba83 thru 0xdbabdf (0xf15d bytes) 2024-09-14T06:17:26.961 INFO:teuthology.orchestra.run.smithi039.stdout:4623 writesame 0x53adef4 thru 0x53b3a1d data_size 0x7(0x5b2a bytes) 2024-09-14T06:17:26.969 INFO:teuthology.orchestra.run.smithi039.stdout:4626 read 0x4260d72 thru 0x426adb5 (0xa044 bytes) 2024-09-14T06:17:26.969 INFO:teuthology.orchestra.run.smithi039.stdout:4627 writesame 0x9357bad thru 0x935b711 data_size 0x1(0x3b65 bytes) 2024-09-14T06:17:26.981 INFO:teuthology.orchestra.run.smithi039.stdout:4628 write 0x4492198 thru 0x4492525 (0x38e bytes) 2024-09-14T06:17:26.985 INFO:teuthology.orchestra.run.smithi039.stdout:4629 trunc from 0xed919ab to 0xe3f6b14 2024-09-14T06:17:26.991 INFO:teuthology.orchestra.run.smithi039.stdout:4630 read 0x6b68633 thru 0x6b6ce6d (0x483b bytes) 2024-09-14T06:17:26.992 INFO:teuthology.orchestra.run.smithi039.stdout:4631 write 0x3c3b6a8 thru 0x3c4a068 (0xe9c1 bytes) 2024-09-14T06:17:26.999 INFO:teuthology.orchestra.run.smithi039.stdout:4632 write 0x50573cb thru 0x505b62d (0x4263 bytes) 2024-09-14T06:17:27.002 INFO:teuthology.orchestra.run.smithi039.stdout:4633 write 0xd26dc58 thru 0xd27039d (0x2746 bytes) 2024-09-14T06:17:27.006 INFO:teuthology.orchestra.run.smithi039.stdout:4634 punch from 0xdc7b7c7 to 0xdc88138, (0xc971 bytes) 2024-09-14T06:17:27.006 INFO:teuthology.orchestra.run.smithi039.stdout:4635 read 0x6044f8 thru 0x60f141 (0xac4a bytes) 2024-09-14T06:17:27.008 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:27.008 INFO:teuthology.orchestra.run.smithi039.stdout:4639 read 0x5db4a31 thru 0x5dbdf7f (0x954f bytes) 2024-09-14T06:17:27.008 INFO:teuthology.orchestra.run.smithi039.stdout:4641 write 0x2dce10d thru 0x2dd745e (0x9352 bytes) 2024-09-14T06:17:27.013 INFO:teuthology.orchestra.run.smithi039.stdout:4642 read 0x6a52825 thru 0x6a53ab6 (0x1292 bytes) 2024-09-14T06:17:27.019 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:27.019 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:27.020 INFO:teuthology.orchestra.run.smithi039.stdout:4643 write 0xea5f7ab thru 0xea61bf6 (0x244c bytes) 2024-09-14T06:17:27.026 INFO:teuthology.orchestra.run.smithi039.stdout:4644 punch from 0xe598cc to 0xe64158, (0xa88c bytes) 2024-09-14T06:17:27.026 INFO:teuthology.orchestra.run.smithi039.stdout:4646 read 0xd8929ba thru 0xd898844 (0x5e8b bytes) 2024-09-14T06:17:27.026 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:27.027 INFO:teuthology.orchestra.run.smithi039.stdout:4649 writesame 0xe9345fc thru 0xe94003f data_size 0x1c(0xba44 bytes) 2024-09-14T06:17:27.031 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:27.031 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:27.031 INFO:teuthology.orchestra.run.smithi039.stdout:4652 writesame 0xbd9f306 thru 0xbda01f8 data_size 0x1(0xef3 bytes) 2024-09-14T06:17:27.034 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:27.029+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:27.038 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:17:27.039 INFO:teuthology.orchestra.run.smithi039.stdout:4653 read 0x38376de thru 0x3843a1c (0xc33f bytes) 2024-09-14T06:17:27.048 INFO:teuthology.orchestra.run.smithi039.stdout:4654 trunc from 0xea61bf7 to 0xa32b779 2024-09-14T06:17:27.060 INFO:teuthology.orchestra.run.smithi039.stdout:4655 read 0x1825c5b thru 0x182ee62 (0x9208 bytes) 2024-09-14T06:17:27.066 INFO:teuthology.orchestra.run.smithi039.stdout:4656 writesame 0xa9afde8 thru 0xa9bd0c0 data_size 0xb(0xd2d9 bytes) 2024-09-14T06:17:27.072 INFO:teuthology.orchestra.run.smithi039.stdout:4657 trunc from 0xa9bd0c1 to 0xec2091a 2024-09-14T06:17:27.079 INFO:teuthology.orchestra.run.smithi039.stdout:4658 writesame 0x23bffe8 thru 0x23c51aa data_size 0x1(0x51c3 bytes) 2024-09-14T06:17:27.096 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:27.097 INFO:teuthology.orchestra.run.smithi039.stdout:4660 writesame 0xcf64c4a thru 0xcf6a2af data_size 0x1(0x5666 bytes) 2024-09-14T06:17:27.119 INFO:teuthology.orchestra.run.smithi039.stdout:4661 write 0xb2f5f87 thru 0xb302df3 (0xce6d bytes) 2024-09-14T06:17:27.121 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:17:27.127 INFO:teuthology.orchestra.run.smithi039.stdout:4662 write 0x15e86bc thru 0x15f40fa (0xba3f bytes) 2024-09-14T06:17:27.133 INFO:teuthology.orchestra.run.smithi039.stdout:4663 read 0x99d0999 thru 0x99d3eb8 (0x3520 bytes) 2024-09-14T06:17:27.134 INFO:teuthology.orchestra.run.smithi039.stdout:4666 writesame 0x4dfd1e6 thru 0x4e01736 data_size 0x5b(0x4551 bytes) 2024-09-14T06:17:27.134 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:27.130+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:27.139 INFO:teuthology.orchestra.run.smithi039.stdout:4667 write 0xd13a06f thru 0xd13f851 (0x57e3 bytes) 2024-09-14T06:17:27.143 INFO:teuthology.orchestra.run.smithi039.stdout:4668 writesame 0xeaf1b8c thru 0xeaf8533 data_size 0x31(0x69a8 bytes) 2024-09-14T06:17:27.149 INFO:teuthology.orchestra.run.smithi039.stdout:4669 punch from 0x8c1b3ae to 0x8c22e3c, (0x7a8e bytes) 2024-09-14T06:17:27.165 INFO:teuthology.orchestra.run.smithi039.stdout:4670 trunc from 0xec2091a to 0x58070a4 2024-09-14T06:17:27.180 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:27.181 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:27.183 INFO:teuthology.orchestra.run.smithi039.stdout:4675 write 0x2d563b thru 0x2d5b1a (0x4e0 bytes) 2024-09-14T06:17:27.197 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:17:27.200 INFO:teuthology.orchestra.run.smithi039.stdout:4676 write 0x12b3f29 thru 0x12c3859 (0xf931 bytes) 2024-09-14T06:17:27.210 INFO:teuthology.orchestra.run.smithi039.stdout:4677 trunc from 0x58070a4 to 0x7ffe767 2024-09-14T06:17:27.217 INFO:teuthology.orchestra.run.smithi039.stdout:4678 punch from 0x1634e98 to 0x1638e5d, (0x3fc5 bytes) 2024-09-14T06:17:27.217 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:27.225 INFO:teuthology.orchestra.run.smithi039.stdout:4680 trunc from 0x7ffe767 to 0xd18bed1 2024-09-14T06:17:27.247 INFO:teuthology.orchestra.run.smithi039.stdout:4681 trunc from 0xd18bed1 to 0x43628e6 2024-09-14T06:17:27.247 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:27.245+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:27.265 INFO:teuthology.orchestra.run.smithi039.stdout:4682 read 0x2a6df81 thru 0x2a73192 (0x5212 bytes) 2024-09-14T06:17:27.266 INFO:teuthology.orchestra.run.smithi039.stdout:4683 punch from 0x40f9995 to 0x4106af4, (0xd15f bytes) 2024-09-14T06:17:27.266 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:27.280 INFO:teuthology.orchestra.run.smithi039.stdout:4685 write 0x95b095f thru 0x95b6371 (0x5a13 bytes) 2024-09-14T06:17:27.285 INFO:teuthology.orchestra.run.smithi039.stdout:4686 read 0x855e0d7 thru 0x856308e (0x4fb8 bytes) 2024-09-14T06:17:27.285 INFO:teuthology.orchestra.run.smithi039.stdout:4687 punch from 0x9579bb4 to 0x957f8c5, (0x5d11 bytes) 2024-09-14T06:17:27.285 INFO:teuthology.orchestra.run.smithi039.stdout:4688 write 0x36a658 thru 0x37261f (0x7fc8 bytes) 2024-09-14T06:17:27.293 INFO:teuthology.orchestra.run.smithi039.stdout:4689 trunc from 0x95b6372 to 0x4b5d407 2024-09-14T06:17:27.298 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:27.298 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:27.317 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:27.317 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:27.324 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:27.324 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:27.326 INFO:teuthology.orchestra.run.smithi039.stdout:4690 write 0xe90a2d6 thru 0xe90aaab (0x7d6 bytes) 2024-09-14T06:17:27.331 INFO:teuthology.orchestra.run.smithi039.stdout:4691 write 0x5a46a96 thru 0x5a4f1b1 (0x871c bytes) 2024-09-14T06:17:27.334 INFO:teuthology.orchestra.run.smithi039.stdout:4692 write 0xa748e0b thru 0xa75104b (0x8241 bytes) 2024-09-14T06:17:27.336 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:27.337 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:27.337 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:27.333+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:27.339 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:17:27.339 INFO:teuthology.orchestra.run.smithi039.stdout:4693 write 0xba9ce08 thru 0xbaa3b28 (0x6d21 bytes) 2024-09-14T06:17:27.345 INFO:teuthology.orchestra.run.smithi039.stdout:4695 punch from 0x457e956 to 0x4580ab6, (0x2160 bytes) 2024-09-14T06:17:27.345 INFO:teuthology.orchestra.run.smithi039.stdout:4696 write 0x16c07f5 thru 0x16c658f (0x5d9b bytes) 2024-09-14T06:17:27.347 INFO:teuthology.orchestra.run.smithi039.stdout:4697 read 0x533420a thru 0x533ba37 (0x782e bytes) 2024-09-14T06:17:27.347 INFO:teuthology.orchestra.run.smithi039.stdout:4698 read 0x395ee76 thru 0x3968dcd (0x9f58 bytes) 2024-09-14T06:17:27.347 INFO:teuthology.orchestra.run.smithi039.stdout:4699 write 0x3b90b04 thru 0x3b9e751 (0xdc4e bytes) 2024-09-14T06:17:27.351 INFO:teuthology.orchestra.run.smithi039.stdout:4700 writesame 0xe79dec7 thru 0xe7a1375 data_size 0x1(0x34af bytes) 2024-09-14T06:17:27.356 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:27.356 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:27.361 INFO:teuthology.orchestra.run.smithi039.stdout:4701 write 0x28bf980 thru 0x28c8ffb (0x967c bytes) 2024-09-14T06:17:27.365 INFO:teuthology.orchestra.run.smithi039.stdout:4702 read 0x4351df1 thru 0x4354985 (0x2b95 bytes) 2024-09-14T06:17:27.365 INFO:teuthology.orchestra.run.smithi039.stdout:4703 writesame 0x3ae9be8 thru 0x3aead82 data_size 0x1(0x119b bytes) 2024-09-14T06:17:27.369 INFO:teuthology.orchestra.run.smithi039.stdout:4704 punch from 0x8614623 to 0x861d088, (0x8a65 bytes) 2024-09-14T06:17:27.369 INFO:teuthology.orchestra.run.smithi039.stdout:4706 read 0x13b094a thru 0x13ba9e4 (0xa09b bytes) 2024-09-14T06:17:27.370 INFO:teuthology.orchestra.run.smithi039.stdout:4707 write 0xdb1b824 thru 0xdb22e4d (0x762a bytes) 2024-09-14T06:17:27.375 INFO:teuthology.orchestra.run.smithi039.stdout:4708 writesame 0x7b1b157 thru 0x7b21493 data_size 0x1(0x633d bytes) 2024-09-14T06:17:27.379 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:27.379 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:27.389 INFO:teuthology.orchestra.run.smithi039.stdout:4710 write 0x3651851 thru 0x365c9d0 (0xb180 bytes) 2024-09-14T06:17:27.393 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:27.393 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:27.393 INFO:teuthology.orchestra.run.smithi039.stdout:4714 read 0x5d63dc9 thru 0x5d73db7 (0xffef bytes) 2024-09-14T06:17:27.393 INFO:teuthology.orchestra.run.smithi039.stdout:4716 writesame 0xd1223a7 thru 0xd13204f data_size 0x1(0xfca9 bytes) 2024-09-14T06:17:27.398 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:27.399 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:27.418 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:27.418 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:27.431 INFO:teuthology.orchestra.run.smithi039.stdout:4717 punch from 0x62964d4 to 0x629cfa9, (0x6ad5 bytes) 2024-09-14T06:17:27.431 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:27.431 INFO:teuthology.orchestra.run.smithi039.stdout:4719 write 0xdba3ad2 thru 0xdba7cf9 (0x4228 bytes) 2024-09-14T06:17:27.435 INFO:teuthology.orchestra.run.smithi039.stdout:4720 writesame 0xcb08a00 thru 0xcb0b617 data_size 0x11(0x2c18 bytes) 2024-09-14T06:17:27.437 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:27.433+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:27.439 INFO:teuthology.orchestra.run.smithi039.stdout:4721 write 0xd7f3d84 thru 0xd800223 (0xc4a0 bytes) 2024-09-14T06:17:27.439 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:27.440 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:27.443 INFO:teuthology.orchestra.run.smithi039.stdout:4722 write 0x675e61e thru 0x6767c49 (0x962c bytes) 2024-09-14T06:17:27.449 INFO:teuthology.orchestra.run.smithi039.stdout:4723 punch from 0xc18db77 to 0xc19449a, (0x6923 bytes) 2024-09-14T06:17:27.450 INFO:teuthology.orchestra.run.smithi039.stdout:4724 write 0x3e9cebe thru 0x3ea8044 (0xb187 bytes) 2024-09-14T06:17:27.453 INFO:teuthology.orchestra.run.smithi039.stdout:4725 read 0xdb96471 thru 0xdba559d (0xf12d bytes) 2024-09-14T06:17:27.454 INFO:teuthology.orchestra.run.smithi039.stdout:4727 write 0x1bd7a9a thru 0x1bda437 (0x299e bytes) 2024-09-14T06:17:27.458 INFO:teuthology.orchestra.run.smithi039.stdout:4728 read 0xbb283ab thru 0xbb3205b (0x9cb1 bytes) 2024-09-14T06:17:27.458 INFO:teuthology.orchestra.run.smithi039.stdout:4729 writesame 0x781ba07 thru 0x782977c data_size 0x21(0xdd76 bytes) 2024-09-14T06:17:27.462 INFO:teuthology.orchestra.run.smithi039.stdout:4730 punch from 0x72f3095 to 0x72fdd12, (0xac7d bytes) 2024-09-14T06:17:27.462 INFO:teuthology.orchestra.run.smithi039.stdout:4731 write 0x3c5655e thru 0x3c62d9d (0xc840 bytes) 2024-09-14T06:17:27.463 INFO:teuthology.orchestra.run.smithi039.stdout:4732 writesame 0xe2e8cd3 thru 0xe2efa01 data_size 0x21(0x6d2f bytes) 2024-09-14T06:17:27.468 INFO:teuthology.orchestra.run.smithi039.stdout:truncating to largest ever: 0xedcd91b 2024-09-14T06:17:27.468 INFO:teuthology.orchestra.run.smithi039.stdout:4733 trunc from 0xe90aaac to 0xedcd91b 2024-09-14T06:17:27.474 INFO:teuthology.orchestra.run.smithi039.stdout:4734 write 0xc601349 thru 0xc60313b (0x1df3 bytes) 2024-09-14T06:17:27.478 INFO:teuthology.orchestra.run.smithi039.stdout:4735 read 0x16da47b thru 0x16e83e1 (0xdf67 bytes) 2024-09-14T06:17:27.478 INFO:teuthology.orchestra.run.smithi039.stdout:4736 read 0x23bb10c thru 0x23c7018 (0xbf0d bytes) 2024-09-14T06:17:27.491 INFO:teuthology.orchestra.run.smithi039.stdout:4738 trunc from 0xedcd91b to 0x7ab298e 2024-09-14T06:17:27.512 INFO:teuthology.orchestra.run.smithi039.stdout:4739 trunc from 0x7ab298e to 0x926bfc9 2024-09-14T06:17:27.519 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:27.519 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:27.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:27.519 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:27.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.520 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13229 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.520 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:27.520 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.418038351, 2024-09-14T06:17:27.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081798868999999996, 2024-09-14T06:17:27.520 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.520 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.520 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.520 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.520 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.520 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.520 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13229 2024-09-14T06:17:27.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.520 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:27.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:27.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098710+0000", 2024-09-14T06:17:27.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8199999999999999e-07 2024-09-14T06:17:27.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098714+0000", 2024-09-14T06:17:27.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:17:27.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.523 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.523 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098727+0000", 2024-09-14T06:17:27.523 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2520999999999999e-05 2024-09-14T06:17:27.523 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.523 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.523 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098749+0000", 2024-09-14T06:17:27.523 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2773999999999999e-05 2024-09-14T06:17:27.523 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.523 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.523 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160234+0000", 2024-09-14T06:17:27.524 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.061484430999999999 2024-09-14T06:17:27.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.524 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.524 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160281+0000", 2024-09-14T06:17:27.524 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7437e-05 2024-09-14T06:17:27.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.524 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.524 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160569+0000", 2024-09-14T06:17:27.524 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028788700000000002 2024-09-14T06:17:27.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163121+0000", 2024-09-14T06:17:27.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025514090000000001 2024-09-14T06:17:27.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180410+0000", 2024-09-14T06:17:27.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.017289098999999999 2024-09-14T06:17:27.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180472+0000", 2024-09-14T06:17:27.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2547999999999998e-05 2024-09-14T06:17:27.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180508+0000", 2024-09-14T06:17:27.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5534000000000002e-05 2024-09-14T06:17:27.526 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.526 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.526 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.527 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13230 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.527 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:27.527 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.417957149000003, 2024-09-14T06:17:27.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081851558000000005, 2024-09-14T06:17:27.527 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.527 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.527 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.527 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.527 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.527 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.527 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13230 2024-09-14T06:17:27.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.527 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:27.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:27.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098791+0000", 2024-09-14T06:17:27.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8800000000000001e-07 2024-09-14T06:17:27.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.529 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.529 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098795+0000", 2024-09-14T06:17:27.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.391e-06 2024-09-14T06:17:27.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.529 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.529 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098804+0000", 2024-09-14T06:17:27.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3440000000000001e-06 2024-09-14T06:17:27.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.530 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.530 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098810+0000", 2024-09-14T06:17:27.530 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2339999999999999e-06 2024-09-14T06:17:27.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.530 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.530 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.530 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160930+0000", 2024-09-14T06:17:27.530 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062120018999999999 2024-09-14T06:17:27.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.530 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.530 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.530 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161017+0000", 2024-09-14T06:17:27.530 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6868999999999998e-05 2024-09-14T06:17:27.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161300+0000", 2024-09-14T06:17:27.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028349499999999998 2024-09-14T06:17:27.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164612+0000", 2024-09-14T06:17:27.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033119680000000002 2024-09-14T06:17:27.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180559+0000", 2024-09-14T06:17:27.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015946612999999998 2024-09-14T06:17:27.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180605+0000", 2024-09-14T06:17:27.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5890000000000003e-05 2024-09-14T06:17:27.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180642+0000", 2024-09-14T06:17:27.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:17:27.532 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.532 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.532 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.533 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.533 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13231 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.533 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:27.533 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.417694734999998, 2024-09-14T06:17:27.533 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081750685000000003, 2024-09-14T06:17:27.533 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.533 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.533 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.533 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.533 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.533 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.533 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13231 2024-09-14T06:17:27.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.534 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.534 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.534 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:27.534 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.534 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.534 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:27.534 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.534 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.534 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099053+0000", 2024-09-14T06:17:27.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:17:27.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.535 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099059+0000", 2024-09-14T06:17:27.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1789999999999999e-06 2024-09-14T06:17:27.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.535 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099072+0000", 2024-09-14T06:17:27.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3179999999999999e-05 2024-09-14T06:17:27.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099080+0000", 2024-09-14T06:17:27.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9030000000000004e-06 2024-09-14T06:17:27.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161681+0000", 2024-09-14T06:17:27.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062601641999999999 2024-09-14T06:17:27.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161766+0000", 2024-09-14T06:17:27.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4909999999999998e-05 2024-09-14T06:17:27.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162109+0000", 2024-09-14T06:17:27.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00034264899999999999 2024-09-14T06:17:27.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164615+0000", 2024-09-14T06:17:27.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025061979999999998 2024-09-14T06:17:27.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180718+0000", 2024-09-14T06:17:27.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016103240000000001 2024-09-14T06:17:27.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.538 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.538 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180768+0000", 2024-09-14T06:17:27.538 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9781999999999999e-05 2024-09-14T06:17:27.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.538 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.538 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180803+0000", 2024-09-14T06:17:27.538 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5021999999999998e-05 2024-09-14T06:17:27.538 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.538 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.538 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.539 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.539 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13232 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.539 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:27.539 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.417652961000002, 2024-09-14T06:17:27.539 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081861981, 2024-09-14T06:17:27.539 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.539 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.539 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.539 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.539 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.539 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.539 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13232 2024-09-14T06:17:27.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.539 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:27.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:27.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099095+0000", 2024-09-14T06:17:27.541 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:17:27.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.541 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.541 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.541 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099098+0000", 2024-09-14T06:17:27.541 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5420000000000001e-06 2024-09-14T06:17:27.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.541 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.541 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.541 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099106+0000", 2024-09-14T06:17:27.541 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.824e-06 2024-09-14T06:17:27.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.541 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.542 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.542 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099112+0000", 2024-09-14T06:17:27.542 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9789999999999998e-06 2024-09-14T06:17:27.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.542 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.542 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162576+0000", 2024-09-14T06:17:27.542 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.063463723999999999 2024-09-14T06:17:27.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.542 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.542 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162627+0000", 2024-09-14T06:17:27.542 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1530000000000003e-05 2024-09-14T06:17:27.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162854+0000", 2024-09-14T06:17:27.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000226159 2024-09-14T06:17:27.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166090+0000", 2024-09-14T06:17:27.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032362319999999999 2024-09-14T06:17:27.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180877+0000", 2024-09-14T06:17:27.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014786726 2024-09-14T06:17:27.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.544 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.544 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180920+0000", 2024-09-14T06:17:27.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3427000000000003e-05 2024-09-14T06:17:27.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.544 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.544 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180956+0000", 2024-09-14T06:17:27.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6195999999999997e-05 2024-09-14T06:17:27.544 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.544 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.544 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.545 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13233 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.545 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:27.545 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.417627697999997, 2024-09-14T06:17:27.545 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081924860000000002, 2024-09-14T06:17:27.545 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.545 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.545 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.545 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.545 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.545 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.545 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13233 2024-09-14T06:17:27.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.546 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.546 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.546 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:27.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.546 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.546 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:27.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.546 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.547 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099120+0000", 2024-09-14T06:17:27.547 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:17:27.547 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.547 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.547 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.547 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099122+0000", 2024-09-14T06:17:27.547 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:17:27.547 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.547 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.547 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.547 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099126+0000", 2024-09-14T06:17:27.547 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2540000000000003e-06 2024-09-14T06:17:27.547 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.548 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.548 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.548 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099130+0000", 2024-09-14T06:17:27.548 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4060000000000001e-06 2024-09-14T06:17:27.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.548 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.548 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.548 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163169+0000", 2024-09-14T06:17:27.548 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064039415000000002 2024-09-14T06:17:27.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.548 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.548 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.548 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163260+0000", 2024-09-14T06:17:27.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1038000000000005e-05 2024-09-14T06:17:27.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163640+0000", 2024-09-14T06:17:27.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037954699999999999 2024-09-14T06:17:27.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166093+0000", 2024-09-14T06:17:27.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024533229999999999 2024-09-14T06:17:27.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180993+0000", 2024-09-14T06:17:27.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01489997 2024-09-14T06:17:27.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181020+0000", 2024-09-14T06:17:27.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6806999999999999e-05 2024-09-14T06:17:27.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181044+0000", 2024-09-14T06:17:27.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4405000000000001e-05 2024-09-14T06:17:27.550 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.551 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.551 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.551 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.551 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.551 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13234 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.551 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:27.551 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.417585492999997, 2024-09-14T06:17:27.551 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082015597999999995, 2024-09-14T06:17:27.551 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.551 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.551 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.551 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.551 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.552 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.552 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13234 2024-09-14T06:17:27.552 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.552 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.552 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.552 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.552 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:27.552 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.552 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.552 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.552 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.552 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:27.552 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.553 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.553 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.553 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.553 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:27.553 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0999999999999998e-07 2024-09-14T06:17:27.553 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.553 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.553 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.553 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099166+0000", 2024-09-14T06:17:27.553 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:17:27.553 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.553 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.553 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.553 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099171+0000", 2024-09-14T06:17:27.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6949999999999997e-06 2024-09-14T06:17:27.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099175+0000", 2024-09-14T06:17:27.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9029999999999997e-06 2024-09-14T06:17:27.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164063+0000", 2024-09-14T06:17:27.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064887608999999999 2024-09-14T06:17:27.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.555 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.555 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164111+0000", 2024-09-14T06:17:27.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7648999999999997e-05 2024-09-14T06:17:27.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.555 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.555 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.555 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164351+0000", 2024-09-14T06:17:27.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024072899999999999 2024-09-14T06:17:27.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.555 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.555 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.555 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167561+0000", 2024-09-14T06:17:27.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032094179999999999 2024-09-14T06:17:27.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.556 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.556 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181093+0000", 2024-09-14T06:17:27.556 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013532245 2024-09-14T06:17:27.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.556 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.556 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181139+0000", 2024-09-14T06:17:27.556 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6483999999999997e-05 2024-09-14T06:17:27.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.556 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.556 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181177+0000", 2024-09-14T06:17:27.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7861000000000002e-05 2024-09-14T06:17:27.557 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.557 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.557 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.557 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.557 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13235 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.557 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:27.557 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.417478219000003, 2024-09-14T06:17:27.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081988088000000001, 2024-09-14T06:17:27.557 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.557 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.557 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.558 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.558 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.558 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.558 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13235 2024-09-14T06:17:27.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.558 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.558 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.558 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:27.558 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.558 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.558 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:27.559 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.559 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.559 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.559 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099270+0000", 2024-09-14T06:17:27.559 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7400000000000003e-07 2024-09-14T06:17:27.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.559 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.559 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.559 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099276+0000", 2024-09-14T06:17:27.559 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:17:27.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.559 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.560 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099284+0000", 2024-09-14T06:17:27.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7979999999999995e-06 2024-09-14T06:17:27.560 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.560 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.560 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099289+0000", 2024-09-14T06:17:27.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:17:27.560 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.560 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.560 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164664+0000", 2024-09-14T06:17:27.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065374901999999999 2024-09-14T06:17:27.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.561 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.561 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164755+0000", 2024-09-14T06:17:27.561 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0790000000000003e-05 2024-09-14T06:17:27.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.561 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.561 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165143+0000", 2024-09-14T06:17:27.561 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000387904 2024-09-14T06:17:27.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.561 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.561 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167564+0000", 2024-09-14T06:17:27.562 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024208989999999998 2024-09-14T06:17:27.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.562 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.562 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181212+0000", 2024-09-14T06:17:27.562 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013648664 2024-09-14T06:17:27.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.562 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.562 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181237+0000", 2024-09-14T06:17:27.562 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5177999999999998e-05 2024-09-14T06:17:27.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.562 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181257+0000", 2024-09-14T06:17:27.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9660000000000002e-05 2024-09-14T06:17:27.563 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.563 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.563 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.563 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.563 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13236 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.563 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:27.563 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.4173714, 2024-09-14T06:17:27.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082010939000000005, 2024-09-14T06:17:27.563 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.563 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.563 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.564 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.564 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.564 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.564 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13236 2024-09-14T06:17:27.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.564 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.564 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.564 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:27.564 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.564 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.565 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:27.565 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.565 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.565 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.565 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.565 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:27.565 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.61e-07 2024-09-14T06:17:27.565 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.565 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.565 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.565 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099382+0000", 2024-09-14T06:17:27.565 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3589999999999999e-06 2024-09-14T06:17:27.565 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.566 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.566 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099388+0000", 2024-09-14T06:17:27.566 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9619999999999998e-06 2024-09-14T06:17:27.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.566 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.566 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099393+0000", 2024-09-14T06:17:27.566 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3410000000000002e-06 2024-09-14T06:17:27.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.566 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.566 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165553+0000", 2024-09-14T06:17:27.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066159604999999996 2024-09-14T06:17:27.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165603+0000", 2024-09-14T06:17:27.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0427999999999999e-05 2024-09-14T06:17:27.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165827+0000", 2024-09-14T06:17:27.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022358299999999999 2024-09-14T06:17:27.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.568 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177005+0000", 2024-09-14T06:17:27.568 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011178044 2024-09-14T06:17:27.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.568 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.568 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.568 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181302+0000", 2024-09-14T06:17:27.568 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042970669999999999 2024-09-14T06:17:27.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.568 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.568 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.568 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181349+0000", 2024-09-14T06:17:27.568 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7287999999999999e-05 2024-09-14T06:17:27.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.569 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.569 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181387+0000", 2024-09-14T06:17:27.569 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7601000000000001e-05 2024-09-14T06:17:27.569 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.569 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.569 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.569 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13237 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.569 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:27.569 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.417213253, 2024-09-14T06:17:27.569 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081927299999999995, 2024-09-14T06:17:27.569 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.570 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.570 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.570 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.570 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.570 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.570 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13237 2024-09-14T06:17:27.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.570 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.570 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.570 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:27.570 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.570 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.571 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:27.571 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.571 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.571 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.571 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099535+0000", 2024-09-14T06:17:27.571 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:17:27.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.571 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.571 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.571 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099538+0000", 2024-09-14T06:17:27.571 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:17:27.572 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.572 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.572 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099545+0000", 2024-09-14T06:17:27.572 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:17:27.572 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.572 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.572 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099551+0000", 2024-09-14T06:17:27.572 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1770000000000003e-06 2024-09-14T06:17:27.572 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.572 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.573 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166131+0000", 2024-09-14T06:17:27.573 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066580499000000001 2024-09-14T06:17:27.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.573 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.573 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.573 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166221+0000", 2024-09-14T06:17:27.573 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0000000000000006e-05 2024-09-14T06:17:27.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.573 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.573 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.573 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166622+0000", 2024-09-14T06:17:27.573 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040029799999999999 2024-09-14T06:17:27.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.573 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.574 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177008+0000", 2024-09-14T06:17:27.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010386772000000001 2024-09-14T06:17:27.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.574 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181420+0000", 2024-09-14T06:17:27.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044119190000000003 2024-09-14T06:17:27.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.574 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181443+0000", 2024-09-14T06:17:27.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2603999999999999e-05 2024-09-14T06:17:27.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181461+0000", 2024-09-14T06:17:27.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8405000000000001e-05 2024-09-14T06:17:27.575 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.575 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.575 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.575 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13238 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.575 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:27.575 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.417176945000001, 2024-09-14T06:17:27.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082014927000000001, 2024-09-14T06:17:27.576 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.576 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.576 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.576 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.576 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.576 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.576 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13238 2024-09-14T06:17:27.576 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.576 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.576 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.576 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.576 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:27.577 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.577 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.577 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.577 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:27.577 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.577 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.577 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.577 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099571+0000", 2024-09-14T06:17:27.577 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9699999999999996e-07 2024-09-14T06:17:27.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.577 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.577 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.578 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099577+0000", 2024-09-14T06:17:27.578 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2890000000000003e-06 2024-09-14T06:17:27.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.578 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.578 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.578 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099587+0000", 2024-09-14T06:17:27.578 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0002e-05 2024-09-14T06:17:27.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.578 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.578 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.578 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099593+0000", 2024-09-14T06:17:27.578 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4539999999999998e-06 2024-09-14T06:17:27.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.579 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.579 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.579 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167024+0000", 2024-09-14T06:17:27.579 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067431480000000002 2024-09-14T06:17:27.579 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.579 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.579 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.579 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167074+0000", 2024-09-14T06:17:27.579 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9996e-05 2024-09-14T06:17:27.579 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.579 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.579 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.579 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167288+0000", 2024-09-14T06:17:27.579 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021410900000000001 2024-09-14T06:17:27.580 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.580 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.580 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.580 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177997+0000", 2024-09-14T06:17:27.580 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010708756 2024-09-14T06:17:27.580 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.580 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.580 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.580 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181508+0000", 2024-09-14T06:17:27.580 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003511363 2024-09-14T06:17:27.580 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.580 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.580 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.580 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181552+0000", 2024-09-14T06:17:27.581 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3609999999999998e-05 2024-09-14T06:17:27.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.581 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.581 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181585+0000", 2024-09-14T06:17:27.581 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3271e-05 2024-09-14T06:17:27.581 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.581 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.581 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.581 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13239 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.581 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:27.582 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.417085127, 2024-09-14T06:17:27.582 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081998772999999997, 2024-09-14T06:17:27.582 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.582 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.582 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.582 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.582 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.582 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.582 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13239 2024-09-14T06:17:27.582 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.582 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.582 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.582 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.582 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:27.582 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.583 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.583 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.583 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.583 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:27.583 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.583 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.583 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.583 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.583 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099663+0000", 2024-09-14T06:17:27.583 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6100000000000001e-07 2024-09-14T06:17:27.583 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.583 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.583 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.583 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099666+0000", 2024-09-14T06:17:27.584 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.056e-06 2024-09-14T06:17:27.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.584 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.584 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099672+0000", 2024-09-14T06:17:27.584 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8120000000000002e-06 2024-09-14T06:17:27.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.584 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.584 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099676+0000", 2024-09-14T06:17:27.584 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:17:27.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.584 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167602+0000", 2024-09-14T06:17:27.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067925867000000001 2024-09-14T06:17:27.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.585 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167693+0000", 2024-09-14T06:17:27.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0736000000000007e-05 2024-09-14T06:17:27.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.585 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168065+0000", 2024-09-14T06:17:27.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037245399999999998 2024-09-14T06:17:27.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.586 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.586 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178003+0000", 2024-09-14T06:17:27.586 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0099372360000000003 2024-09-14T06:17:27.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.586 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.586 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.586 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181621+0000", 2024-09-14T06:17:27.586 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036178489999999998 2024-09-14T06:17:27.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.586 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.586 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.586 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181643+0000", 2024-09-14T06:17:27.586 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2155e-05 2024-09-14T06:17:27.586 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.587 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.587 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.587 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181661+0000", 2024-09-14T06:17:27.587 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8147000000000001e-05 2024-09-14T06:17:27.587 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.587 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.587 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.587 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.587 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.587 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13240 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.587 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:27.587 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.416966033999998, 2024-09-14T06:17:27.587 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081942607000000001, 2024-09-14T06:17:27.588 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.588 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.588 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.588 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.588 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.588 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.588 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13240 2024-09-14T06:17:27.588 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.588 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.588 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.588 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.588 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:27.588 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.589 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.589 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.589 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.589 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:27.589 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.589 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.589 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.589 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.589 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099782+0000", 2024-09-14T06:17:27.589 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3900000000000004e-07 2024-09-14T06:17:27.589 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.589 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.589 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.590 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099787+0000", 2024-09-14T06:17:27.590 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6189999999999999e-06 2024-09-14T06:17:27.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.590 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.590 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.590 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099793+0000", 2024-09-14T06:17:27.590 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.998e-06 2024-09-14T06:17:27.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.590 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.590 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.590 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099797+0000", 2024-09-14T06:17:27.590 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8449999999999996e-06 2024-09-14T06:17:27.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.591 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.591 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.591 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168494+0000", 2024-09-14T06:17:27.591 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.068696500999999993 2024-09-14T06:17:27.591 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.591 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.591 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.591 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168590+0000", 2024-09-14T06:17:27.591 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6733999999999999e-05 2024-09-14T06:17:27.591 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.591 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.591 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.591 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168973+0000", 2024-09-14T06:17:27.592 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038229399999999998 2024-09-14T06:17:27.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.592 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.592 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.592 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178013+0000", 2024-09-14T06:17:27.592 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0090397199999999994 2024-09-14T06:17:27.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.592 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.592 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.592 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181683+0000", 2024-09-14T06:17:27.592 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036705819999999999 2024-09-14T06:17:27.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.592 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.593 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.593 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181705+0000", 2024-09-14T06:17:27.593 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1551000000000001e-05 2024-09-14T06:17:27.593 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.593 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.593 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.593 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181724+0000", 2024-09-14T06:17:27.593 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9123999999999999e-05 2024-09-14T06:17:27.593 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.593 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.593 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.593 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.593 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.594 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13243 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.594 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:27.594 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.416432303999997, 2024-09-14T06:17:27.594 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081767815999999993, 2024-09-14T06:17:27.594 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.594 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.594 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.594 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.594 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.594 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.594 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13243 2024-09-14T06:17:27.594 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.594 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.594 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.595 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.595 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:27.595 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.595 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.595 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.595 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:27.595 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.595 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.595 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.595 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:27.595 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1e-07 2024-09-14T06:17:27.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.596 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.596 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.596 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100317+0000", 2024-09-14T06:17:27.596 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.092e-06 2024-09-14T06:17:27.596 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.596 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.596 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.596 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100321+0000", 2024-09-14T06:17:27.596 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7809999999999999e-06 2024-09-14T06:17:27.596 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.596 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.596 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.597 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100324+0000", 2024-09-14T06:17:27.597 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.117e-06 2024-09-14T06:17:27.597 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.597 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.597 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.597 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171113+0000", 2024-09-14T06:17:27.597 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.070788816000000004 2024-09-14T06:17:27.597 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.597 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.597 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.597 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171202+0000", 2024-09-14T06:17:27.597 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8843000000000003e-05 2024-09-14T06:17:27.597 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.598 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.598 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.598 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171584+0000", 2024-09-14T06:17:27.598 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038231099999999998 2024-09-14T06:17:27.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.598 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.598 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.598 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178101+0000", 2024-09-14T06:17:27.598 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0065168329999999997 2024-09-14T06:17:27.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.598 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.598 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.598 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182041+0000", 2024-09-14T06:17:27.598 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039401810000000001 2024-09-14T06:17:27.599 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.599 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.599 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.599 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182065+0000", 2024-09-14T06:17:27.599 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3623000000000001e-05 2024-09-14T06:17:27.599 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.599 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.599 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.599 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182083+0000", 2024-09-14T06:17:27.599 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7909000000000001e-05 2024-09-14T06:17:27.599 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.599 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.599 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.599 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.600 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.600 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13244 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.600 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:27.600 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.416417484, 2024-09-14T06:17:27.600 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081837800000000002, 2024-09-14T06:17:27.600 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.600 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.600 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.600 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.600 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.600 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.600 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13244 2024-09-14T06:17:27.600 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.601 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.601 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.601 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:27.601 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.601 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.601 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:27.601 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.601 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.602 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:27.602 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:17:27.602 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.602 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.602 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.602 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100337+0000", 2024-09-14T06:17:27.602 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.939e-06 2024-09-14T06:17:27.602 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.602 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.602 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.602 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100341+0000", 2024-09-14T06:17:27.602 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2320000000000003e-06 2024-09-14T06:17:27.602 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.602 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.603 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.603 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100344+0000", 2024-09-14T06:17:27.603 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0759999999999999e-06 2024-09-14T06:17:27.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.603 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.603 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171996+0000", 2024-09-14T06:17:27.603 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.071651919999999994 2024-09-14T06:17:27.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.603 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.603 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172079+0000", 2024-09-14T06:17:27.603 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2579999999999999e-05 2024-09-14T06:17:27.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.604 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.604 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.604 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172457+0000", 2024-09-14T06:17:27.604 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037785100000000001 2024-09-14T06:17:27.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.604 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.604 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.604 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178104+0000", 2024-09-14T06:17:27.604 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056471730000000001 2024-09-14T06:17:27.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.604 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.604 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.604 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182123+0000", 2024-09-14T06:17:27.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040190019999999998 2024-09-14T06:17:27.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.605 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.605 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182147+0000", 2024-09-14T06:17:27.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4241000000000001e-05 2024-09-14T06:17:27.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.605 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.605 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182168+0000", 2024-09-14T06:17:27.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0492999999999998e-05 2024-09-14T06:17:27.605 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.605 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.605 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.606 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.606 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.606 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13245 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.606 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:27.606 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.416397977000003, 2024-09-14T06:17:27.606 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081893899000000006, 2024-09-14T06:17:27.606 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.606 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.606 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.606 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.606 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.606 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.606 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13245 2024-09-14T06:17:27.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.607 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.607 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.607 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.607 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:27.607 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.607 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.607 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.607 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:27.607 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.607 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100350+0000", 2024-09-14T06:17:27.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5499999999999999e-07 2024-09-14T06:17:27.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100353+0000", 2024-09-14T06:17:27.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:17:27.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100357+0000", 2024-09-14T06:17:27.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8500000000000004e-06 2024-09-14T06:17:27.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.609 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.609 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.609 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100360+0000", 2024-09-14T06:17:27.609 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8700000000000001e-06 2024-09-14T06:17:27.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.609 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.609 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.609 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172862+0000", 2024-09-14T06:17:27.609 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.072501992000000001 2024-09-14T06:17:27.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.609 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.609 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.609 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172950+0000", 2024-09-14T06:17:27.610 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8988999999999998e-05 2024-09-14T06:17:27.610 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.610 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.610 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.610 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173324+0000", 2024-09-14T06:17:27.610 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037366199999999997 2024-09-14T06:17:27.610 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.610 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.610 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.610 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178107+0000", 2024-09-14T06:17:27.610 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047830520000000003 2024-09-14T06:17:27.610 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.610 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.611 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.611 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182202+0000", 2024-09-14T06:17:27.611 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040949280000000003 2024-09-14T06:17:27.611 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.611 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.611 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.611 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182226+0000", 2024-09-14T06:17:27.611 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3535000000000001e-05 2024-09-14T06:17:27.611 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.611 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.611 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.611 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182243+0000", 2024-09-14T06:17:27.611 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7445e-05 2024-09-14T06:17:27.611 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.612 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.612 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.612 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.612 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.612 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13246 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.612 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:27.612 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.416383654999997, 2024-09-14T06:17:27.612 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081962276000000001, 2024-09-14T06:17:27.612 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.612 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.612 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.612 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.612 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.613 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.613 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13246 2024-09-14T06:17:27.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.613 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.613 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.613 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.613 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:27.613 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.613 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.613 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.613 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:27.613 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.614 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.614 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.614 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.614 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:27.614 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0800000000000001e-07 2024-09-14T06:17:27.614 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.614 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.614 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.614 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100367+0000", 2024-09-14T06:17:27.614 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8439999999999998e-06 2024-09-14T06:17:27.614 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.614 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.614 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.615 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100372+0000", 2024-09-14T06:17:27.615 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4009999999999996e-06 2024-09-14T06:17:27.615 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.615 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.615 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.615 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100378+0000", 2024-09-14T06:17:27.615 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7180000000000001e-06 2024-09-14T06:17:27.615 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.615 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.615 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.615 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173780+0000", 2024-09-14T06:17:27.615 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.073401991 2024-09-14T06:17:27.615 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.615 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.616 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.616 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173879+0000", 2024-09-14T06:17:27.616 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9449000000000003e-05 2024-09-14T06:17:27.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.616 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.616 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.616 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174240+0000", 2024-09-14T06:17:27.616 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036118100000000002 2024-09-14T06:17:27.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.616 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.616 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.616 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178109+0000", 2024-09-14T06:17:27.617 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038690619999999999 2024-09-14T06:17:27.617 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.617 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.617 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.617 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182281+0000", 2024-09-14T06:17:27.617 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041715900000000002 2024-09-14T06:17:27.617 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.617 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.617 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.617 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182306+0000", 2024-09-14T06:17:27.617 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4477e-05 2024-09-14T06:17:27.617 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.617 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.618 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.618 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182326+0000", 2024-09-14T06:17:27.618 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0254999999999998e-05 2024-09-14T06:17:27.618 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.618 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.618 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.618 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.618 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.618 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13247 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.618 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:27.618 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.416286478000004, 2024-09-14T06:17:27.618 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966593000000004, 2024-09-14T06:17:27.618 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.619 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.619 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.619 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.619 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.619 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.619 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13247 2024-09-14T06:17:27.619 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.619 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.619 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.619 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.619 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:27.619 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.619 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.620 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.620 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.620 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:27.620 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.620 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.620 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.620 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.620 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100462+0000", 2024-09-14T06:17:27.620 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9800000000000002e-07 2024-09-14T06:17:27.620 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.620 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.620 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.620 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100466+0000", 2024-09-14T06:17:27.620 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4039999999999997e-06 2024-09-14T06:17:27.621 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.621 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.621 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.621 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100475+0000", 2024-09-14T06:17:27.621 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3680000000000008e-06 2024-09-14T06:17:27.621 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.621 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.621 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.621 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100482+0000", 2024-09-14T06:17:27.621 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8630000000000002e-06 2024-09-14T06:17:27.621 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.621 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.621 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.621 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174669+0000", 2024-09-14T06:17:27.622 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074186849999999999 2024-09-14T06:17:27.622 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.622 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.622 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.622 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174758+0000", 2024-09-14T06:17:27.622 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9021e-05 2024-09-14T06:17:27.622 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.622 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.623 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.623 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175137+0000", 2024-09-14T06:17:27.623 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378511 2024-09-14T06:17:27.623 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.623 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.623 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.623 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178158+0000", 2024-09-14T06:17:27.623 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003020956 2024-09-14T06:17:27.623 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.623 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.623 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.623 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182386+0000", 2024-09-14T06:17:27.624 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042278840000000003 2024-09-14T06:17:27.624 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.624 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.624 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.624 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182410+0000", 2024-09-14T06:17:27.624 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4026e-05 2024-09-14T06:17:27.624 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.624 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.624 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.624 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182427+0000", 2024-09-14T06:17:27.624 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7711999999999999e-05 2024-09-14T06:17:27.624 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.624 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.625 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.625 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.625 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.625 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13248 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.625 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:27.625 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.416204397000001, 2024-09-14T06:17:27.625 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966527999999997, 2024-09-14T06:17:27.625 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.625 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.625 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.625 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.625 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.626 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.626 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13248 2024-09-14T06:17:27.626 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.626 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.626 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.626 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.626 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:27.626 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.626 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.626 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.626 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.627 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:27.627 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.627 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.627 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.627 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.627 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:27.627 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:17:27.627 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.627 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.627 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.627 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100548+0000", 2024-09-14T06:17:27.627 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:17:27.627 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.627 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.628 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.628 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100557+0000", 2024-09-14T06:17:27.628 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:17:27.628 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.628 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.628 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.628 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100563+0000", 2024-09-14T06:17:27.628 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2519999999999996e-06 2024-09-14T06:17:27.628 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.628 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.628 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.628 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175572+0000", 2024-09-14T06:17:27.628 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075008459 2024-09-14T06:17:27.628 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.629 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.629 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.629 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175656+0000", 2024-09-14T06:17:27.629 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3906999999999995e-05 2024-09-14T06:17:27.629 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.629 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.629 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.629 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176010+0000", 2024-09-14T06:17:27.629 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00035447499999999998 2024-09-14T06:17:27.629 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.629 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.629 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.629 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178304+0000", 2024-09-14T06:17:27.629 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0022941379999999998 2024-09-14T06:17:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182467+0000", 2024-09-14T06:17:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041632020000000004 2024-09-14T06:17:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182491+0000", 2024-09-14T06:17:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3733000000000001e-05 2024-09-14T06:17:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182509+0000", 2024-09-14T06:17:27.630 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8113000000000001e-05 2024-09-14T06:17:27.631 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.631 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.631 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.631 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.631 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.631 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13249 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.631 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:27.631 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.416169463000003, 2024-09-14T06:17:27.631 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082037225000000005, 2024-09-14T06:17:27.631 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.631 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.631 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.631 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.631 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.632 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.632 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13249 2024-09-14T06:17:27.632 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.632 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.632 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.632 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.632 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:27.632 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.632 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.632 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.632 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.632 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:27.632 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.632 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.633 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.633 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.633 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:27.633 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4600000000000005e-07 2024-09-14T06:17:27.633 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.633 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.633 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.633 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100584+0000", 2024-09-14T06:17:27.633 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.947e-06 2024-09-14T06:17:27.633 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.633 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.633 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.634 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100593+0000", 2024-09-14T06:17:27.634 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9760000000000001e-06 2024-09-14T06:17:27.634 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.634 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.634 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.634 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100597+0000", 2024-09-14T06:17:27.634 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9689999999999996e-06 2024-09-14T06:17:27.634 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.634 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.634 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.634 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176461+0000", 2024-09-14T06:17:27.634 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075864132000000001 2024-09-14T06:17:27.634 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.635 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.635 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.635 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176516+0000", 2024-09-14T06:17:27.635 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4416999999999998e-05 2024-09-14T06:17:27.635 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.635 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.635 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.635 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176734+0000", 2024-09-14T06:17:27.635 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021859399999999999 2024-09-14T06:17:27.635 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.635 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.635 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.635 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178308+0000", 2024-09-14T06:17:27.635 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015733660000000001 2024-09-14T06:17:27.636 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.636 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.636 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.636 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182573+0000", 2024-09-14T06:17:27.636 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042649200000000002 2024-09-14T06:17:27.636 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.636 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.636 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.636 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182597+0000", 2024-09-14T06:17:27.636 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4355e-05 2024-09-14T06:17:27.636 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.636 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.636 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.636 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182615+0000", 2024-09-14T06:17:27.637 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8003e-05 2024-09-14T06:17:27.637 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.637 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.637 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.637 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.637 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.637 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13250 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:27.637 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:27.637 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 51.416004858000001, 2024-09-14T06:17:27.637 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081983890000000004, 2024-09-14T06:17:27.637 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:27.637 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:27.637 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:27.637 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:27.638 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:27.638 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:27.638 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13250 2024-09-14T06:17:27.638 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.638 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:27.638 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.638 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:27.638 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:27.638 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.638 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.638 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.638 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:27.638 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:27.638 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:27.639 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.639 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.639 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:27.639 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100743+0000", 2024-09-14T06:17:27.639 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:17:27.639 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.639 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.639 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:27.639 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100753+0000", 2024-09-14T06:17:27.639 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9979999999999998e-06 2024-09-14T06:17:27.639 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.639 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.639 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:27.639 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100772+0000", 2024-09-14T06:17:27.640 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8664e-05 2024-09-14T06:17:27.640 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.640 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.640 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:27.640 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100779+0000", 2024-09-14T06:17:27.640 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.046e-06 2024-09-14T06:17:27.640 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.640 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.640 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:27.640 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177049+0000", 2024-09-14T06:17:27.640 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076270819000000004 2024-09-14T06:17:27.640 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.640 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.641 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:27.641 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177140+0000", 2024-09-14T06:17:27.641 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0746000000000002e-05 2024-09-14T06:17:27.641 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.641 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.641 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:27.641 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177519+0000", 2024-09-14T06:17:27.641 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378656 2024-09-14T06:17:27.641 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.641 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.641 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:27.641 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178349+0000", 2024-09-14T06:17:27.641 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00083061599999999995 2024-09-14T06:17:27.641 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.642 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.642 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:27.642 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182679+0000", 2024-09-14T06:17:27.642 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043298379999999999 2024-09-14T06:17:27.642 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.642 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.642 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:27.642 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182705+0000", 2024-09-14T06:17:27.642 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5602999999999999e-05 2024-09-14T06:17:27.642 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:27.642 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:27.642 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:27.642 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182726+0000", 2024-09-14T06:17:27.642 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1297e-05 2024-09-14T06:17:27.643 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.643 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.643 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.643 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:27.643 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:27.643 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:27.645 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:17:27.646 INFO:teuthology.orchestra.run.smithi039.stdout:4740 writesame 0x8e10491 thru 0x8e1f1b8 data_size 0x4(0xed28 bytes) 2024-09-14T06:17:27.646 INFO:teuthology.orchestra.run.smithi039.stdout:4741 write 0xd98cd41 thru 0xd98f7a1 (0x2a61 bytes) 2024-09-14T06:17:27.646 INFO:teuthology.orchestra.run.smithi039.stdout:4742 read 0xac1f397 thru 0xac2cfab (0xdc15 bytes) 2024-09-14T06:17:27.646 INFO:teuthology.orchestra.run.smithi039.stdout:4743 read 0x7d402ae thru 0x7d4be3b (0xbb8e bytes) 2024-09-14T06:17:27.646 INFO:teuthology.orchestra.run.smithi039.stdout:4744 write 0x6a0d1f9 thru 0x6a0d670 (0x478 bytes) 2024-09-14T06:17:27.646 INFO:teuthology.orchestra.run.smithi039.stdout:4745 write 0xd32f8d0 thru 0xd33d65d (0xdd8e bytes) 2024-09-14T06:17:27.646 INFO:teuthology.orchestra.run.smithi039.stdout:4747 punch from 0x25b18eb to 0x25bf218, (0xd92d bytes) 2024-09-14T06:17:27.647 INFO:teuthology.orchestra.run.smithi039.stdout:4748 read 0x6c9e95d thru 0x6ca4b23 (0x61c7 bytes) 2024-09-14T06:17:27.647 INFO:teuthology.orchestra.run.smithi039.stdout:4749 read 0xba19c9e thru 0xba1ce2e (0x3191 bytes) 2024-09-14T06:17:27.647 INFO:teuthology.orchestra.run.smithi039.stdout:4750 write 0xdf048a8 thru 0xdf079d4 (0x312d bytes) 2024-09-14T06:17:27.647 INFO:teuthology.orchestra.run.smithi039.stdout:4751 punch from 0xd49e62c to 0xd4ae104, (0xfad8 bytes) 2024-09-14T06:17:27.647 INFO:teuthology.orchestra.run.smithi039.stdout:4752 write 0xd8932c5 thru 0xd89c7f7 (0x9533 bytes) 2024-09-14T06:17:27.647 INFO:teuthology.orchestra.run.smithi039.stdout:4753 trunc from 0xdf079d5 to 0x8cdfe3 2024-09-14T06:17:27.647 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:27.647 INFO:teuthology.orchestra.run.smithi039.stdout:4755 punch from 0xf277e to 0xf3267, (0xae9 bytes) 2024-09-14T06:17:27.647 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:27.642+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:27.661 INFO:teuthology.orchestra.run.smithi039.stdout:4756 write 0x9b5c021 thru 0x9b6b72b (0xf70b bytes) 2024-09-14T06:17:27.672 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:27.672 INFO:teuthology.orchestra.run.smithi039.stdout:4761 write 0x76c6a87 thru 0x76d303d (0xc5b7 bytes) 2024-09-14T06:17:27.678 INFO:teuthology.orchestra.run.smithi039.stdout:4762 read 0x964190 thru 0x971aee (0xd95f bytes) 2024-09-14T06:17:27.694 INFO:teuthology.orchestra.run.smithi039.stdout:4763 trunc from 0x9b6b72c to 0x2f48f22 2024-09-14T06:17:27.721 INFO:teuthology.orchestra.run.smithi039.stdout:4767 punch from 0x1480ad7 to 0x1483d22, (0x324b bytes) 2024-09-14T06:17:27.734 INFO:teuthology.orchestra.run.smithi039.stdout:4769 trunc from 0x2f48f22 to 0x841819c 2024-09-14T06:17:27.745 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:27.741+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:27.748 INFO:teuthology.orchestra.run.smithi039.stdout:4770 write 0x173ee1f thru 0x174d618 (0xe7fa bytes) 2024-09-14T06:17:27.756 INFO:teuthology.orchestra.run.smithi039.stdout:4771 read 0x3322573 thru 0x3330c84 (0xe712 bytes) 2024-09-14T06:17:27.756 INFO:teuthology.orchestra.run.smithi039.stdout:4772 read 0x640711a thru 0x6415abd (0xe9a4 bytes) 2024-09-14T06:17:27.756 INFO:teuthology.orchestra.run.smithi039.stdout:4773 punch from 0x750b7c to 0x75a048, (0x94cc bytes) 2024-09-14T06:17:27.756 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:27.774 INFO:teuthology.orchestra.run.smithi039.stdout:4777 writesame 0xab1c2c2 thru 0xab23a5e data_size 0x3(0x779d bytes) 2024-09-14T06:17:27.774 INFO:teuthology.orchestra.run.smithi039.stdout:4779 read 0x8ffbff thru 0x90afc0 (0xb3c2 bytes) 2024-09-14T06:17:27.775 INFO:teuthology.orchestra.run.smithi039.stdout:4780 punch from 0x1392106 to 0x1396bd8, (0x4ad2 bytes) 2024-09-14T06:17:27.775 INFO:teuthology.orchestra.run.smithi039.stdout:4781 write 0x121b0eb thru 0x1229e25 (0xed3b bytes) 2024-09-14T06:17:27.782 INFO:teuthology.orchestra.run.smithi039.stdout:4782 writesame 0x9004173 thru 0x901049c data_size 0x21(0xc32a bytes) 2024-09-14T06:17:27.792 INFO:teuthology.orchestra.run.smithi039.stdout:4784 punch from 0x7389c4 to 0x741b5f, (0x919b bytes) 2024-09-14T06:17:27.792 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:27.818 INFO:teuthology.orchestra.run.smithi039.stdout:4786 trunc from 0xab23a5f to 0x229e4f 2024-09-14T06:17:27.819 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:27.819 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:27.819 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:27.819 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:27.835 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:17:27.846 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:27.842+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:27.848 INFO:teuthology.orchestra.run.smithi039.stdout:4789 read 0x123e69 thru 0x12ea35 (0xabcd bytes) 2024-09-14T06:17:27.881 INFO:teuthology.orchestra.run.smithi039.stdout:4791 write 0x6b71fa9 thru 0x6b75fbd (0x4015 bytes) 2024-09-14T06:17:27.887 INFO:teuthology.orchestra.run.smithi039.stdout:4793 read 0xbf7e43 thru 0xc02a0c (0xabca bytes) 2024-09-14T06:17:27.888 INFO:teuthology.orchestra.run.smithi039.stdout:4794 flatten 2024-09-14T06:17:27.901 INFO:teuthology.orchestra.run.smithi039.stdout:4796 write 0x310030f thru 0x3100cb5 (0x9a7 bytes) 2024-09-14T06:17:27.908 INFO:teuthology.orchestra.run.smithi039.stdout:4797 punch from 0x537bd9f to 0x538408c, (0x82ed bytes) 2024-09-14T06:17:27.917 INFO:teuthology.orchestra.run.smithi039.stdout:4798 trunc from 0x6b75fbe to 0x3964297 2024-09-14T06:17:27.941 INFO:teuthology.orchestra.run.smithi039.stdout:4799 read 0x2d95311 thru 0x2d9e2c6 (0x8fb6 bytes) 2024-09-14T06:17:27.941 INFO:teuthology.orchestra.run.smithi039.stdout:4800 read 0x1a232f0 thru 0x1a311d3 (0xdee4 bytes) 2024-09-14T06:17:27.941 INFO:teuthology.orchestra.run.smithi039.stdout:4801 writesame 0x333eba6 thru 0x33460da data_size 0x11(0x7535 bytes) 2024-09-14T06:17:27.946 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:27.941+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:27.954 INFO:teuthology.orchestra.run.smithi039.stdout:4802 trunc from 0x3964297 to 0x1db0918 2024-09-14T06:17:27.980 INFO:teuthology.orchestra.run.smithi039.stdout:4803 read 0x1b4351 thru 0x1bc40c (0x80bc bytes) 2024-09-14T06:17:27.987 INFO:teuthology.orchestra.run.smithi039.stdout:4805 trunc from 0x1db0918 to 0x3d093e3 2024-09-14T06:17:28.000 INFO:teuthology.orchestra.run.smithi039.stdout:4806 punch from 0x3833236 to 0x383c741, (0x950b bytes) 2024-09-14T06:17:28.003 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:28.003 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:28.003 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:28.003 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:28.003 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:28.012 INFO:teuthology.orchestra.run.smithi039.stdout:4808 write 0x3d4dbea thru 0x3d57aad (0x9ec4 bytes) 2024-09-14T06:17:28.018 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:17:28.029 INFO:teuthology.orchestra.run.smithi039.stdout:4809 trunc from 0x3d57aae to 0x155c80 2024-09-14T06:17:28.048 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:28.046+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:28.055 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:28.055 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:28.096 INFO:teuthology.orchestra.run.smithi039.stdout:4813 write 0xa19d3f8 thru 0xa19e432 (0x103b bytes) 2024-09-14T06:17:28.104 INFO:teuthology.orchestra.run.smithi039.stdout:4814 read 0x8ea6236 thru 0x8ea77e0 (0x15ab bytes) 2024-09-14T06:17:28.104 INFO:teuthology.orchestra.run.smithi039.stdout:4815 punch from 0x9ef6234 to 0x9f01c75, (0xba41 bytes) 2024-09-14T06:17:28.104 INFO:teuthology.orchestra.run.smithi039.stdout:4820 read 0x1ffd6a3 thru 0x2005e50 (0x87ae bytes) 2024-09-14T06:17:28.104 INFO:teuthology.orchestra.run.smithi039.stdout:4821 write 0x95aafd6 thru 0x95b6f3d (0xbf68 bytes) 2024-09-14T06:17:28.110 INFO:teuthology.orchestra.run.smithi039.stdout:4822 read 0x4fd337e thru 0x4fd8578 (0x51fb bytes) 2024-09-14T06:17:28.132 INFO:teuthology.orchestra.run.smithi039.stdout:4823 writesame 0xcb9f51e thru 0xcba13ff data_size 0x1(0x1ee2 bytes) 2024-09-14T06:17:28.132 INFO:teuthology.orchestra.run.smithi039.stdout:4825 punch from 0x497c819 to 0x498b59b, (0xed82 bytes) 2024-09-14T06:17:28.148 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:28.145+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:28.150 INFO:teuthology.orchestra.run.smithi039.stdout:4826 write 0xe81b4ea thru 0xe826cb0 (0xb7c7 bytes) 2024-09-14T06:17:28.162 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:28.162 INFO:teuthology.orchestra.run.smithi039.stdout:4831 writesame 0x93a705b thru 0x93a7e4e data_size 0x4(0xdf4 bytes) 2024-09-14T06:17:28.182 INFO:teuthology.orchestra.run.smithi039.stdout:4832 trunc from 0xe826cb1 to 0xa355c47 2024-09-14T06:17:28.183 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:28.183 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:28.183 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:28.183 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:28.183 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.183 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:30547 3.1f 3:f84dfb53:::rbd_data.11529eb229bd.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write 16226533~50642] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T06:17:28.183 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:28.183 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 28.991195738000002, 2024-09-14T06:17:28.183 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0079770020000001, 2024-09-14T06:17:28.183 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.183 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.183 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:28.184 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.184 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.184 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.184 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 30547 2024-09-14T06:17:28.184 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.184 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.184 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.184 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.184 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:28.184 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.184 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.184 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.184 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.185 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:28.185 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.185 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.185 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.185 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189811+0000", 2024-09-14T06:17:28.185 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2060000000000001e-06 2024-09-14T06:17:28.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.185 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.185 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.185 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189871+0000", 2024-09-14T06:17:28.185 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9862999999999999e-05 2024-09-14T06:17:28.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.185 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189899+0000", 2024-09-14T06:17:28.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7849000000000001e-05 2024-09-14T06:17:28.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189915+0000", 2024-09-14T06:17:28.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:17:28.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190024+0000", 2024-09-14T06:17:28.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000109202 2024-09-14T06:17:28.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:17:28.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190041+0000", 2024-09-14T06:17:28.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7665000000000001e-05 2024-09-14T06:17:28.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188512+0000", 2024-09-14T06:17:28.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.99847100700000002 2024-09-14T06:17:28.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for missing object", 2024-09-14T06:17:28.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188646+0000", 2024-09-14T06:17:28.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000133986 2024-09-14T06:17:28.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.188 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.188 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193097+0000", 2024-09-14T06:17:28.188 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044507059999999996 2024-09-14T06:17:28.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.188 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.188 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.188 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193183+0000", 2024-09-14T06:17:28.188 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5496999999999998e-05 2024-09-14T06:17:28.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.188 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.188 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:17:28.189 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193307+0000", 2024-09-14T06:17:28.189 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000124611 2024-09-14T06:17:28.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.189 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.189 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:28.189 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.194318+0000", 2024-09-14T06:17:28.189 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0010106 2024-09-14T06:17:28.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.189 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.189 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:28.189 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197670+0000", 2024-09-14T06:17:28.189 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033523170000000001 2024-09-14T06:17:28.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.190 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:28.190 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197712+0000", 2024-09-14T06:17:28.190 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1681000000000003e-05 2024-09-14T06:17:28.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.190 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.190 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197786+0000", 2024-09-14T06:17:28.190 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4148999999999998e-05 2024-09-14T06:17:28.190 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.190 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.190 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.191 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32022 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:28.191 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:28.191 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.312815527000001, 2024-09-14T06:17:28.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095598104000000003, 2024-09-14T06:17:28.191 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.191 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.191 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.191 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.191 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.191 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.191 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32022 2024-09-14T06:17:28.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.191 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:28.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:28.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.193 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.193 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868190+0000", 2024-09-14T06:17:28.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6000000000000003e-07 2024-09-14T06:17:28.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.193 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.194 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868243+0000", 2024-09-14T06:17:28.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3443999999999999e-05 2024-09-14T06:17:28.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.194 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.194 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868260+0000", 2024-09-14T06:17:28.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6799e-05 2024-09-14T06:17:28.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.194 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.194 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868271+0000", 2024-09-14T06:17:28.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0523000000000001e-05 2024-09-14T06:17:28.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963502+0000", 2024-09-14T06:17:28.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095231466000000001 2024-09-14T06:17:28.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963636+0000", 2024-09-14T06:17:28.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013388699999999999 2024-09-14T06:17:28.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963787+0000", 2024-09-14T06:17:28.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000151225 2024-09-14T06:17:28.196 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.196 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.196 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.196 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32023 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5816320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:28.196 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:28.196 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.312720401, 2024-09-14T06:17:28.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095879753999999998, 2024-09-14T06:17:28.196 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.196 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.196 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.197 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.197 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.197 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.197 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32023 2024-09-14T06:17:28.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.197 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:28.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.198 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.198 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:28.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.198 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.198 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868285+0000", 2024-09-14T06:17:28.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6299999999999993e-07 2024-09-14T06:17:28.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.198 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.198 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868288+0000", 2024-09-14T06:17:28.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.18e-06 2024-09-14T06:17:28.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.199 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868298+0000", 2024-09-14T06:17:28.199 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0029e-05 2024-09-14T06:17:28.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.199 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868307+0000", 2024-09-14T06:17:28.200 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7229999999999993e-06 2024-09-14T06:17:28.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.200 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.200 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.200 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963871+0000", 2024-09-14T06:17:28.200 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095563876000000006 2024-09-14T06:17:28.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.200 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.200 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.200 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964015+0000", 2024-09-14T06:17:28.200 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143573 2024-09-14T06:17:28.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.200 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.201 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.201 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964164+0000", 2024-09-14T06:17:28.201 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014940999999999999 2024-09-14T06:17:28.201 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.201 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.201 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.201 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.201 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.201 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32024 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:28.201 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:28.201 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.312687257, 2024-09-14T06:17:28.201 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096230231999999999, 2024-09-14T06:17:28.201 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.201 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.201 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.202 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.202 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.202 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.202 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32024 2024-09-14T06:17:28.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.202 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.202 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.202 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.202 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:28.202 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.202 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.202 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.203 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:28.203 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.203 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.203 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.203 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.203 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868318+0000", 2024-09-14T06:17:28.203 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1699999999999997e-07 2024-09-14T06:17:28.203 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.204 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868321+0000", 2024-09-14T06:17:28.204 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6769999999999999e-06 2024-09-14T06:17:28.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.204 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868334+0000", 2024-09-14T06:17:28.204 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2928999999999999e-05 2024-09-14T06:17:28.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.205 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868341+0000", 2024-09-14T06:17:28.205 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9800000000000001e-06 2024-09-14T06:17:28.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.205 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.205 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.205 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964258+0000", 2024-09-14T06:17:28.205 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095917099000000006 2024-09-14T06:17:28.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.205 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.205 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.205 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964392+0000", 2024-09-14T06:17:28.205 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013396999999999999 2024-09-14T06:17:28.206 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.206 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.206 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964548+0000", 2024-09-14T06:17:28.206 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015585999999999999 2024-09-14T06:17:28.206 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.206 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.206 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.206 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.206 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32025 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:28.206 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:28.206 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.312654600999998, 2024-09-14T06:17:28.206 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096576591000000003, 2024-09-14T06:17:28.206 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.207 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.207 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.207 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.207 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.207 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.207 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32025 2024-09-14T06:17:28.207 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.207 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.208 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.208 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.208 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:28.208 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.208 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.208 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.208 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:28.208 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.208 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.208 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.208 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868351+0000", 2024-09-14T06:17:28.209 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1600000000000001e-07 2024-09-14T06:17:28.209 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.209 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.209 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.209 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868353+0000", 2024-09-14T06:17:28.209 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.627e-06 2024-09-14T06:17:28.209 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.209 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.209 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.209 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868364+0000", 2024-09-14T06:17:28.210 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0893000000000001e-05 2024-09-14T06:17:28.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.210 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.210 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.210 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868381+0000", 2024-09-14T06:17:28.210 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6769000000000002e-05 2024-09-14T06:17:28.211 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.211 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.211 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.211 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964632+0000", 2024-09-14T06:17:28.211 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096250589999999997 2024-09-14T06:17:28.211 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.211 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.212 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.212 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964772+0000", 2024-09-14T06:17:28.212 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014045299999999999 2024-09-14T06:17:28.212 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.212 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.212 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.212 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964927+0000", 2024-09-14T06:17:28.212 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015454299999999999 2024-09-14T06:17:28.212 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.212 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.213 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.213 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.213 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.213 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32026 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:28.213 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:28.213 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.312613649999999, 2024-09-14T06:17:28.213 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096901622000000007, 2024-09-14T06:17:28.213 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.213 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.214 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.214 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.214 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.214 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.214 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32026 2024-09-14T06:17:28.214 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.214 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.214 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.214 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.214 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:28.214 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.214 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.215 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.215 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.215 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:28.215 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.215 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.215 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.215 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.215 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868392+0000", 2024-09-14T06:17:28.215 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0699999999999996e-07 2024-09-14T06:17:28.215 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.215 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.216 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.216 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868395+0000", 2024-09-14T06:17:28.216 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9859999999999999e-06 2024-09-14T06:17:28.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.216 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.216 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.216 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868405+0000", 2024-09-14T06:17:28.216 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9229999999999997e-06 2024-09-14T06:17:28.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.216 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.216 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.217 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868412+0000", 2024-09-14T06:17:28.217 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9670000000000004e-06 2024-09-14T06:17:28.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.217 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.217 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.217 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965010+0000", 2024-09-14T06:17:28.217 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096598562999999998 2024-09-14T06:17:28.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.217 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.217 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.217 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965143+0000", 2024-09-14T06:17:28.217 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013247700000000001 2024-09-14T06:17:28.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.217 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.218 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.218 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965293+0000", 2024-09-14T06:17:28.218 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000149899 2024-09-14T06:17:28.218 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.218 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.218 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.218 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.218 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.218 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32027 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:28.218 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:28.219 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.312545313000001, 2024-09-14T06:17:28.219 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097211306999999997, 2024-09-14T06:17:28.219 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.219 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.219 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.219 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.219 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.219 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.219 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32027 2024-09-14T06:17:28.219 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.219 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.219 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.219 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.220 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:28.220 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.220 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.220 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.220 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.220 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:28.220 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.220 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.220 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.220 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.220 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868461+0000", 2024-09-14T06:17:28.220 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.296e-06 2024-09-14T06:17:28.220 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.221 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.221 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.221 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868465+0000", 2024-09-14T06:17:28.221 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9570000000000004e-06 2024-09-14T06:17:28.221 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.221 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.221 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.221 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868476+0000", 2024-09-14T06:17:28.221 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1365999999999999e-05 2024-09-14T06:17:28.221 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.221 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.221 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.221 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868483+0000", 2024-09-14T06:17:28.222 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2529999999999998e-06 2024-09-14T06:17:28.222 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.222 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.222 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.222 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965375+0000", 2024-09-14T06:17:28.222 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096891973000000006 2024-09-14T06:17:28.222 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.222 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.222 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.222 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965518+0000", 2024-09-14T06:17:28.222 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143017 2024-09-14T06:17:28.222 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.222 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.223 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.223 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965671+0000", 2024-09-14T06:17:28.223 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015244499999999999 2024-09-14T06:17:28.223 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.223 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.223 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.223 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.223 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.223 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32028 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:28.223 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:28.223 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.312510485000001, 2024-09-14T06:17:28.224 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097841480999999994, 2024-09-14T06:17:28.224 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.224 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.224 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.224 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.224 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.224 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.224 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32028 2024-09-14T06:17:28.224 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.225 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.225 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.225 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.225 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:28.225 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.225 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.225 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.225 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.225 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:28.225 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.225 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.225 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.225 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.225 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868495+0000", 2024-09-14T06:17:28.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6400000000000001e-07 2024-09-14T06:17:28.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.226 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.226 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868498+0000", 2024-09-14T06:17:28.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7760000000000002e-06 2024-09-14T06:17:28.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.226 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.226 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868507+0000", 2024-09-14T06:17:28.227 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1530000000000003e-06 2024-09-14T06:17:28.227 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.227 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.227 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.227 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868516+0000", 2024-09-14T06:17:28.227 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8929999999999994e-06 2024-09-14T06:17:28.227 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.227 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.227 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.227 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965753+0000", 2024-09-14T06:17:28.227 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097236970000000006 2024-09-14T06:17:28.227 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.228 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.228 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.228 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965890+0000", 2024-09-14T06:17:28.228 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013677599999999999 2024-09-14T06:17:28.228 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.228 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.228 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.228 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966336+0000", 2024-09-14T06:17:28.228 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00044614900000000001 2024-09-14T06:17:28.228 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.228 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.229 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.229 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.229 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.229 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32029 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:28.229 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:28.229 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.312479410000002, 2024-09-14T06:17:28.229 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098577779000000004, 2024-09-14T06:17:28.229 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.230 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.230 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.230 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.230 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.230 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.230 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32029 2024-09-14T06:17:28.230 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.230 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.230 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.230 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.230 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:28.230 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.230 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.231 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.231 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.231 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:28.231 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.231 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.231 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.231 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.231 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868526+0000", 2024-09-14T06:17:28.231 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9300000000000006e-07 2024-09-14T06:17:28.231 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.231 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.231 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.231 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868529+0000", 2024-09-14T06:17:28.231 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5969999999999999e-06 2024-09-14T06:17:28.232 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.232 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.232 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.232 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868538+0000", 2024-09-14T06:17:28.232 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1700000000000003e-06 2024-09-14T06:17:28.232 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.232 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.232 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.232 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868560+0000", 2024-09-14T06:17:28.232 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1596000000000001e-05 2024-09-14T06:17:28.232 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.232 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.232 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.232 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966423+0000", 2024-09-14T06:17:28.233 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097863396000000005 2024-09-14T06:17:28.233 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.233 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.233 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.233 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966578+0000", 2024-09-14T06:17:28.233 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000154792 2024-09-14T06:17:28.233 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.233 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.233 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.233 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967103+0000", 2024-09-14T06:17:28.233 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00052523499999999998 2024-09-14T06:17:28.233 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.233 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.234 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.234 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.234 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.234 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32030 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:28.234 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:28.234 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.312434492000001, 2024-09-14T06:17:28.234 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099324030999999993, 2024-09-14T06:17:28.234 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.235 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.235 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.235 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.235 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.235 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.235 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32030 2024-09-14T06:17:28.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.235 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.235 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.235 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.235 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:28.235 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.235 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.236 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.236 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:28.236 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.236 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.236 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.236 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.236 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868571+0000", 2024-09-14T06:17:28.236 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9699999999999995e-07 2024-09-14T06:17:28.236 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.237 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.237 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.237 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868574+0000", 2024-09-14T06:17:28.237 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9529999999999999e-06 2024-09-14T06:17:28.237 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.237 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.237 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.237 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868584+0000", 2024-09-14T06:17:28.237 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7860000000000008e-06 2024-09-14T06:17:28.237 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.237 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.237 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.237 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868591+0000", 2024-09-14T06:17:28.237 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0500000000000003e-06 2024-09-14T06:17:28.238 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.238 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.238 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.238 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967191+0000", 2024-09-14T06:17:28.238 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098600035000000003 2024-09-14T06:17:28.238 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.238 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.238 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.238 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967333+0000", 2024-09-14T06:17:28.238 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014190599999999999 2024-09-14T06:17:28.238 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.238 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.238 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.238 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967894+0000", 2024-09-14T06:17:28.238 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000561504 2024-09-14T06:17:28.239 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.239 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.239 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.239 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.239 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.239 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32031 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:28.239 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:28.239 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.31240493, 2024-09-14T06:17:28.239 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.10010279, 2024-09-14T06:17:28.239 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.239 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.239 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.239 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.239 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.239 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.240 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32031 2024-09-14T06:17:28.240 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.240 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.240 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.240 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.240 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:28.240 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.240 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.240 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.240 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.240 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:28.240 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.241 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.241 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.241 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.241 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:28.241 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:17:28.241 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.241 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.241 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.241 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868603+0000", 2024-09-14T06:17:28.241 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.599e-06 2024-09-14T06:17:28.241 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.241 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.241 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.241 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868612+0000", 2024-09-14T06:17:28.241 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.986e-06 2024-09-14T06:17:28.242 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.242 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.242 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.242 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868619+0000", 2024-09-14T06:17:28.242 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8900000000000001e-06 2024-09-14T06:17:28.242 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.242 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.242 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.242 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967983+0000", 2024-09-14T06:17:28.242 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099363612000000004 2024-09-14T06:17:28.242 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.242 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.242 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.242 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968140+0000", 2024-09-14T06:17:28.243 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015788600000000001 2024-09-14T06:17:28.243 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.243 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.243 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.243 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968703+0000", 2024-09-14T06:17:28.243 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00056208300000000001 2024-09-14T06:17:28.243 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.243 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.243 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.243 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.243 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.243 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32032 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6258688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:28.243 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:28.243 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.312376454999999, 2024-09-14T06:17:28.244 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100488088, 2024-09-14T06:17:28.244 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.244 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.244 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.244 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.244 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.244 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.244 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32032 2024-09-14T06:17:28.244 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.244 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.244 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.245 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.245 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:28.245 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.245 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.245 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.245 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.246 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:28.246 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.246 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.246 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.246 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.246 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868629+0000", 2024-09-14T06:17:28.246 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:17:28.246 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.246 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.246 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.246 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868631+0000", 2024-09-14T06:17:28.246 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4099999999999998e-06 2024-09-14T06:17:28.247 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.247 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.247 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.247 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868640+0000", 2024-09-14T06:17:28.247 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0939999999999997e-06 2024-09-14T06:17:28.247 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.247 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.247 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.247 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868647+0000", 2024-09-14T06:17:28.247 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9460000000000001e-06 2024-09-14T06:17:28.247 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.248 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.248 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.248 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968797+0000", 2024-09-14T06:17:28.248 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100149867 2024-09-14T06:17:28.248 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.248 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.248 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.248 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968937+0000", 2024-09-14T06:17:28.248 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013981399999999999 2024-09-14T06:17:28.248 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.248 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.248 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.248 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969116+0000", 2024-09-14T06:17:28.249 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017927399999999999 2024-09-14T06:17:28.249 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.249 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.249 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.249 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.249 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.249 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32044 3.1e 3:7e6619d4:::rbd_data.115274ed0aa8.0000000000000068:head [set-alloc-hint object_size 1048576 write_size 1048576,write 835584~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:28.249 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:28.249 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 24.304687372, 2024-09-14T06:17:28.249 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096039367, 2024-09-14T06:17:28.249 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.249 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.249 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:28.250 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.250 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.250 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.250 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32044 2024-09-14T06:17:28.250 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.250 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.250 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.250 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.250 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:28.250 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.250 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.250 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.250 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.250 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:28.251 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.251 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.251 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.251 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.251 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876320+0000", 2024-09-14T06:17:28.251 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8270000000000002e-06 2024-09-14T06:17:28.251 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.251 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.251 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.251 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876397+0000", 2024-09-14T06:17:28.251 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6985999999999998e-05 2024-09-14T06:17:28.251 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.251 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.251 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.252 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876431+0000", 2024-09-14T06:17:28.252 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3719e-05 2024-09-14T06:17:28.252 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.252 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.252 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.252 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876452+0000", 2024-09-14T06:17:28.252 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0891999999999999e-05 2024-09-14T06:17:28.252 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.252 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.252 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.252 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969186+0000", 2024-09-14T06:17:28.253 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.092734681999999999 2024-09-14T06:17:28.253 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.253 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.253 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.253 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969550+0000", 2024-09-14T06:17:28.253 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036402199999999998 2024-09-14T06:17:28.254 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.254 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.254 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:17:28.254 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969987+0000", 2024-09-14T06:17:28.254 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000436759 2024-09-14T06:17:28.254 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.255 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.255 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:28.255 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971569+0000", 2024-09-14T06:17:28.255 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001581821 2024-09-14T06:17:28.255 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.256 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.256 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:28.256 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971947+0000", 2024-09-14T06:17:28.256 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378446 2024-09-14T06:17:28.256 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.256 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.257 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:28.257 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972146+0000", 2024-09-14T06:17:28.257 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019819 2024-09-14T06:17:28.257 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.257 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.257 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.258 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972357+0000", 2024-09-14T06:17:28.258 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021102299999999999 2024-09-14T06:17:28.258 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.258 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.258 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.259 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.259 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.259 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40252 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 827392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:28.259 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:28.259 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 18.464051265999998, 2024-09-14T06:17:28.259 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095633474999999996, 2024-09-14T06:17:28.259 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.260 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.260 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.260 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.260 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.260 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.260 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40252 2024-09-14T06:17:28.260 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.261 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.261 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.261 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.261 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:28.261 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.261 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.261 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.262 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.262 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:28.262 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.262 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.262 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.262 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.263 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717376+0000", 2024-09-14T06:17:28.263 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000422216 2024-09-14T06:17:28.263 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.263 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.263 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.263 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717386+0000", 2024-09-14T06:17:28.263 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0396e-05 2024-09-14T06:17:28.263 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717395+0000", 2024-09-14T06:17:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7819999999999999e-06 2024-09-14T06:17:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.264 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.265 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717400+0000", 2024-09-14T06:17:28.265 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1599999999999997e-06 2024-09-14T06:17:28.265 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.265 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.266 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.266 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.811922+0000", 2024-09-14T06:17:28.266 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.094522018999999999 2024-09-14T06:17:28.266 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.266 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.266 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.266 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812007+0000", 2024-09-14T06:17:28.267 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5328999999999996e-05 2024-09-14T06:17:28.267 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.267 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.267 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.267 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812587+0000", 2024-09-14T06:17:28.267 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000579573 2024-09-14T06:17:28.267 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.267 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.267 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.267 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.267 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.267 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40254 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:28.267 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:28.268 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 18.463593093, 2024-09-14T06:17:28.268 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095562731999999997, 2024-09-14T06:17:28.268 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.268 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.268 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.268 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.268 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.268 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.268 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40254 2024-09-14T06:17:28.268 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.268 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.268 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.268 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.269 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:28.269 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.269 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.269 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.269 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.269 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:28.269 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.269 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.269 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.269 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.269 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:28.269 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1699999999999999e-07 2024-09-14T06:17:28.270 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.270 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.270 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.270 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717414+0000", 2024-09-14T06:17:28.270 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:17:28.270 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.270 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.270 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.270 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717419+0000", 2024-09-14T06:17:28.270 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2839999999999997e-06 2024-09-14T06:17:28.270 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.270 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.270 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.270 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717424+0000", 2024-09-14T06:17:28.271 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1549999999999998e-06 2024-09-14T06:17:28.271 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.271 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.271 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.271 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812619+0000", 2024-09-14T06:17:28.271 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095194940000000006 2024-09-14T06:17:28.271 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.271 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.272 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.272 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812717+0000", 2024-09-14T06:17:28.272 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7647999999999997e-05 2024-09-14T06:17:28.272 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.272 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.272 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.272 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812974+0000", 2024-09-14T06:17:28.272 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025752099999999998 2024-09-14T06:17:28.272 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.272 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.273 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.273 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.273 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.273 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40256 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 843776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:28.273 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:28.273 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 18.463570488999999, 2024-09-14T06:17:28.273 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095932066999999996, 2024-09-14T06:17:28.273 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.273 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.273 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.273 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.274 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.274 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.274 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40256 2024-09-14T06:17:28.274 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.274 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.274 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.274 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.274 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:28.274 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.274 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.274 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.274 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.274 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:28.274 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.275 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.275 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.275 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.275 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717628+0000", 2024-09-14T06:17:28.275 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019339499999999999 2024-09-14T06:17:28.275 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.275 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.275 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.275 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717632+0000", 2024-09-14T06:17:28.275 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2230000000000001e-06 2024-09-14T06:17:28.275 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.275 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.275 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.275 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717641+0000", 2024-09-14T06:17:28.276 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6309999999999998e-06 2024-09-14T06:17:28.276 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.276 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.276 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.276 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717656+0000", 2024-09-14T06:17:28.276 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4724e-05 2024-09-14T06:17:28.276 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.276 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.276 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.276 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813006+0000", 2024-09-14T06:17:28.276 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095349801999999997 2024-09-14T06:17:28.276 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.276 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.276 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.277 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813107+0000", 2024-09-14T06:17:28.277 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000100973 2024-09-14T06:17:28.277 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.277 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.277 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.277 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813366+0000", 2024-09-14T06:17:28.277 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025931899999999999 2024-09-14T06:17:28.277 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.277 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.277 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.278 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.278 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.278 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40258 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 851968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:28.278 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:28.278 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 18.463334389, 2024-09-14T06:17:28.278 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096062543, 2024-09-14T06:17:28.278 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.278 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.278 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.278 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.278 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.278 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.279 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40258 2024-09-14T06:17:28.279 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.279 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.279 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.279 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.279 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:28.279 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.279 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.279 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.279 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.279 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:28.279 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.279 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.279 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.280 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.280 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717671+0000", 2024-09-14T06:17:28.280 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8299999999999997e-07 2024-09-14T06:17:28.280 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.280 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.280 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.280 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717674+0000", 2024-09-14T06:17:28.280 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7080000000000002e-06 2024-09-14T06:17:28.280 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.280 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.280 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.280 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717681+0000", 2024-09-14T06:17:28.280 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7889999999999995e-06 2024-09-14T06:17:28.280 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.281 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.281 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.281 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717687+0000", 2024-09-14T06:17:28.281 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7529999999999998e-06 2024-09-14T06:17:28.281 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.281 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.281 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.281 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813408+0000", 2024-09-14T06:17:28.281 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095720451999999998 2024-09-14T06:17:28.281 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.281 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.281 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.281 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813499+0000", 2024-09-14T06:17:28.281 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0933000000000005e-05 2024-09-14T06:17:28.282 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.282 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.282 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.282 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813733+0000", 2024-09-14T06:17:28.282 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00023432499999999999 2024-09-14T06:17:28.282 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.282 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.282 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.282 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.282 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.282 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40260 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 860160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:28.282 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:28.283 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 18.46330854, 2024-09-14T06:17:28.283 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096337381999999999, 2024-09-14T06:17:28.283 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.283 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.283 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.283 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.283 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.283 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.283 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40260 2024-09-14T06:17:28.283 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.283 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.283 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.283 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.283 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:28.283 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.284 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.284 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.284 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.284 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:28.284 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.284 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.284 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.284 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.284 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718008+0000", 2024-09-14T06:17:28.284 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00031192699999999998 2024-09-14T06:17:28.284 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.284 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.284 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.285 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718013+0000", 2024-09-14T06:17:28.285 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1830000000000002e-06 2024-09-14T06:17:28.285 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.285 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.285 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.285 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718036+0000", 2024-09-14T06:17:28.285 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2721000000000002e-05 2024-09-14T06:17:28.285 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.285 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.285 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.285 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718053+0000", 2024-09-14T06:17:28.285 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6736e-05 2024-09-14T06:17:28.285 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.285 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.286 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.286 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813767+0000", 2024-09-14T06:17:28.286 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095714658999999994 2024-09-14T06:17:28.286 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.286 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.286 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.286 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813816+0000", 2024-09-14T06:17:28.286 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8848999999999999e-05 2024-09-14T06:17:28.286 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.286 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.286 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.286 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814034+0000", 2024-09-14T06:17:28.286 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000217307 2024-09-14T06:17:28.286 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.287 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.287 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.287 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.287 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.287 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40262 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 868352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:28.287 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:28.287 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 18.462908856999999, 2024-09-14T06:17:28.287 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096354165000000006, 2024-09-14T06:17:28.287 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.287 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.287 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.287 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.287 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.287 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.288 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40262 2024-09-14T06:17:28.288 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.288 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.288 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.288 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.288 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:28.288 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.288 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.288 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.288 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.288 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:28.288 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.288 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.288 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.289 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.289 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718281+0000", 2024-09-14T06:17:28.289 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000185069 2024-09-14T06:17:28.289 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.289 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.289 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.289 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718286+0000", 2024-09-14T06:17:28.289 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.118e-06 2024-09-14T06:17:28.289 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.289 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.289 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.289 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718295+0000", 2024-09-14T06:17:28.289 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0429999999999996e-06 2024-09-14T06:17:28.289 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.290 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.290 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.290 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718302+0000", 2024-09-14T06:17:28.290 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2790000000000001e-06 2024-09-14T06:17:28.290 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.290 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.290 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.290 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814076+0000", 2024-09-14T06:17:28.290 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095774076999999999 2024-09-14T06:17:28.290 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.290 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.290 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.290 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814191+0000", 2024-09-14T06:17:28.290 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00011500700000000001 2024-09-14T06:17:28.290 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.291 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.291 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.291 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814450+0000", 2024-09-14T06:17:28.291 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025857199999999999 2024-09-14T06:17:28.291 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.291 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.291 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.291 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.291 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.291 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40264 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 876544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:28.291 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:28.291 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 18.462551788999999, 2024-09-14T06:17:28.291 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096372478999999997, 2024-09-14T06:17:28.291 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.292 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.292 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:28.292 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.292 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.292 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.292 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40264 2024-09-14T06:17:28.292 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.292 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.292 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.292 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.292 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:28.292 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.292 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.292 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.293 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.293 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:28.293 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.293 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.293 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.293 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.293 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718793+0000", 2024-09-14T06:17:28.293 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00033991099999999998 2024-09-14T06:17:28.293 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.293 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.293 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.293 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718805+0000", 2024-09-14T06:17:28.293 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2514000000000001e-05 2024-09-14T06:17:28.293 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.293 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.294 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.294 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718815+0000", 2024-09-14T06:17:28.294 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9590000000000007e-06 2024-09-14T06:17:28.294 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.294 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.294 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.294 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718822+0000", 2024-09-14T06:17:28.294 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2030000000000003e-06 2024-09-14T06:17:28.294 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.294 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.294 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.294 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814490+0000", 2024-09-14T06:17:28.294 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095667346 2024-09-14T06:17:28.294 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.294 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.295 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.295 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814577+0000", 2024-09-14T06:17:28.295 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6920000000000001e-05 2024-09-14T06:17:28.295 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.295 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.295 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.295 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814825+0000", 2024-09-14T06:17:28.295 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000248626 2024-09-14T06:17:28.295 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.295 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.295 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.295 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.295 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.295 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:28.298 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:17:28.298 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:28.293+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:28.298 INFO:teuthology.orchestra.run.smithi039.stdout:4833 write 0x3c94180 thru 0x3c98672 (0x44f3 bytes) 2024-09-14T06:17:28.299 INFO:teuthology.orchestra.run.smithi039.stdout:4834 writesame 0x47f9adc thru 0x47ff9fa data_size 0x1(0x5f1f bytes) 2024-09-14T06:17:28.299 INFO:teuthology.orchestra.run.smithi039.stdout:4835 writesame 0x8452948 thru 0x845f3a3 data_size 0x12(0xca5c bytes) 2024-09-14T06:17:28.299 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:28.299 INFO:teuthology.orchestra.run.smithi039.stdout:4838 write 0x4c4d1e5 thru 0x4c54e56 (0x7c72 bytes) 2024-09-14T06:17:28.299 INFO:teuthology.orchestra.run.smithi039.stdout:4840 write 0xb1d968f thru 0xb1dfbe1 (0x6553 bytes) 2024-09-14T06:17:28.299 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:28.299 INFO:teuthology.orchestra.run.smithi039.stdout:4846 write 0x51ab44f thru 0x51b99f0 (0xe5a2 bytes) 2024-09-14T06:17:28.301 INFO:teuthology.orchestra.run.smithi039.stdout:4847 writesame 0x9daa5c1 thru 0x9db79c6 data_size 0x7(0xd406 bytes) 2024-09-14T06:17:28.317 INFO:teuthology.orchestra.run.smithi039.stdout:4848 write 0xa2a7129 thru 0xa2a9efb (0x2dd3 bytes) 2024-09-14T06:17:28.333 INFO:teuthology.orchestra.run.smithi039.stdout:4850 trunc from 0xb1dfbe2 to 0xdc31ca7 2024-09-14T06:17:28.359 INFO:teuthology.orchestra.run.smithi039.stdout:4852 trunc from 0xdc31ca7 to 0x922b0a6 2024-09-14T06:17:28.386 INFO:teuthology.orchestra.run.smithi039.stdout:4853 read 0x886099e thru 0x88660ff (0x5762 bytes) 2024-09-14T06:17:28.387 INFO:teuthology.orchestra.run.smithi039.stdout:4854 writesame 0x6d42db7 thru 0x6d4bd2a data_size 0x2(0x8f74 bytes) 2024-09-14T06:17:28.397 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:28.394+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:28.418 INFO:teuthology.orchestra.run.smithi039.stdout:4855 read 0x47083cf thru 0x471764c (0xf27e bytes) 2024-09-14T06:17:28.418 INFO:teuthology.orchestra.run.smithi039.stdout:4856 read 0x6432e18 thru 0x643810d (0x52f6 bytes) 2024-09-14T06:17:28.418 INFO:teuthology.orchestra.run.smithi039.stdout:4857 write 0x75bc5ee thru 0x75c8523 (0xbf36 bytes) 2024-09-14T06:17:28.429 INFO:teuthology.orchestra.run.smithi039.stdout:4858 punch from 0x3abacee to 0x3ac5174, (0xa486 bytes) 2024-09-14T06:17:28.430 INFO:teuthology.orchestra.run.smithi039.stdout:4859 writesame 0x3def7a8 thru 0x3dfdd38 data_size 0x1(0xe591 bytes) 2024-09-14T06:17:28.473 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:28.474 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:28.474 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:28.474 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:28.488 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:17:28.507 INFO:teuthology.orchestra.run.smithi039.stdout:4862 read 0x7791763 thru 0x77932bd (0x1b5b bytes) 2024-09-14T06:17:28.507 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:28.508 INFO:teuthology.orchestra.run.smithi039.stdout:4864 read 0x20a3e7f thru 0x20afc1d (0xbd9f bytes) 2024-09-14T06:17:28.517 INFO:teuthology.orchestra.run.smithi039.stdout:4865 trunc from 0x922b0a6 to 0x5cd2c6d 2024-09-14T06:17:28.535 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:28.533+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:28.557 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:17:28.573 INFO:teuthology.orchestra.run.smithi039.stdout:4866 write 0xcf7edf3 thru 0xcf80254 (0x1462 bytes) 2024-09-14T06:17:28.589 INFO:teuthology.orchestra.run.smithi039.stdout:4868 trunc from 0xcf80255 to 0x9d8fa9a 2024-09-14T06:17:28.599 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:28.598+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:28.605 INFO:teuthology.orchestra.run.smithi039.stdout:4869 punch from 0x453c073 to 0x4542e2f, (0x6dbc bytes) 2024-09-14T06:17:28.606 INFO:teuthology.orchestra.run.smithi039.stdout:4872 punch from 0x76203d8 to 0x7621326, (0xf4e bytes) 2024-09-14T06:17:28.606 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:28.606 INFO:teuthology.orchestra.run.smithi039.stdout:4874 read 0x54aed8d thru 0x54b3060 (0x42d4 bytes) 2024-09-14T06:17:28.606 INFO:teuthology.orchestra.run.smithi039.stdout:4875 writesame 0x891f631 thru 0x8927584 data_size 0x1(0x7f54 bytes) 2024-09-14T06:17:28.614 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:28.615 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:28.615 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:28.615 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:28.615 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:28.630 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:17:28.648 INFO:teuthology.orchestra.run.smithi039.stdout:4876 read 0x95b9ae0 thru 0x95c78f0 (0xde11 bytes) 2024-09-14T06:17:28.648 INFO:teuthology.orchestra.run.smithi039.stdout:4877 read 0x2302bb3 thru 0x230e025 (0xb473 bytes) 2024-09-14T06:17:28.662 INFO:teuthology.orchestra.run.smithi039.stdout:4879 trunc from 0x9d8fa9a to 0x39069de 2024-09-14T06:17:28.709 INFO:teuthology.orchestra.run.smithi039.stdout:4880 write 0xb5f40bb thru 0xb6028c8 (0xe80e bytes) 2024-09-14T06:17:28.719 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:28.717+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:28.730 INFO:teuthology.orchestra.run.smithi039.stdout:4883 trunc from 0xb6028c9 to 0xb5341d 2024-09-14T06:17:28.743 INFO:teuthology.orchestra.run.smithi039.stdout:4884 punch from 0x20d59 to 0x237fd, (0x2aa4 bytes) 2024-09-14T06:17:28.744 INFO:teuthology.orchestra.run.smithi039.stdout:4885 read 0xad9496 thru 0xae820e (0xed79 bytes) 2024-09-14T06:17:28.744 INFO:teuthology.orchestra.run.smithi039.stdout:4886 punch from 0x25fe0a to 0x26ee59, (0xf04f bytes) 2024-09-14T06:17:28.768 INFO:teuthology.orchestra.run.smithi039.stdout:4888 write 0xd2b2800 thru 0xd2b5b27 (0x3328 bytes) 2024-09-14T06:17:28.772 INFO:teuthology.orchestra.run.smithi039.stdout:4889 punch from 0x9aa6e31 to 0x9ab57c5, (0xe994 bytes) 2024-09-14T06:17:28.772 INFO:teuthology.orchestra.run.smithi039.stdout:4890 write 0x126e763 thru 0x127454f (0x5ded bytes) 2024-09-14T06:17:28.776 INFO:teuthology.orchestra.run.smithi039.stdout:4892 trunc from 0xd2b5b28 to 0xcabba55 2024-09-14T06:17:28.791 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:28.800 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:28.800 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:28.800 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:28.800 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:28.801 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.801 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:3595 3.1 3:811c1c24:::rbd_data.1152bd92e04f.000000000000006e:head [set-alloc-hint object_size 2097152 write_size 2097152,write 440183~2185] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T06:17:28.801 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:28.801 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 62.575761866999997, 2024-09-14T06:17:28.801 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0365806689999999, 2024-09-14T06:17:28.801 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.801 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.801 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.801 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.801 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.801 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.801 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 3595 2024-09-14T06:17:28.802 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.802 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.802 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.802 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.802 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:28.802 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.802 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.802 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.802 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.802 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:28.802 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.802 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.802 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.803 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.803 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222823+0000", 2024-09-14T06:17:28.803 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8219999999999998e-06 2024-09-14T06:17:28.803 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.803 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.803 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.803 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222835+0000", 2024-09-14T06:17:28.803 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2052000000000001e-05 2024-09-14T06:17:28.803 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.803 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.803 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.803 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222859+0000", 2024-09-14T06:17:28.803 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3694e-05 2024-09-14T06:17:28.804 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.804 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.804 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.804 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222872+0000", 2024-09-14T06:17:28.804 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3203999999999999e-05 2024-09-14T06:17:28.804 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.804 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.804 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.804 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223068+0000", 2024-09-14T06:17:28.804 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000195995 2024-09-14T06:17:28.804 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.804 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.804 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:28.805 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223312+0000", 2024-09-14T06:17:28.805 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024424199999999998 2024-09-14T06:17:28.805 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.805 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.805 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:28.805 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223786+0000", 2024-09-14T06:17:28.805 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00047356600000000003 2024-09-14T06:17:28.805 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.805 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.805 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:28.805 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.225374+0000", 2024-09-14T06:17:28.805 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015875150000000001 2024-09-14T06:17:28.805 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.805 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.806 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.806 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.233122+0000", 2024-09-14T06:17:28.806 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0077480700000000001 2024-09-14T06:17:28.806 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.806 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.806 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:17:28.806 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.233157+0000", 2024-09-14T06:17:28.806 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5113000000000001e-05 2024-09-14T06:17:28.806 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.806 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.806 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.806 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.252998+0000", 2024-09-14T06:17:28.806 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.019841142 2024-09-14T06:17:28.807 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.807 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.807 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for degraded object", 2024-09-14T06:17:28.807 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.254121+0000", 2024-09-14T06:17:28.807 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0011227979999999999 2024-09-14T06:17:28.807 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.807 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.807 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.807 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.259263+0000", 2024-09-14T06:17:28.807 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0051426700000000002 2024-09-14T06:17:28.807 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.807 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.807 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.808 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.259401+0000", 2024-09-14T06:17:28.808 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013778600000000001 2024-09-14T06:17:28.808 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.808 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.808 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.808 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.808 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.808 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11421 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.808 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:28.808 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.812866681000003, 2024-09-14T06:17:28.808 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078553170000000005, 2024-09-14T06:17:28.808 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.808 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.809 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.809 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.809 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.809 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.809 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11421 2024-09-14T06:17:28.809 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.809 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.809 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.809 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.809 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:28.809 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.809 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.809 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.809 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.809 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:28.810 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.810 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.810 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.810 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.810 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985718+0000", 2024-09-14T06:17:28.810 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8840000000000001e-06 2024-09-14T06:17:28.810 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.810 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.810 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.810 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985723+0000", 2024-09-14T06:17:28.810 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8799999999999996e-06 2024-09-14T06:17:28.810 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.810 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.810 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.811 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985742+0000", 2024-09-14T06:17:28.811 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8776000000000001e-05 2024-09-14T06:17:28.811 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.811 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.811 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.811 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985759+0000", 2024-09-14T06:17:28.811 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7119999999999999e-05 2024-09-14T06:17:28.811 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.812 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.812 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.812 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064259+0000", 2024-09-14T06:17:28.812 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078499418000000001 2024-09-14T06:17:28.812 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.812 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.812 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.812 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064269+0000", 2024-09-14T06:17:28.812 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:17:28.812 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.812 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.813 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.813 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.813 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.813 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11422 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.813 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:28.813 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.812796714000001, 2024-09-14T06:17:28.813 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078521540000000001, 2024-09-14T06:17:28.813 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.813 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.813 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.813 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.813 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.813 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.814 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11422 2024-09-14T06:17:28.814 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.814 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.814 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.814 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.814 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:28.814 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.814 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.814 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.814 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.814 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:28.814 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.814 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.815 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.815 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.815 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985787+0000", 2024-09-14T06:17:28.815 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7630000000000001e-06 2024-09-14T06:17:28.815 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.815 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.815 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.815 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985842+0000", 2024-09-14T06:17:28.815 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4555999999999997e-05 2024-09-14T06:17:28.815 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.815 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.815 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.815 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985864+0000", 2024-09-14T06:17:28.816 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2189999999999999e-05 2024-09-14T06:17:28.816 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.816 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.816 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.816 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985879+0000", 2024-09-14T06:17:28.816 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4382999999999999e-05 2024-09-14T06:17:28.816 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.816 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.816 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.816 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064298+0000", 2024-09-14T06:17:28.816 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078419121999999994 2024-09-14T06:17:28.816 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.816 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.816 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.817 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064307+0000", 2024-09-14T06:17:28.817 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5259999999999999e-06 2024-09-14T06:17:28.817 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.817 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.817 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.817 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.817 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.817 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11423 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.817 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:28.817 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.812679342000003, 2024-09-14T06:17:28.817 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078437175999999997, 2024-09-14T06:17:28.817 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.817 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.818 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.818 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.818 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.818 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.818 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11423 2024-09-14T06:17:28.818 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.818 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.818 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.818 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.818 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:28.818 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.818 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.818 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.818 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.819 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:28.819 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.819 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.819 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.819 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.819 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985905+0000", 2024-09-14T06:17:28.819 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8139999999999999e-06 2024-09-14T06:17:28.819 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.819 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.819 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.819 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985911+0000", 2024-09-14T06:17:28.819 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7230000000000001e-06 2024-09-14T06:17:28.819 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.819 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.820 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.820 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985930+0000", 2024-09-14T06:17:28.820 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9360000000000001e-05 2024-09-14T06:17:28.820 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.820 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.820 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.820 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985944+0000", 2024-09-14T06:17:28.820 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4479e-05 2024-09-14T06:17:28.820 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.820 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.820 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.820 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064332+0000", 2024-09-14T06:17:28.820 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078387686999999998 2024-09-14T06:17:28.821 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.821 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.821 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.821 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064340+0000", 2024-09-14T06:17:28.821 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1130000000000001e-06 2024-09-14T06:17:28.821 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.821 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.821 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.821 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.821 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.821 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11480 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.821 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:28.821 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.811328865999997, 2024-09-14T06:17:28.821 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077122298000000006, 2024-09-14T06:17:28.822 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.822 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.822 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.822 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.822 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.822 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.822 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11480 2024-09-14T06:17:28.822 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.822 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.822 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.822 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.822 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:28.822 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.822 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.823 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.823 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.823 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:28.823 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.823 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.823 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.823 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.823 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987255+0000", 2024-09-14T06:17:28.823 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:17:28.823 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.823 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.823 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.823 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987263+0000", 2024-09-14T06:17:28.824 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6499999999999996e-06 2024-09-14T06:17:28.824 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.824 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.824 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.824 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987285+0000", 2024-09-14T06:17:28.824 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1860000000000001e-05 2024-09-14T06:17:28.824 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.824 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.824 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.824 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987341+0000", 2024-09-14T06:17:28.824 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6648999999999999e-05 2024-09-14T06:17:28.824 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.824 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.825 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.825 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064367+0000", 2024-09-14T06:17:28.825 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077025622000000002 2024-09-14T06:17:28.825 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.825 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.825 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.825 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064376+0000", 2024-09-14T06:17:28.825 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7499999999999992e-06 2024-09-14T06:17:28.825 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.825 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.825 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.825 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.825 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.825 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11481 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.826 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:28.826 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.811193164000002, 2024-09-14T06:17:28.826 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077022057000000005, 2024-09-14T06:17:28.826 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.826 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.826 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.826 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.826 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.826 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.826 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11481 2024-09-14T06:17:28.826 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.826 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.826 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.826 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.827 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:28.827 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.827 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.827 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.827 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.827 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:28.827 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.827 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.827 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.827 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.827 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987391+0000", 2024-09-14T06:17:28.827 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4839999999999999e-06 2024-09-14T06:17:28.827 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.828 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.828 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.828 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987399+0000", 2024-09-14T06:17:28.828 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8660000000000006e-06 2024-09-14T06:17:28.828 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.828 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.828 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.828 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987419+0000", 2024-09-14T06:17:28.828 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9984e-05 2024-09-14T06:17:28.828 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.828 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.828 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.828 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987434+0000", 2024-09-14T06:17:28.829 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5396e-05 2024-09-14T06:17:28.829 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.829 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.829 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.829 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064403+0000", 2024-09-14T06:17:28.829 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076969212999999995 2024-09-14T06:17:28.829 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.829 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.829 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.829 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064411+0000", 2024-09-14T06:17:28.829 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1140000000000006e-06 2024-09-14T06:17:28.829 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.829 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.829 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.829 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.830 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.830 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11482 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.830 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:28.830 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.811108304000001, 2024-09-14T06:17:28.830 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076971172000000004, 2024-09-14T06:17:28.830 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.830 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.830 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.830 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.830 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.830 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.830 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11482 2024-09-14T06:17:28.830 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.831 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.831 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.831 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.831 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:28.831 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.831 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.831 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.831 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.831 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:28.831 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.831 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.831 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.831 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.832 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987476+0000", 2024-09-14T06:17:28.832 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.793e-06 2024-09-14T06:17:28.832 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.832 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.832 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.832 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987482+0000", 2024-09-14T06:17:28.832 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8270000000000003e-06 2024-09-14T06:17:28.832 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.832 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.832 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.832 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987502+0000", 2024-09-14T06:17:28.832 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0656e-05 2024-09-14T06:17:28.832 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.832 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.833 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.833 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987518+0000", 2024-09-14T06:17:28.833 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:17:28.833 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.833 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.833 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.833 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064438+0000", 2024-09-14T06:17:28.833 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076919623000000006 2024-09-14T06:17:28.833 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.833 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.833 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.833 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064445+0000", 2024-09-14T06:17:28.833 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.61e-06 2024-09-14T06:17:28.834 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.834 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.834 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.834 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.834 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.834 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11483 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.834 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:28.834 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.811040454999997, 2024-09-14T06:17:28.834 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076936562999999999, 2024-09-14T06:17:28.834 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.834 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.834 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.834 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.834 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.835 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.835 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11483 2024-09-14T06:17:28.835 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.835 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.835 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.835 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.835 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:28.835 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.835 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.835 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.835 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.835 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:28.836 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.836 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.836 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.836 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.836 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987544+0000", 2024-09-14T06:17:28.836 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.654e-06 2024-09-14T06:17:28.836 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.836 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.836 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.836 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987550+0000", 2024-09-14T06:17:28.836 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0299999999999999e-06 2024-09-14T06:17:28.836 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.837 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.837 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.837 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987570+0000", 2024-09-14T06:17:28.837 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0080000000000001e-05 2024-09-14T06:17:28.837 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.837 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.837 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.837 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987619+0000", 2024-09-14T06:17:28.837 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9228999999999999e-05 2024-09-14T06:17:28.837 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.837 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.837 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.837 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064471+0000", 2024-09-14T06:17:28.837 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076852086999999999 2024-09-14T06:17:28.838 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.838 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.838 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.838 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064478+0000", 2024-09-14T06:17:28.838 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4830000000000002e-06 2024-09-14T06:17:28.838 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.838 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.838 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.838 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.838 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.838 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11485 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.838 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:28.838 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.810941962000001, 2024-09-14T06:17:28.839 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076868463999999997, 2024-09-14T06:17:28.839 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.839 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.839 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.839 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.839 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.839 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.839 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11485 2024-09-14T06:17:28.839 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.839 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.839 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.839 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.839 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:28.839 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.840 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.840 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.840 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.840 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:28.840 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.840 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.840 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.840 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.840 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987642+0000", 2024-09-14T06:17:28.840 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4100000000000001e-06 2024-09-14T06:17:28.840 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.840 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.840 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.840 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987648+0000", 2024-09-14T06:17:28.841 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2500000000000003e-06 2024-09-14T06:17:28.841 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.841 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.841 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.841 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987683+0000", 2024-09-14T06:17:28.841 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5193e-05 2024-09-14T06:17:28.841 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.841 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.841 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.841 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987698+0000", 2024-09-14T06:17:28.841 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4893e-05 2024-09-14T06:17:28.841 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.841 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.842 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.842 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064501+0000", 2024-09-14T06:17:28.842 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076803263999999996 2024-09-14T06:17:28.842 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.842 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.842 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.842 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064509+0000", 2024-09-14T06:17:28.842 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4540000000000001e-06 2024-09-14T06:17:28.842 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.842 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.842 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.842 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.842 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.842 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11490 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.843 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:28.843 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.810792393, 2024-09-14T06:17:28.843 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076755043999999994, 2024-09-14T06:17:28.843 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.843 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.843 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.843 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.843 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.843 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.843 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11490 2024-09-14T06:17:28.843 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.843 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.843 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.843 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.844 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:28.844 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.844 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.844 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.844 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.844 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:28.844 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.844 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.844 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.844 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.844 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987791+0000", 2024-09-14T06:17:28.844 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:17:28.844 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.845 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.845 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.845 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987797+0000", 2024-09-14T06:17:28.845 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7969999999999997e-06 2024-09-14T06:17:28.845 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.845 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.845 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.845 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987817+0000", 2024-09-14T06:17:28.845 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9746e-05 2024-09-14T06:17:28.845 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.845 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.845 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.845 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987832+0000", 2024-09-14T06:17:28.846 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.452e-05 2024-09-14T06:17:28.846 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.846 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.846 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.846 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064535+0000", 2024-09-14T06:17:28.846 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076703478000000005 2024-09-14T06:17:28.846 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.846 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.846 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.846 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064545+0000", 2024-09-14T06:17:28.846 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9799999999999993e-06 2024-09-14T06:17:28.846 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.846 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.847 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.847 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.847 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.847 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11492 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.847 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:28.847 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.810665528000001, 2024-09-14T06:17:28.847 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076661759999999995, 2024-09-14T06:17:28.847 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.847 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.847 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.847 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.847 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.847 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.847 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11492 2024-09-14T06:17:28.848 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.848 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.848 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.848 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.848 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:28.848 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.848 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.848 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.848 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.848 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:28.848 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.848 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.848 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.849 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.849 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987918+0000", 2024-09-14T06:17:28.849 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3239999999999999e-06 2024-09-14T06:17:28.849 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.849 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.849 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.849 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987924+0000", 2024-09-14T06:17:28.849 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6930000000000004e-06 2024-09-14T06:17:28.849 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.849 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.849 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.849 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987943+0000", 2024-09-14T06:17:28.850 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9487000000000002e-05 2024-09-14T06:17:28.850 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.850 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.850 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.850 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987958+0000", 2024-09-14T06:17:28.850 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4453e-05 2024-09-14T06:17:28.850 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.850 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.850 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.850 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064571+0000", 2024-09-14T06:17:28.850 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076612988000000007 2024-09-14T06:17:28.850 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.850 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.850 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.851 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064579+0000", 2024-09-14T06:17:28.851 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8150000000000006e-06 2024-09-14T06:17:28.851 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.851 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.851 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.851 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.851 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.851 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11495 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.851 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:28.851 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.810496839000002, 2024-09-14T06:17:28.851 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076524746000000005, 2024-09-14T06:17:28.851 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.851 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.851 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.852 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.852 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.852 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.852 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11495 2024-09-14T06:17:28.852 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.852 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.852 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.852 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.853 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:28.853 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.853 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.853 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.853 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.853 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:28.853 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.853 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.853 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.853 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.853 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988087+0000", 2024-09-14T06:17:28.853 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0499999999999999e-06 2024-09-14T06:17:28.853 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.854 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.854 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.854 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988118+0000", 2024-09-14T06:17:28.854 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1900000000000003e-05 2024-09-14T06:17:28.854 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.854 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.854 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.854 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988138+0000", 2024-09-14T06:17:28.854 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.933e-05 2024-09-14T06:17:28.854 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.854 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.854 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.854 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988152+0000", 2024-09-14T06:17:28.854 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3889e-05 2024-09-14T06:17:28.855 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.855 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.855 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.855 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064603+0000", 2024-09-14T06:17:28.855 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076451082000000004 2024-09-14T06:17:28.855 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.855 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.855 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.855 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064610+0000", 2024-09-14T06:17:28.855 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4950000000000002e-06 2024-09-14T06:17:28.855 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.855 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.855 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.856 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.856 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11498 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.856 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:28.856 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.810410763, 2024-09-14T06:17:28.856 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076471883000000004, 2024-09-14T06:17:28.856 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.856 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.856 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.856 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.856 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.856 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.856 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11498 2024-09-14T06:17:28.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.857 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.857 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.857 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.857 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:28.857 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.857 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.857 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.857 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.857 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:28.857 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.857 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.857 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.857 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.858 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988173+0000", 2024-09-14T06:17:28.858 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2300000000000001e-06 2024-09-14T06:17:28.858 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.858 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.858 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.858 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988178+0000", 2024-09-14T06:17:28.858 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4229999999999996e-06 2024-09-14T06:17:28.858 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.858 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.858 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.858 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988235+0000", 2024-09-14T06:17:28.858 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6616999999999997e-05 2024-09-14T06:17:28.858 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.858 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.859 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.859 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988264+0000", 2024-09-14T06:17:28.859 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9189000000000001e-05 2024-09-14T06:17:28.859 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.859 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.859 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.859 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064636+0000", 2024-09-14T06:17:28.859 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076371573999999998 2024-09-14T06:17:28.859 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.859 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.859 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.859 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064643+0000", 2024-09-14T06:17:28.859 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8499999999999994e-06 2024-09-14T06:17:28.860 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.860 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.860 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.860 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.860 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.860 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11523 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.860 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:28.860 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.809690615000001, 2024-09-14T06:17:28.860 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076062454000000002, 2024-09-14T06:17:28.860 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.860 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.860 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.861 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.861 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.861 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.861 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11523 2024-09-14T06:17:28.861 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.861 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.861 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.861 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.861 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:28.861 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.861 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.861 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.861 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.862 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:28.862 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.862 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.862 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.862 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.862 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988894+0000", 2024-09-14T06:17:28.862 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.843e-06 2024-09-14T06:17:28.862 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.862 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.862 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.862 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988903+0000", 2024-09-14T06:17:28.862 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0440000000000001e-06 2024-09-14T06:17:28.862 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.862 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.863 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.863 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988929+0000", 2024-09-14T06:17:28.863 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6679e-05 2024-09-14T06:17:28.863 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.863 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.863 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.863 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988967+0000", 2024-09-14T06:17:28.863 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7453000000000002e-05 2024-09-14T06:17:28.863 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.863 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.863 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.863 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064668+0000", 2024-09-14T06:17:28.863 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075701389999999993 2024-09-14T06:17:28.864 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.864 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.864 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.864 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064954+0000", 2024-09-14T06:17:28.864 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028604500000000002 2024-09-14T06:17:28.864 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.864 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.864 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.864 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.864 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.864 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11530 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.864 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:28.864 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.809200064999999, 2024-09-14T06:17:28.865 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075620674999999998, 2024-09-14T06:17:28.865 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.865 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.865 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.865 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.865 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.865 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.865 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11530 2024-09-14T06:17:28.865 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.865 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.865 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.865 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.866 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:28.866 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.866 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.866 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.866 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.866 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:28.866 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.866 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.866 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.866 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.866 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989384+0000", 2024-09-14T06:17:28.866 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6160000000000001e-06 2024-09-14T06:17:28.867 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.867 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.867 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.867 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989390+0000", 2024-09-14T06:17:28.867 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7039999999999999e-06 2024-09-14T06:17:28.867 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.867 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.867 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.867 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989409+0000", 2024-09-14T06:17:28.867 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.95e-05 2024-09-14T06:17:28.867 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.867 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.867 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.868 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989429+0000", 2024-09-14T06:17:28.868 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0173000000000002e-05 2024-09-14T06:17:28.868 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.868 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.868 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.868 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064992+0000", 2024-09-14T06:17:28.868 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075562764000000004 2024-09-14T06:17:28.868 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.868 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.868 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.868 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065003+0000", 2024-09-14T06:17:28.868 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0917999999999999e-05 2024-09-14T06:17:28.869 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.869 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.869 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.869 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.869 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.869 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11532 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.869 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:28.869 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.809129669000001, 2024-09-14T06:17:28.869 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075604656000000006, 2024-09-14T06:17:28.869 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.869 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.869 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.869 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.869 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.870 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.870 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11532 2024-09-14T06:17:28.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.870 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.870 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.870 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:28.870 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.870 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.870 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:28.870 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.871 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.871 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989454+0000", 2024-09-14T06:17:28.871 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6500000000000001e-06 2024-09-14T06:17:28.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.871 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.871 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.871 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989460+0000", 2024-09-14T06:17:28.871 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7459999999999997e-06 2024-09-14T06:17:28.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.871 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.871 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.871 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989480+0000", 2024-09-14T06:17:28.871 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.948e-05 2024-09-14T06:17:28.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.872 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.872 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.872 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989497+0000", 2024-09-14T06:17:28.872 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7153e-05 2024-09-14T06:17:28.872 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.872 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.872 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.872 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065046+0000", 2024-09-14T06:17:28.872 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075548905999999999 2024-09-14T06:17:28.872 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.872 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.872 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.872 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065057+0000", 2024-09-14T06:17:28.873 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1721e-05 2024-09-14T06:17:28.873 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.873 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.873 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.873 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.873 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11534 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.873 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:28.873 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.809063182999999, 2024-09-14T06:17:28.873 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075574350999999998, 2024-09-14T06:17:28.873 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.873 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.874 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.874 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.874 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.874 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.874 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11534 2024-09-14T06:17:28.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.874 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.874 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.874 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:28.874 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.875 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.875 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:28.875 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.875 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.875 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.875 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989521+0000", 2024-09-14T06:17:28.875 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.533e-06 2024-09-14T06:17:28.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.875 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.875 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.875 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989527+0000", 2024-09-14T06:17:28.875 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0700000000000003e-06 2024-09-14T06:17:28.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.876 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.876 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.876 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989546+0000", 2024-09-14T06:17:28.876 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.948e-05 2024-09-14T06:17:28.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.876 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.876 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.876 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989561+0000", 2024-09-14T06:17:28.876 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4827000000000001e-05 2024-09-14T06:17:28.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.876 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.876 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.876 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065086+0000", 2024-09-14T06:17:28.876 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075524542 2024-09-14T06:17:28.877 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.877 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.877 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.877 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065094+0000", 2024-09-14T06:17:28.877 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8990000000000001e-06 2024-09-14T06:17:28.877 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.877 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.877 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.877 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.877 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.877 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11535 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.877 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:28.877 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.80899582, 2024-09-14T06:17:28.878 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075542466000000003, 2024-09-14T06:17:28.878 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.878 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.878 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.878 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.878 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.878 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.878 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11535 2024-09-14T06:17:28.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.878 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.878 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.878 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.878 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:28.879 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.879 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.879 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:28.879 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.879 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.879 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989588+0000", 2024-09-14T06:17:28.879 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1659999999999999e-06 2024-09-14T06:17:28.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.879 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989593+0000", 2024-09-14T06:17:28.880 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6230000000000002e-06 2024-09-14T06:17:28.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.880 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.880 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989613+0000", 2024-09-14T06:17:28.880 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9604000000000001e-05 2024-09-14T06:17:28.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.880 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.880 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989643+0000", 2024-09-14T06:17:28.880 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0450000000000002e-05 2024-09-14T06:17:28.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.881 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.881 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.881 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065119+0000", 2024-09-14T06:17:28.881 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075475930999999996 2024-09-14T06:17:28.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.881 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.881 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.881 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065129+0000", 2024-09-14T06:17:28.881 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6919999999999997e-06 2024-09-14T06:17:28.881 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.881 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.881 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.881 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.882 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11537 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.882 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:28.882 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.808917897000001, 2024-09-14T06:17:28.882 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075497868999999995, 2024-09-14T06:17:28.882 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.882 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.882 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.882 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.882 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.882 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.882 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11537 2024-09-14T06:17:28.882 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.882 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.883 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.883 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.883 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:28.883 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.883 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.883 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.883 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:28.883 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.883 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.883 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.883 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989666+0000", 2024-09-14T06:17:28.883 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.48e-06 2024-09-14T06:17:28.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.884 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.884 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.884 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989672+0000", 2024-09-14T06:17:28.884 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7960000000000001e-06 2024-09-14T06:17:28.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.884 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.884 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.884 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989690+0000", 2024-09-14T06:17:28.884 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8743e-05 2024-09-14T06:17:28.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.884 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.884 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.884 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989704+0000", 2024-09-14T06:17:28.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3913e-05 2024-09-14T06:17:28.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065155+0000", 2024-09-14T06:17:28.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075450351999999998 2024-09-14T06:17:28.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065162+0000", 2024-09-14T06:17:28.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5850000000000002e-06 2024-09-14T06:17:28.885 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.885 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.885 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.886 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11543 3.12 3.4d02a8b2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:28.886 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:28.886 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 52.808640333, 2024-09-14T06:17:28.886 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074335152000000002, 2024-09-14T06:17:28.886 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:28.886 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:28.886 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:28.886 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:28.886 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:28.886 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:28.886 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11543 2024-09-14T06:17:28.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.887 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:28.887 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.887 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:28.887 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:28.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.887 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.887 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:28.887 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:28.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:28.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.887 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.887 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:28.888 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989944+0000", 2024-09-14T06:17:28.888 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.694e-06 2024-09-14T06:17:28.888 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.888 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.888 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:28.888 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989950+0000", 2024-09-14T06:17:28.888 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3389999999999998e-06 2024-09-14T06:17:28.888 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.888 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.888 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:28.888 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989969+0000", 2024-09-14T06:17:28.888 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8963000000000001e-05 2024-09-14T06:17:28.888 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.888 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.889 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:28.889 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989985+0000", 2024-09-14T06:17:28.889 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5586999999999998e-05 2024-09-14T06:17:28.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.889 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.889 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:28.889 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064270+0000", 2024-09-14T06:17:28.889 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074285552000000005 2024-09-14T06:17:28.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:28.889 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:28.889 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:28.889 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064277+0000", 2024-09-14T06:17:28.889 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0169999999999999e-06 2024-09-14T06:17:28.890 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.890 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.890 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.890 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:28.890 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:28.890 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:28.890 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:28.798+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:28.892 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:17:28.894 INFO:teuthology.orchestra.run.smithi039.stdout:4896 writesame 0xdea0bb0 thru 0xdea4481 data_size 0x7(0x38d2 bytes) 2024-09-14T06:17:28.894 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:28.894 INFO:teuthology.orchestra.run.smithi039.stdout:4898 write 0x448e30a thru 0x449986c (0xb563 bytes) 2024-09-14T06:17:28.894 INFO:teuthology.orchestra.run.smithi039.stdout:4900 write 0xbc5b163 thru 0xbc680f5 (0xcf93 bytes) 2024-09-14T06:17:28.894 INFO:teuthology.orchestra.run.smithi039.stdout:4901 write 0xeb42baa thru 0xeb485f4 (0x5a4b bytes) 2024-09-14T06:17:28.894 INFO:teuthology.orchestra.run.smithi039.stdout:4902 writesame 0xd08c2f6 thru 0xd09526d data_size 0x4(0x8f78 bytes) 2024-09-14T06:17:28.894 INFO:teuthology.orchestra.run.smithi039.stdout:4904 write 0xaa1ee8b thru 0xaa294be (0xa634 bytes) 2024-09-14T06:17:28.894 INFO:teuthology.orchestra.run.smithi039.stdout:4906 writesame 0xe686a9d thru 0xe68a573 data_size 0x1(0x3ad7 bytes) 2024-09-14T06:17:28.894 INFO:teuthology.orchestra.run.smithi039.stdout:4907 punch from 0x9f426d2 to 0x9f4438a, (0x1cb8 bytes) 2024-09-14T06:17:28.894 INFO:teuthology.orchestra.run.smithi039.stdout:4911 writesame 0x7d2daf8 thru 0x7d36f37 data_size 0x20(0x9440 bytes) 2024-09-14T06:17:28.894 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:28.894 INFO:teuthology.orchestra.run.smithi039.stdout:4913 punch from 0x9e73e2a to 0x9e7888e, (0x4a64 bytes) 2024-09-14T06:17:28.895 INFO:teuthology.orchestra.run.smithi039.stdout:4914 punch from 0x33abdfa to 0x33ae63d, (0x2843 bytes) 2024-09-14T06:17:28.895 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:28.895 INFO:teuthology.orchestra.run.smithi039.stdout:4917 read 0xa2c24f7 thru 0xa2c44eb (0x1ff5 bytes) 2024-09-14T06:17:28.895 INFO:teuthology.orchestra.run.smithi039.stdout:4923 read 0xc1dcae1 thru 0xc1e6872 (0x9d92 bytes) 2024-09-14T06:17:28.895 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:28.895 INFO:teuthology.orchestra.run.smithi039.stdout:4925 punch from 0x30aa987 to 0x30b869f, (0xdd18 bytes) 2024-09-14T06:17:28.895 INFO:teuthology.orchestra.run.smithi039.stdout:4926 trunc from 0xeb485f5 to 0x6c3d75a 2024-09-14T06:17:28.895 INFO:teuthology.orchestra.run.smithi039.stdout:4927 punch from 0x3f02868 to 0x3f04003, (0x179b bytes) 2024-09-14T06:17:28.895 INFO:teuthology.orchestra.run.smithi039.stdout:4928 write 0x3b435a7 thru 0x3b4d864 (0xa2be bytes) 2024-09-14T06:17:28.900 INFO:teuthology.orchestra.run.smithi039.stdout:4930 punch from 0x3b31b0d to 0x3b3e10c, (0xc5ff bytes) 2024-09-14T06:17:28.901 INFO:teuthology.orchestra.run.smithi039.stdout:4931 read 0x784a98 thru 0x791cd6 (0xd23f bytes) 2024-09-14T06:17:28.901 INFO:teuthology.orchestra.run.smithi039.stdout:4932 read 0x11ce9a thru 0x11f5b6 (0x271d bytes) 2024-09-14T06:17:28.906 INFO:teuthology.orchestra.run.smithi039.stdout:4933 trunc from 0x6c3d75a to 0x912350f 2024-09-14T06:17:28.915 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:28.926 INFO:teuthology.orchestra.run.smithi039.stdout:4937 write 0xaec2af6 thru 0xaecbfa7 (0x94b2 bytes) 2024-09-14T06:17:28.933 INFO:teuthology.orchestra.run.smithi039.stdout:4938 punch from 0x6193491 to 0x6194c51, (0x17c0 bytes) 2024-09-14T06:17:28.933 INFO:teuthology.orchestra.run.smithi039.stdout:4939 read 0x76f866a thru 0x76fe561 (0x5ef8 bytes) 2024-09-14T06:17:28.933 INFO:teuthology.orchestra.run.smithi039.stdout:4940 punch from 0x31a2996 to 0x31b22cc, (0xf936 bytes) 2024-09-14T06:17:28.933 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:28.933 INFO:teuthology.orchestra.run.smithi039.stdout:4942 punch from 0x5ff8294 to 0x6005837, (0xd5a3 bytes) 2024-09-14T06:17:28.933 INFO:teuthology.orchestra.run.smithi039.stdout:4943 write 0x6a8e76 thru 0x6b79ed (0xeb78 bytes) 2024-09-14T06:17:28.938 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:28.938 INFO:teuthology.orchestra.run.smithi039.stdout:4945 read 0x7837894 thru 0x78444a0 (0xcc0d bytes) 2024-09-14T06:17:28.938 INFO:teuthology.orchestra.run.smithi039.stdout:4946 write 0x458ed27 thru 0x4597c55 (0x8f2f bytes) 2024-09-14T06:17:28.939 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:28.937+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:28.942 INFO:teuthology.orchestra.run.smithi039.stdout:4947 read 0xa1d9fd3 thru 0xa1e5dbe (0xbdec bytes) 2024-09-14T06:17:28.963 INFO:teuthology.orchestra.run.smithi039.stdout:4950 trunc from 0xaecbfa8 to 0x80585 2024-09-14T06:17:28.983 INFO:teuthology.orchestra.run.smithi039.stdout:4951 read 0x3b6dc thru 0x3f54c (0x3e71 bytes) 2024-09-14T06:17:28.984 INFO:teuthology.orchestra.run.smithi039.stdout:4952 punch from 0x5b759 to 0x69256, (0xdafd bytes) 2024-09-14T06:17:28.984 INFO:teuthology.orchestra.run.smithi039.stdout:4953 punch from 0x65ec5 to 0x732ff, (0xd43a bytes) 2024-09-14T06:17:28.984 INFO:teuthology.orchestra.run.smithi039.stdout:4955 read 0x5f7fb thru 0x6ef96 (0xf79c bytes) 2024-09-14T06:17:29.001 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:28.997+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:29.022 INFO:teuthology.orchestra.run.smithi039.stdout:4956 writesame 0xb2d6981 thru 0xb2d973a data_size 0x3(0x2dba bytes) 2024-09-14T06:17:29.022 INFO:teuthology.orchestra.run.smithi039.stdout:4958 write 0x30033ac thru 0x3010996 (0xd5eb bytes) 2024-09-14T06:17:29.029 INFO:teuthology.orchestra.run.smithi039.stdout:4959 write 0x685a2b0 thru 0x6862a0d (0x875e bytes) 2024-09-14T06:17:29.039 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.044 INFO:teuthology.orchestra.run.smithi039.stdout:4963 trunc from 0xb2d973b to 0xd035563 2024-09-14T06:17:29.059 INFO:teuthology.orchestra.run.smithi039.stdout:4964 punch from 0x3348783 to 0x3353fb2, (0xb82f bytes) 2024-09-14T06:17:29.059 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.060 INFO:teuthology.orchestra.run.smithi039.stdout:4967 read 0xabc4ab5 thru 0xabc7e27 (0x3373 bytes) 2024-09-14T06:17:29.060 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.060 INFO:teuthology.orchestra.run.smithi039.stdout:4969 punch from 0x858e15d to 0x858fb73, (0x1a16 bytes) 2024-09-14T06:17:29.060 INFO:teuthology.orchestra.run.smithi039.stdout:4970 write 0x1765fd9 thru 0x1767841 (0x1869 bytes) 2024-09-14T06:17:29.066 INFO:teuthology.orchestra.run.smithi039.stdout:4971 write 0x51d21bc thru 0x51db051 (0x8e96 bytes) 2024-09-14T06:17:29.070 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:29.070 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:29.070 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:29.070 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:29.073 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.073 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.073 INFO:teuthology.orchestra.run.smithi039.stdout:4975 write 0x9baf094 thru 0x9bb9cad (0xac1a bytes) 2024-09-14T06:17:29.080 INFO:teuthology.orchestra.run.smithi039.stdout:4976 writesame 0x31caf92 thru 0x31cdd1a data_size 0x1(0x2d89 bytes) 2024-09-14T06:17:29.085 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:17:29.103 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:29.098+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:29.111 INFO:teuthology.orchestra.run.smithi039.stdout:4977 trunc from 0xd035563 to 0x87d61ef 2024-09-14T06:17:29.148 INFO:teuthology.orchestra.run.smithi039.stdout:4978 write 0x891c59d thru 0x892b082 (0xeae6 bytes) 2024-09-14T06:17:29.158 INFO:teuthology.orchestra.run.smithi039.stdout:4980 read 0x5764a27 thru 0x5773aae (0xf088 bytes) 2024-09-14T06:17:29.158 INFO:teuthology.orchestra.run.smithi039.stdout:4981 read 0x4d110a0 thru 0x4d20190 (0xf0f1 bytes) 2024-09-14T06:17:29.178 INFO:teuthology.orchestra.run.smithi039.stdout:4984 write 0xad2e809 thru 0xad345be (0x5db6 bytes) 2024-09-14T06:17:29.185 INFO:teuthology.orchestra.run.smithi039.stdout:4985 read 0xa61f070 thru 0xa628f49 (0x9eda bytes) 2024-09-14T06:17:29.185 INFO:teuthology.orchestra.run.smithi039.stdout:4986 write 0x2215077 thru 0x221dd39 (0x8cc3 bytes) 2024-09-14T06:17:29.193 INFO:teuthology.orchestra.run.smithi039.stdout:4987 writesame 0x53da854 thru 0x53e6967 data_size 0x12(0xc114 bytes) 2024-09-14T06:17:29.203 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:29.201+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:29.205 INFO:teuthology.orchestra.run.smithi039.stdout:4989 read 0x1c83bc0 thru 0x1c92acd (0xef0e bytes) 2024-09-14T06:17:29.205 INFO:teuthology.orchestra.run.smithi039.stdout:4992 writesame 0x990834a thru 0x9909c67 data_size 0x5(0x191e bytes) 2024-09-14T06:17:29.213 INFO:teuthology.orchestra.run.smithi039.stdout:4993 write 0x304e972 thru 0x305667f (0x7d0e bytes) 2024-09-14T06:17:29.217 INFO:teuthology.orchestra.run.smithi039.stdout:4994 punch from 0x34517ce to 0x34616c3, (0xfef5 bytes) 2024-09-14T06:17:29.217 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.217 INFO:teuthology.orchestra.run.smithi039.stdout:4997 read 0x728166f thru 0x728cf98 (0xb92a bytes) 2024-09-14T06:17:29.246 INFO:teuthology.orchestra.run.smithi039.stdout:4999 trunc from 0xad345bf to 0x54596b 2024-09-14T06:17:29.250 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:29.251 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:29.251 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:29.251 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:29.251 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:29.268 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:17:29.278 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.278 INFO:teuthology.orchestra.run.smithi039.stdout:5001 read 0x40e370 thru 0x414342 (0x5fd3 bytes) 2024-09-14T06:17:29.279 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.310 INFO:teuthology.orchestra.run.smithi039.stdout:5003 trunc from 0x54596b to 0xba73f2d 2024-09-14T06:17:29.311 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:29.309+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:29.323 INFO:teuthology.orchestra.run.smithi039.stdout:5004 write 0xb847338 thru 0xb850abd (0x9786 bytes) 2024-09-14T06:17:29.329 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.329 INFO:teuthology.orchestra.run.smithi039.stdout:5006 write 0x23ef71b thru 0x23f0664 (0xf4a bytes) 2024-09-14T06:17:29.336 INFO:teuthology.orchestra.run.smithi039.stdout:5007 write 0x4acbe2e thru 0x4ad4a57 (0x8c2a bytes) 2024-09-14T06:17:29.343 INFO:teuthology.orchestra.run.smithi039.stdout:5009 read 0x78b322 thru 0x791ee3 (0x6bc2 bytes) 2024-09-14T06:17:29.360 INFO:teuthology.orchestra.run.smithi039.stdout:5011 writesame 0xc2f1e50 thru 0xc2fd184 data_size 0x1(0xb335 bytes) 2024-09-14T06:17:29.360 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.360 INFO:teuthology.orchestra.run.smithi039.stdout:5013 read 0x105684a thru 0x10603e1 (0x9b98 bytes) 2024-09-14T06:17:29.360 INFO:teuthology.orchestra.run.smithi039.stdout:5015 punch from 0x7d26bb3 to 0x7d35203, (0xe650 bytes) 2024-09-14T06:17:29.360 INFO:teuthology.orchestra.run.smithi039.stdout:5016 write 0x5530682 thru 0x553aeae (0xa82d bytes) 2024-09-14T06:17:29.396 INFO:teuthology.orchestra.run.smithi039.stdout:5017 trunc from 0xc2fd185 to 0x13f5507 2024-09-14T06:17:29.405 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:29.401+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:29.436 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:29.436 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:29.436 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:29.436 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:29.436 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.437 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12584 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.437 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.039784+0000", 2024-09-14T06:17:29.437 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.396103598000003, 2024-09-14T06:17:29.437 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.024433808000000001, 2024-09-14T06:17:29.437 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.437 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.437 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.437 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.437 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.437 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.437 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12584 2024-09-14T06:17:29.437 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.437 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.437 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.438 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.438 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039784+0000", 2024-09-14T06:17:29.438 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.438 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.438 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.438 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.438 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039784+0000", 2024-09-14T06:17:29.438 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.438 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.438 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.438 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.438 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039784+0000", 2024-09-14T06:17:29.438 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1999999999999999e-07 2024-09-14T06:17:29.439 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.439 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.439 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.439 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039788+0000", 2024-09-14T06:17:29.439 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.41e-06 2024-09-14T06:17:29.439 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.439 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.439 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.439 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039797+0000", 2024-09-14T06:17:29.439 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7969999999999995e-06 2024-09-14T06:17:29.439 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.439 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.439 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.439 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039804+0000", 2024-09-14T06:17:29.440 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7870000000000004e-06 2024-09-14T06:17:29.440 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.440 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.440 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.440 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.048222+0000", 2024-09-14T06:17:29.440 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0084172770000000008 2024-09-14T06:17:29.440 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.440 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.440 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.440 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.048334+0000", 2024-09-14T06:17:29.440 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000112895 2024-09-14T06:17:29.440 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.440 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.441 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.441 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064217+0000", 2024-09-14T06:17:29.441 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015882922000000001 2024-09-14T06:17:29.441 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.441 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.441 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.441 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.441 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.441 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12661 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.441 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.040519+0000", 2024-09-14T06:17:29.441 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.395367817999997, 2024-09-14T06:17:29.441 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014404233000000001, 2024-09-14T06:17:29.441 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.441 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.442 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.442 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.442 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.442 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.442 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12661 2024-09-14T06:17:29.442 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.442 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.442 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.442 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.442 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040519+0000", 2024-09-14T06:17:29.442 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.443 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.443 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.443 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.443 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040519+0000", 2024-09-14T06:17:29.443 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.443 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.443 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.443 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.443 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040520+0000", 2024-09-14T06:17:29.443 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6899999999999996e-07 2024-09-14T06:17:29.443 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.443 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.443 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.444 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040530+0000", 2024-09-14T06:17:29.444 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6409999999999997e-06 2024-09-14T06:17:29.444 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.444 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.444 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.444 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040540+0000", 2024-09-14T06:17:29.444 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0247e-05 2024-09-14T06:17:29.444 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.444 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.444 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.444 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040547+0000", 2024-09-14T06:17:29.444 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3830000000000003e-06 2024-09-14T06:17:29.444 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.444 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.445 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.054677+0000", 2024-09-14T06:17:29.445 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014130095000000001 2024-09-14T06:17:29.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.445 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.445 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.054786+0000", 2024-09-14T06:17:29.445 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000108441 2024-09-14T06:17:29.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.445 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.445 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.054924+0000", 2024-09-14T06:17:29.445 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013765699999999999 2024-09-14T06:17:29.445 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.446 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.446 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.446 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.446 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.446 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12662 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.446 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.040559+0000", 2024-09-14T06:17:29.446 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.395328481, 2024-09-14T06:17:29.446 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014659604999999999, 2024-09-14T06:17:29.446 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.446 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.446 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.446 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.446 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.446 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.447 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12662 2024-09-14T06:17:29.447 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.447 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.447 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.447 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.447 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040559+0000", 2024-09-14T06:17:29.447 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.447 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.447 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.447 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.447 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040559+0000", 2024-09-14T06:17:29.447 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.448 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.448 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.448 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040559+0000", 2024-09-14T06:17:29.448 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5799999999999998e-07 2024-09-14T06:17:29.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.448 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.448 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.448 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040562+0000", 2024-09-14T06:17:29.448 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9349999999999999e-06 2024-09-14T06:17:29.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.448 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.448 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.449 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040572+0000", 2024-09-14T06:17:29.449 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9480000000000003e-06 2024-09-14T06:17:29.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.449 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.449 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040579+0000", 2024-09-14T06:17:29.449 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1459999999999999e-06 2024-09-14T06:17:29.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.449 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.449 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.054963+0000", 2024-09-14T06:17:29.449 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014383986 2024-09-14T06:17:29.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.450 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.450 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055084+0000", 2024-09-14T06:17:29.450 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000120822 2024-09-14T06:17:29.450 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.450 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.450 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.450 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055218+0000", 2024-09-14T06:17:29.450 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013401 2024-09-14T06:17:29.450 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.450 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.450 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.450 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.450 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.450 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12666 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.451 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.040590+0000", 2024-09-14T06:17:29.451 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.395297333000002, 2024-09-14T06:17:29.451 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014895584999999999, 2024-09-14T06:17:29.451 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.451 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.451 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.451 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.451 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.451 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.451 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12666 2024-09-14T06:17:29.451 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.451 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.451 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.452 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.452 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040590+0000", 2024-09-14T06:17:29.452 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.452 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.452 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.452 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.452 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040590+0000", 2024-09-14T06:17:29.452 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.452 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.453 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.453 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.453 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040590+0000", 2024-09-14T06:17:29.453 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:17:29.453 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.453 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.453 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.453 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040593+0000", 2024-09-14T06:17:29.453 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0189999999999998e-06 2024-09-14T06:17:29.453 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.453 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.453 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.453 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040603+0000", 2024-09-14T06:17:29.454 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2820000000000002e-06 2024-09-14T06:17:29.454 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.454 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.454 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.454 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040610+0000", 2024-09-14T06:17:29.454 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1219999999999997e-06 2024-09-14T06:17:29.454 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.454 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.454 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.454 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055257+0000", 2024-09-14T06:17:29.454 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014646762000000001 2024-09-14T06:17:29.454 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.454 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.454 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.455 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055379+0000", 2024-09-14T06:17:29.455 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012186899999999999 2024-09-14T06:17:29.455 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.455 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.455 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.455 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055485+0000", 2024-09-14T06:17:29.455 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000106848 2024-09-14T06:17:29.455 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.455 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.455 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.455 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.455 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.455 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12672 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.455 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.040620+0000", 2024-09-14T06:17:29.456 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.395266849999999, 2024-09-14T06:17:29.456 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015140836, 2024-09-14T06:17:29.456 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.456 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.456 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.456 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.456 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.456 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.456 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12672 2024-09-14T06:17:29.456 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.456 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.456 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.457 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.457 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040620+0000", 2024-09-14T06:17:29.457 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.457 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.457 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.457 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.457 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040620+0000", 2024-09-14T06:17:29.457 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.457 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.457 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.457 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.457 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040621+0000", 2024-09-14T06:17:29.457 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9699999999999995e-07 2024-09-14T06:17:29.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.458 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.458 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040632+0000", 2024-09-14T06:17:29.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1012999999999999e-05 2024-09-14T06:17:29.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.458 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.458 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040642+0000", 2024-09-14T06:17:29.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0054999999999999e-05 2024-09-14T06:17:29.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.458 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.458 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040649+0000", 2024-09-14T06:17:29.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3769999999999998e-06 2024-09-14T06:17:29.459 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.459 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.459 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.459 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055521+0000", 2024-09-14T06:17:29.459 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014871383 2024-09-14T06:17:29.459 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.459 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.459 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.459 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055634+0000", 2024-09-14T06:17:29.459 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000112757 2024-09-14T06:17:29.459 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.459 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.459 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.459 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055761+0000", 2024-09-14T06:17:29.460 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012755399999999999 2024-09-14T06:17:29.460 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.460 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.460 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.460 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.460 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.460 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12673 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 385024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.460 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.041758+0000", 2024-09-14T06:17:29.460 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.394128635999998, 2024-09-14T06:17:29.460 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014243184000000001, 2024-09-14T06:17:29.460 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.460 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.460 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.460 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.461 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.461 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.461 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12673 2024-09-14T06:17:29.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.461 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041758+0000", 2024-09-14T06:17:29.461 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041758+0000", 2024-09-14T06:17:29.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.462 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.462 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041760+0000", 2024-09-14T06:17:29.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0860000000000001e-06 2024-09-14T06:17:29.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.462 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.462 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041766+0000", 2024-09-14T06:17:29.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-06 2024-09-14T06:17:29.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.462 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.462 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041794+0000", 2024-09-14T06:17:29.463 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7993000000000001e-05 2024-09-14T06:17:29.463 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.463 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.463 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.463 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041813+0000", 2024-09-14T06:17:29.463 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.823e-05 2024-09-14T06:17:29.463 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.463 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.463 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.463 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055797+0000", 2024-09-14T06:17:29.463 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013984612 2024-09-14T06:17:29.463 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.463 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.463 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.464 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055900+0000", 2024-09-14T06:17:29.464 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000102464 2024-09-14T06:17:29.464 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.464 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.464 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.464 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.056002+0000", 2024-09-14T06:17:29.464 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000101869 2024-09-14T06:17:29.464 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.464 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.464 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.464 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.464 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.464 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12674 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.464 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.041833+0000", 2024-09-14T06:17:29.465 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.394054107000002, 2024-09-14T06:17:29.465 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022511527, 2024-09-14T06:17:29.465 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.465 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.465 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.465 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.465 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.465 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.465 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12674 2024-09-14T06:17:29.465 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.465 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.465 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.465 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.465 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041833+0000", 2024-09-14T06:17:29.466 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.466 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.466 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.466 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.466 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041833+0000", 2024-09-14T06:17:29.466 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.466 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.466 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.466 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.466 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041834+0000", 2024-09-14T06:17:29.466 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7999999999999995e-07 2024-09-14T06:17:29.466 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.466 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.466 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.467 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041869+0000", 2024-09-14T06:17:29.467 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5766000000000003e-05 2024-09-14T06:17:29.467 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.467 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.467 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.467 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041922+0000", 2024-09-14T06:17:29.467 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2086000000000002e-05 2024-09-14T06:17:29.467 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.467 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.467 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.467 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041931+0000", 2024-09-14T06:17:29.467 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6160000000000008e-06 2024-09-14T06:17:29.467 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.467 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.468 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.468 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.056038+0000", 2024-09-14T06:17:29.468 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01410729 2024-09-14T06:17:29.468 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.468 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.468 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.468 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.056143+0000", 2024-09-14T06:17:29.468 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000104697 2024-09-14T06:17:29.468 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.468 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.468 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.468 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064345+0000", 2024-09-14T06:17:29.468 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0082013920000000001 2024-09-14T06:17:29.468 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.469 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.469 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.469 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.469 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.469 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12677 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.469 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.041952+0000", 2024-09-14T06:17:29.469 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.393934690000002, 2024-09-14T06:17:29.469 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022599910000000001, 2024-09-14T06:17:29.469 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.469 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.469 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.469 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.469 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12677 2024-09-14T06:17:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041952+0000", 2024-09-14T06:17:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041952+0000", 2024-09-14T06:17:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.470 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041953+0000", 2024-09-14T06:17:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1100000000000005e-07 2024-09-14T06:17:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041958+0000", 2024-09-14T06:17:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3839999999999999e-06 2024-09-14T06:17:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.471 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042800+0000", 2024-09-14T06:17:29.472 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00084272399999999999 2024-09-14T06:17:29.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.472 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.472 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.472 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042817+0000", 2024-09-14T06:17:29.472 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6507e-05 2024-09-14T06:17:29.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.472 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.472 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.472 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064375+0000", 2024-09-14T06:17:29.472 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021558165000000001 2024-09-14T06:17:29.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.472 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.472 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.473 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064458+0000", 2024-09-14T06:17:29.473 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3205e-05 2024-09-14T06:17:29.473 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.473 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.473 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.473 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064552+0000", 2024-09-14T06:17:29.473 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.4114e-05 2024-09-14T06:17:29.473 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.473 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.473 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.473 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.473 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.473 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12678 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.474 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.042838+0000", 2024-09-14T06:17:29.474 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.393049062000003, 2024-09-14T06:17:29.474 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021916718000000002, 2024-09-14T06:17:29.474 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.474 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.474 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.474 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.474 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.474 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.474 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12678 2024-09-14T06:17:29.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.474 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.474 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.475 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.475 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042838+0000", 2024-09-14T06:17:29.475 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.475 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.475 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.475 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042838+0000", 2024-09-14T06:17:29.475 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.475 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.475 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.475 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042839+0000", 2024-09-14T06:17:29.475 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1510000000000001e-06 2024-09-14T06:17:29.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.475 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.476 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.476 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042845+0000", 2024-09-14T06:17:29.476 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9190000000000004e-06 2024-09-14T06:17:29.476 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.476 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.476 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.476 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042859+0000", 2024-09-14T06:17:29.476 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4022000000000001e-05 2024-09-14T06:17:29.476 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.476 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.476 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.476 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042867+0000", 2024-09-14T06:17:29.476 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9549999999999992e-06 2024-09-14T06:17:29.476 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.477 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.477 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.477 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064581+0000", 2024-09-14T06:17:29.477 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021713542999999998 2024-09-14T06:17:29.477 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.477 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.477 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.477 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064656+0000", 2024-09-14T06:17:29.477 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5564999999999999e-05 2024-09-14T06:17:29.477 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.477 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.477 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.477 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064755+0000", 2024-09-14T06:17:29.478 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.8562999999999998e-05 2024-09-14T06:17:29.478 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.478 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.478 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.478 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.478 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.478 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12687 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.478 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.042878+0000", 2024-09-14T06:17:29.478 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.393009268, 2024-09-14T06:17:29.478 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022072561000000001, 2024-09-14T06:17:29.478 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.478 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.478 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.478 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.478 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.479 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.479 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12687 2024-09-14T06:17:29.479 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.479 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.479 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.479 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.479 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042878+0000", 2024-09-14T06:17:29.479 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.479 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.479 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.479 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.479 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042878+0000", 2024-09-14T06:17:29.479 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.479 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.480 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.480 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.480 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042879+0000", 2024-09-14T06:17:29.480 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9999999999999997e-07 2024-09-14T06:17:29.480 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.480 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.480 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.480 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042882+0000", 2024-09-14T06:17:29.480 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.608e-06 2024-09-14T06:17:29.480 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.480 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.480 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.480 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042892+0000", 2024-09-14T06:17:29.480 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9729999999999992e-06 2024-09-14T06:17:29.481 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.481 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.481 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.481 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042911+0000", 2024-09-14T06:17:29.481 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9369999999999999e-05 2024-09-14T06:17:29.481 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.481 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.481 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.481 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064785+0000", 2024-09-14T06:17:29.481 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021873182000000001 2024-09-14T06:17:29.481 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.481 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.481 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.482 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064860+0000", 2024-09-14T06:17:29.482 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5403999999999998e-05 2024-09-14T06:17:29.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.482 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.482 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.482 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064950+0000", 2024-09-14T06:17:29.482 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0323999999999998e-05 2024-09-14T06:17:29.482 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.482 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.482 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.482 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.482 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12688 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.483 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.042922+0000", 2024-09-14T06:17:29.483 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.392964945000003, 2024-09-14T06:17:29.483 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022202118, 2024-09-14T06:17:29.483 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.483 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.483 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.483 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.483 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.483 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.483 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12688 2024-09-14T06:17:29.483 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.483 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.483 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.483 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.484 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042922+0000", 2024-09-14T06:17:29.484 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.484 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.484 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.484 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.484 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042922+0000", 2024-09-14T06:17:29.484 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.484 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.484 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.484 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.484 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042923+0000", 2024-09-14T06:17:29.484 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:17:29.484 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.484 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.485 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.485 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042937+0000", 2024-09-14T06:17:29.485 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4337e-05 2024-09-14T06:17:29.485 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.485 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.485 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.485 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042948+0000", 2024-09-14T06:17:29.485 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0569e-05 2024-09-14T06:17:29.485 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.485 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.485 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.485 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042955+0000", 2024-09-14T06:17:29.485 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3250000000000002e-06 2024-09-14T06:17:29.485 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064981+0000", 2024-09-14T06:17:29.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022026046000000001 2024-09-14T06:17:29.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065055+0000", 2024-09-14T06:17:29.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4016000000000004e-05 2024-09-14T06:17:29.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065124+0000", 2024-09-14T06:17:29.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9141999999999996e-05 2024-09-14T06:17:29.487 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.487 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.487 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.487 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.487 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.487 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12690 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.487 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044605+0000", 2024-09-14T06:17:29.487 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.391281905, 2024-09-14T06:17:29.487 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020723715, 2024-09-14T06:17:29.487 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.487 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.487 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.487 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.487 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.488 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.488 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12690 2024-09-14T06:17:29.488 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.488 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.488 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.488 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.488 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044605+0000", 2024-09-14T06:17:29.488 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.488 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.488 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.488 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.488 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044605+0000", 2024-09-14T06:17:29.488 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.488 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.489 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.489 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.489 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044606+0000", 2024-09-14T06:17:29.489 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2020000000000001e-06 2024-09-14T06:17:29.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.489 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.489 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.489 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044612+0000", 2024-09-14T06:17:29.489 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9830000000000001e-06 2024-09-14T06:17:29.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.489 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.489 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.489 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044631+0000", 2024-09-14T06:17:29.489 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8280000000000001e-05 2024-09-14T06:17:29.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.490 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.490 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044642+0000", 2024-09-14T06:17:29.490 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1018e-05 2024-09-14T06:17:29.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.490 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.490 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065153+0000", 2024-09-14T06:17:29.490 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020511709999999999 2024-09-14T06:17:29.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.490 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.490 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065230+0000", 2024-09-14T06:17:29.491 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7003999999999997e-05 2024-09-14T06:17:29.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.491 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.491 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.491 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065329+0000", 2024-09-14T06:17:29.491 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.8517999999999994e-05 2024-09-14T06:17:29.491 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.491 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.491 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.491 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.491 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12692 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.491 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044659+0000", 2024-09-14T06:17:29.491 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.391228075999997, 2024-09-14T06:17:29.492 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020896346999999999, 2024-09-14T06:17:29.492 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.492 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.492 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.492 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.492 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.492 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.492 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12692 2024-09-14T06:17:29.492 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.492 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.492 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.492 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.492 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044659+0000", 2024-09-14T06:17:29.492 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.493 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.493 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.493 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044659+0000", 2024-09-14T06:17:29.493 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.493 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.493 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.493 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044660+0000", 2024-09-14T06:17:29.493 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0200000000000001e-07 2024-09-14T06:17:29.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.493 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.493 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.493 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044663+0000", 2024-09-14T06:17:29.494 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:17:29.494 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.494 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.494 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.494 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044674+0000", 2024-09-14T06:17:29.494 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.096e-05 2024-09-14T06:17:29.494 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.494 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.494 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.494 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044682+0000", 2024-09-14T06:17:29.494 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7149999999999998e-06 2024-09-14T06:17:29.494 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.494 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.494 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065357+0000", 2024-09-14T06:17:29.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020674794999999999 2024-09-14T06:17:29.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065435+0000", 2024-09-14T06:17:29.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8170999999999994e-05 2024-09-14T06:17:29.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065555+0000", 2024-09-14T06:17:29.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000120609 2024-09-14T06:17:29.495 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.495 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.496 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.496 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12694 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.496 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044692+0000", 2024-09-14T06:17:29.496 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.391194800999997, 2024-09-14T06:17:29.496 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021050059999999999, 2024-09-14T06:17:29.496 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.496 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.496 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.496 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.496 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.496 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.497 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12694 2024-09-14T06:17:29.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.497 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.497 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.497 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044692+0000", 2024-09-14T06:17:29.497 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.497 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.497 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044692+0000", 2024-09-14T06:17:29.497 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.498 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044693+0000", 2024-09-14T06:17:29.498 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8800000000000002e-07 2024-09-14T06:17:29.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.498 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044705+0000", 2024-09-14T06:17:29.498 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2252000000000001e-05 2024-09-14T06:17:29.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.498 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044716+0000", 2024-09-14T06:17:29.498 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1206000000000001e-05 2024-09-14T06:17:29.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.499 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.499 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044724+0000", 2024-09-14T06:17:29.499 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7360000000000001e-06 2024-09-14T06:17:29.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.499 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.499 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065584+0000", 2024-09-14T06:17:29.499 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020859459 2024-09-14T06:17:29.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.499 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.499 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065657+0000", 2024-09-14T06:17:29.499 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3509000000000004e-05 2024-09-14T06:17:29.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.500 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.500 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065742+0000", 2024-09-14T06:17:29.500 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5210000000000006e-05 2024-09-14T06:17:29.500 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.500 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.500 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.500 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12702 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.500 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044734+0000", 2024-09-14T06:17:29.500 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.391152810999998, 2024-09-14T06:17:29.500 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021236848999999999, 2024-09-14T06:17:29.501 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.501 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.501 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.501 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.501 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.501 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.501 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12702 2024-09-14T06:17:29.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.501 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.501 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.501 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044734+0000", 2024-09-14T06:17:29.501 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044734+0000", 2024-09-14T06:17:29.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044735+0000", 2024-09-14T06:17:29.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3600000000000003e-07 2024-09-14T06:17:29.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044738+0000", 2024-09-14T06:17:29.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9189999999999999e-06 2024-09-14T06:17:29.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.503 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.503 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044748+0000", 2024-09-14T06:17:29.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0111e-05 2024-09-14T06:17:29.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.503 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.503 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044766+0000", 2024-09-14T06:17:29.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8199999999999999e-05 2024-09-14T06:17:29.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.503 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.503 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065771+0000", 2024-09-14T06:17:29.504 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021004683 2024-09-14T06:17:29.504 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.504 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.504 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.504 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065849+0000", 2024-09-14T06:17:29.504 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7684999999999999e-05 2024-09-14T06:17:29.504 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.504 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.504 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.504 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065971+0000", 2024-09-14T06:17:29.504 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012251499999999999 2024-09-14T06:17:29.504 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.504 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.504 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.505 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.505 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.505 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12703 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 401408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.505 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044777+0000", 2024-09-14T06:17:29.505 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.391110185000002, 2024-09-14T06:17:29.505 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021382453999999999, 2024-09-14T06:17:29.505 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.505 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.505 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.505 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.505 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.505 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.505 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12703 2024-09-14T06:17:29.505 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.506 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.506 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.506 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.506 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044777+0000", 2024-09-14T06:17:29.506 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.506 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.506 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.506 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.506 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044777+0000", 2024-09-14T06:17:29.506 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.506 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.506 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.506 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.506 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044778+0000", 2024-09-14T06:17:29.506 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2999999999999999e-07 2024-09-14T06:17:29.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.507 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.507 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.507 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044787+0000", 2024-09-14T06:17:29.507 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2429999999999994e-06 2024-09-14T06:17:29.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.507 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.507 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.507 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044798+0000", 2024-09-14T06:17:29.507 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0447e-05 2024-09-14T06:17:29.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.507 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.507 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.507 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044805+0000", 2024-09-14T06:17:29.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.182e-06 2024-09-14T06:17:29.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066001+0000", 2024-09-14T06:17:29.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021196079 2024-09-14T06:17:29.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066082+0000", 2024-09-14T06:17:29.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1011000000000001e-05 2024-09-14T06:17:29.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.509 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.509 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066159+0000", 2024-09-14T06:17:29.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7562e-05 2024-09-14T06:17:29.509 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.509 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.509 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.509 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.509 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12705 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.509 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044815+0000", 2024-09-14T06:17:29.509 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.391072344999998, 2024-09-14T06:17:29.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021573917000000001, 2024-09-14T06:17:29.509 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.509 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.510 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.510 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.510 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.510 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.510 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12705 2024-09-14T06:17:29.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.510 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.510 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044815+0000", 2024-09-14T06:17:29.510 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.511 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044815+0000", 2024-09-14T06:17:29.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.511 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.511 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044815+0000", 2024-09-14T06:17:29.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9999999999999997e-07 2024-09-14T06:17:29.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.511 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.511 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044818+0000", 2024-09-14T06:17:29.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7779999999999999e-06 2024-09-14T06:17:29.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.512 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.512 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044828+0000", 2024-09-14T06:17:29.512 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0095000000000001e-05 2024-09-14T06:17:29.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.512 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.512 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044835+0000", 2024-09-14T06:17:29.512 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9419999999999998e-06 2024-09-14T06:17:29.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.512 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.512 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066188+0000", 2024-09-14T06:17:29.512 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021352560999999999 2024-09-14T06:17:29.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.513 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.513 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066275+0000", 2024-09-14T06:17:29.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7020000000000004e-05 2024-09-14T06:17:29.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.513 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.513 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066389+0000", 2024-09-14T06:17:29.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000113921 2024-09-14T06:17:29.513 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.513 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.513 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.514 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12707 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:29.514 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044844+0000", 2024-09-14T06:17:29.514 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 53.391042775999999, 2024-09-14T06:17:29.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021756622, 2024-09-14T06:17:29.514 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.514 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.514 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:29.514 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.514 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.514 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.514 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12707 2024-09-14T06:17:29.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.514 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044844+0000", 2024-09-14T06:17:29.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044844+0000", 2024-09-14T06:17:29.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.516 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044845+0000", 2024-09-14T06:17:29.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3600000000000003e-07 2024-09-14T06:17:29.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.516 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.516 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044848+0000", 2024-09-14T06:17:29.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4710000000000001e-06 2024-09-14T06:17:29.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.516 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.516 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044857+0000", 2024-09-14T06:17:29.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5219999999999996e-06 2024-09-14T06:17:29.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.517 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.517 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044864+0000", 2024-09-14T06:17:29.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:17:29.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.517 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.517 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.517 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066424+0000", 2024-09-14T06:17:29.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021560461999999999 2024-09-14T06:17:29.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.517 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.517 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.517 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066511+0000", 2024-09-14T06:17:29.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6298999999999995e-05 2024-09-14T06:17:29.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.518 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.518 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066601+0000", 2024-09-14T06:17:29.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0252000000000003e-05 2024-09-14T06:17:29.518 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.518 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.518 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.518 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:24231 3.c e47/23 3:335dc9f4:::rbd_data.1152db28e106.0000000000000002:head v 47'36, mlcod=46'34)", 2024-09-14T06:17:29.518 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:51.909218+0000", 2024-09-14T06:17:29.518 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 37.526669372000001, 2024-09-14T06:17:29.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.018625953000000001, 2024-09-14T06:17:29.519 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.519 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.519 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:29.519 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.909218+0000", 2024-09-14T06:17:29.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.909218+0000", 2024-09-14T06:17:29.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.520 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.520 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.909221+0000", 2024-09-14T06:17:29.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:17:29.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.520 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.520 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.520 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.921926+0000", 2024-09-14T06:17:29.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.012705602 2024-09-14T06:17:29.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.520 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.520 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.520 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.921949+0000", 2024-09-14T06:17:29.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2824e-05 2024-09-14T06:17:29.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.921960+0000", 2024-09-14T06:17:29.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0626e-05 2024-09-14T06:17:29.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.922035+0000", 2024-09-14T06:17:29.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4938999999999995e-05 2024-09-14T06:17:29.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.922058+0000", 2024-09-14T06:17:29.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2847e-05 2024-09-14T06:17:29.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:29.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.927752+0000", 2024-09-14T06:17:29.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056944810000000004 2024-09-14T06:17:29.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.927844+0000", 2024-09-14T06:17:29.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1524e-05 2024-09-14T06:17:29.522 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.522 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.523 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.523 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.523 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:27637 3.b 3:d7b02ca9:::rbd_data.1152ff705991.00000000000000a5:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:17:29.523 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:55.303878+0000", 2024-09-14T06:17:29.523 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 34.132009441999998, 2024-09-14T06:17:29.523 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.027469031000000001, 2024-09-14T06:17:29.523 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:29.523 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:29.523 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:29.524 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:29.524 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:29.524 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:29.524 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 27637 2024-09-14T06:17:29.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.524 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:29.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.524 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:29.524 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303878+0000", 2024-09-14T06:17:29.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:29.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303878+0000", 2024-09-14T06:17:29.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:29.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:29.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303881+0000", 2024-09-14T06:17:29.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8430000000000001e-06 2024-09-14T06:17:29.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:29.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303894+0000", 2024-09-14T06:17:29.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3376e-05 2024-09-14T06:17:29.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:29.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303926+0000", 2024-09-14T06:17:29.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1825000000000001e-05 2024-09-14T06:17:29.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:29.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303941+0000", 2024-09-14T06:17:29.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5048999999999999e-05 2024-09-14T06:17:29.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.527 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:29.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.304106+0000", 2024-09-14T06:17:29.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00016504700000000001 2024-09-14T06:17:29.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.527 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:29.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.304357+0000", 2024-09-14T06:17:29.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025158899999999998 2024-09-14T06:17:29.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 6", 2024-09-14T06:17:29.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.308743+0000", 2024-09-14T06:17:29.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043854419999999998 2024-09-14T06:17:29.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:29.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.331026+0000", 2024-09-14T06:17:29.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022282999000000001 2024-09-14T06:17:29.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:29.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.331072+0000", 2024-09-14T06:17:29.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5694e-05 2024-09-14T06:17:29.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.529 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:29.529 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.331229+0000", 2024-09-14T06:17:29.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015776399999999999 2024-09-14T06:17:29.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:29.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:29.529 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:29.529 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.331347+0000", 2024-09-14T06:17:29.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000117403 2024-09-14T06:17:29.529 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.529 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.529 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.529 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:29.529 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:29.530 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:29.530 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:17:29.532 INFO:teuthology.orchestra.run.smithi039.stdout:5019 write 0x800b273 thru 0x80197e2 (0xe570 bytes) 2024-09-14T06:17:29.532 INFO:teuthology.orchestra.run.smithi039.stdout:5020 writesame 0xd392306 thru 0xd393e2f data_size 0x39(0x1b2a bytes) 2024-09-14T06:17:29.532 INFO:teuthology.orchestra.run.smithi039.stdout:5021 writesame 0x746c464 thru 0x747290c data_size 0x1(0x64a9 bytes) 2024-09-14T06:17:29.532 INFO:teuthology.orchestra.run.smithi039.stdout:5022 write 0xb7e6ea0 thru 0xb7ec9c6 (0x5b27 bytes) 2024-09-14T06:17:29.532 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-09-14T06:17:29.533 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:17:29.534 INFO:teuthology.orchestra.run.smithi039.stdout:5023 read 0xc55f952 thru 0xc56f031 (0xf6e0 bytes) 2024-09-14T06:17:29.534 INFO:teuthology.orchestra.run.smithi039.stdout:5024 punch from 0xc466caa to 0xc47012a, (0x9480 bytes) 2024-09-14T06:17:29.534 INFO:teuthology.orchestra.run.smithi039.stdout:5025 write 0x7f8a813 thru 0x7f8d8f2 (0x30e0 bytes) 2024-09-14T06:17:29.534 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:29.530+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:29.544 INFO:teuthology.orchestra.run.smithi039.stdout:5026 writesame 0x65bea8b thru 0x65c720b data_size 0x3(0x8781 bytes) 2024-09-14T06:17:29.561 INFO:teuthology.orchestra.run.smithi039.stdout:5027 trunc from 0xd393e30 to 0xb769ea0 2024-09-14T06:17:29.571 INFO:teuthology.orchestra.run.smithi039.stdout:5029 read 0x613a435 thru 0x613f516 (0x50e2 bytes) 2024-09-14T06:17:29.572 INFO:teuthology.orchestra.run.smithi039.stdout:5030 write 0x9720af3 thru 0x972b989 (0xae97 bytes) 2024-09-14T06:17:29.579 INFO:teuthology.orchestra.run.smithi039.stdout:5031 trunc from 0xb769ea0 to 0xd60e563 2024-09-14T06:17:29.587 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.587 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.587 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.587 INFO:teuthology.orchestra.run.smithi039.stdout:5035 writesame 0x4487dd thru 0x44ce07 data_size 0x17(0x462b bytes) 2024-09-14T06:17:29.591 INFO:teuthology.orchestra.run.smithi039.stdout:5036 writesame 0x18fb78 thru 0x192cbd data_size 0xe(0x3146 bytes) 2024-09-14T06:17:29.593 INFO:teuthology.orchestra.run.smithi039.stdout:5038 read 0x4aabd20 thru 0x4aaea6a (0x2d4b bytes) 2024-09-14T06:17:29.593 INFO:teuthology.orchestra.run.smithi039.stdout:5039 write 0x3355e9 thru 0x3422bd (0xccd5 bytes) 2024-09-14T06:17:29.597 INFO:teuthology.orchestra.run.smithi039.stdout:5040 write 0x78ee09f thru 0x78ee436 (0x398 bytes) 2024-09-14T06:17:29.603 INFO:teuthology.orchestra.run.smithi039.stdout:5041 write 0xd5c2aad thru 0xd5cd53e (0xaa92 bytes) 2024-09-14T06:17:29.619 INFO:teuthology.orchestra.run.smithi039.stdout:5042 trunc from 0xd60e563 to 0x54ecc60 2024-09-14T06:17:29.632 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:29.630+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:29.634 INFO:teuthology.orchestra.run.smithi039.stdout:5043 read 0x1ed6ee0 thru 0x1edf137 (0x8258 bytes) 2024-09-14T06:17:29.634 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.635 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.645 INFO:teuthology.orchestra.run.smithi039.stdout:5049 write 0x81c208c thru 0x81c76b5 (0x562a bytes) 2024-09-14T06:17:29.661 INFO:teuthology.orchestra.run.smithi039.stdout:5050 write 0xb57bcd2 thru 0xb5898ba (0xdbe9 bytes) 2024-09-14T06:17:29.667 INFO:teuthology.orchestra.run.smithi039.stdout:5053 read 0x425dc32 thru 0x4265bf1 (0x7fc0 bytes) 2024-09-14T06:17:29.667 INFO:teuthology.orchestra.run.smithi039.stdout:5054 punch from 0x8098343 to 0x80a1963, (0x9620 bytes) 2024-09-14T06:17:29.667 INFO:teuthology.orchestra.run.smithi039.stdout:5056 write 0x509510b thru 0x50a0536 (0xb42c bytes) 2024-09-14T06:17:29.673 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.673 INFO:teuthology.orchestra.run.smithi039.stdout:5063 write 0xf5813 thru 0xfab39 (0x5327 bytes) 2024-09-14T06:17:29.676 INFO:teuthology.orchestra.run.smithi039.stdout:5065 read 0xa359ebe thru 0xa363ffb (0xa13e bytes) 2024-09-14T06:17:29.676 INFO:teuthology.orchestra.run.smithi039.stdout:5066 write 0x1ac42f3 thru 0x1ac83d4 (0x40e2 bytes) 2024-09-14T06:17:29.680 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:29.680 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:29.680 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:29.680 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:29.683 INFO:teuthology.orchestra.run.smithi039.stdout:5067 writesame 0x31da215 thru 0x31daf36 data_size 0x2(0xd22 bytes) 2024-09-14T06:17:29.687 INFO:teuthology.orchestra.run.smithi039.stdout:5068 write 0x5045b41 thru 0x504ea6b (0x8f2b bytes) 2024-09-14T06:17:29.701 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:17:29.703 INFO:teuthology.orchestra.run.smithi039.stdout:5070 write 0xee34ae9 thru 0xee390a2 (0x45ba bytes) 2024-09-14T06:17:29.706 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:29.706 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:29.719 INFO:teuthology.orchestra.run.smithi039.stdout:5071 trunc from 0xee390a3 to 0x643d678 2024-09-14T06:17:29.727 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:29.728 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:29.732 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.732 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.732 INFO:teuthology.orchestra.run.smithi039.stdout:5076 writesame 0x9244c6 thru 0x925adf data_size 0x17(0x161a bytes) 2024-09-14T06:17:29.733 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:29.729+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:29.747 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:29.747 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:29.749 INFO:teuthology.orchestra.run.smithi039.stdout:5077 trunc from 0x643d678 to 0xe4c4015 2024-09-14T06:17:29.755 INFO:teuthology.orchestra.run.smithi039.stdout:5078 writesame 0x8aa3361 thru 0x8ab0565 data_size 0x1(0xd205 bytes) 2024-09-14T06:17:29.767 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:29.768 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:29.787 INFO:teuthology.orchestra.run.smithi039.stdout:5079 writesame 0xda903c thru 0xdaae5e data_size 0x1(0x1e23 bytes) 2024-09-14T06:17:29.789 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:29.789 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:29.794 INFO:teuthology.orchestra.run.smithi039.stdout:5080 write 0x117fd9b thru 0x118c153 (0xc3b9 bytes) 2024-09-14T06:17:29.805 INFO:teuthology.orchestra.run.smithi039.stdout:5082 trunc from 0xe4c4015 to 0xc6470dc 2024-09-14T06:17:29.812 INFO:teuthology.orchestra.run.smithi039.stdout:5083 write 0x89d69e8 thru 0x89db859 (0x4e72 bytes) 2024-09-14T06:17:29.812 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:29.813 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:29.818 INFO:teuthology.orchestra.run.smithi039.stdout:5084 write 0x98d775 thru 0x9935fa (0x5e86 bytes) 2024-09-14T06:17:29.821 INFO:teuthology.orchestra.run.smithi039.stdout:5085 read 0xb4f2d57 thru 0xb4fe05e (0xb308 bytes) 2024-09-14T06:17:29.824 INFO:teuthology.orchestra.run.smithi039.stdout:5086 trunc from 0xc6470dc to 0xabdb178 2024-09-14T06:17:29.831 INFO:teuthology.orchestra.run.smithi039.stdout:5087 read 0x91abda0 thru 0x91b6e06 (0xb067 bytes) 2024-09-14T06:17:29.834 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:29.829+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:29.837 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:29.839 INFO:teuthology.orchestra.run.smithi039.stdout:5088 write 0xba989c5 thru 0xba9dc63 (0x529f bytes) 2024-09-14T06:17:29.839 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:29.839 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:29.844 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.845 INFO:teuthology.orchestra.run.smithi039.stdout:5093 trunc from 0xba9dc64 to 0xb5ef1d9 2024-09-14T06:17:29.857 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:17:29.858 INFO:teuthology.orchestra.run.smithi039.stdout:5095 read 0x8fc47ba thru 0x8fc8aee (0x4335 bytes) 2024-09-14T06:17:29.858 INFO:teuthology.orchestra.run.smithi039.stdout:5096 writesame 0x6b5b88b thru 0x6b60e5d data_size 0x1(0x55d3 bytes) 2024-09-14T06:17:29.862 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:29.862 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:29.887 INFO:teuthology.orchestra.run.smithi039.stdout:5097 read 0xb3bfbd9 thru 0xb3c06de (0xb06 bytes) 2024-09-14T06:17:29.887 INFO:teuthology.orchestra.run.smithi039.stdout:5098 punch from 0x633b6ef to 0x6349730, (0xe041 bytes) 2024-09-14T06:17:29.888 INFO:teuthology.orchestra.run.smithi039.stdout:5099 punch from 0x2606fa to 0x26733c, (0x6c42 bytes) 2024-09-14T06:17:29.903 INFO:teuthology.orchestra.run.smithi039.stdout:5101 trunc from 0xb5ef1d9 to 0x5e4adc 2024-09-14T06:17:29.930 INFO:teuthology.orchestra.run.smithi039.stdout:5102 writesame 0x1634ed9 thru 0x1639bde data_size 0x1(0x4d06 bytes) 2024-09-14T06:17:29.935 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:29.933+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:29.939 INFO:teuthology.orchestra.run.smithi039.stdout:5103 write 0x2d9c1b6 thru 0x2da0af5 (0x4940 bytes) 2024-09-14T06:17:29.946 INFO:teuthology.orchestra.run.smithi039.stdout:5105 punch from 0x22ce738 to 0x22cfc71, (0x1539 bytes) 2024-09-14T06:17:29.946 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.946 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:29.959 INFO:teuthology.orchestra.run.smithi039.stdout:5112 writesame 0x7f00474 thru 0x7f0ec9f data_size 0x1a(0xe82c bytes) 2024-09-14T06:17:29.977 INFO:teuthology.orchestra.run.smithi039.stdout:5113 write 0xe86d05f thru 0xe870bec (0x3b8e bytes) 2024-09-14T06:17:29.992 INFO:teuthology.orchestra.run.smithi039.stdout:5115 trunc from 0xe870bed to 0x6e6cd31 2024-09-14T06:17:29.994 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:29.994 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:30.006 INFO:teuthology.orchestra.run.smithi039.stdout:5116 writesame 0x5150a24 thru 0x515313c data_size 0x1(0x2719 bytes) 2024-09-14T06:17:30.013 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:17:30.015 INFO:teuthology.orchestra.run.smithi039.stdout:5117 punch from 0x24fb712 to 0x24fe4f6, (0x2de4 bytes) 2024-09-14T06:17:30.026 INFO:teuthology.orchestra.run.smithi039.stdout:5120 write 0x8a059b9 thru 0x8a09139 (0x3781 bytes) 2024-09-14T06:17:30.030 INFO:teuthology.orchestra.run.smithi039.stdout:5122 read 0x5bcddbe thru 0x5bcedd3 (0x1016 bytes) 2024-09-14T06:17:30.036 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:30.034+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:30.046 INFO:teuthology.orchestra.run.smithi039.stdout:5123 write 0xee3b678 thru 0xee40e9f (0x5828 bytes) 2024-09-14T06:17:30.061 INFO:teuthology.orchestra.run.smithi039.stdout:5124 trunc from 0xee40ea0 to 0x9610e85 2024-09-14T06:17:30.074 INFO:teuthology.orchestra.run.smithi039.stdout:5125 punch from 0x2a7400a to 0x2a83343, (0xf339 bytes) 2024-09-14T06:17:30.084 INFO:teuthology.orchestra.run.smithi039.stdout:5126 write 0xc8b0644 thru 0xc8b1408 (0xdc5 bytes) 2024-09-14T06:17:30.089 INFO:teuthology.orchestra.run.smithi039.stdout:5127 writesame 0x52b8742 thru 0x52c2ed6 data_size 0x1(0xa795 bytes) 2024-09-14T06:17:30.134 INFO:teuthology.orchestra.run.smithi039.stdout:5128 writesame 0x437e70d thru 0x43851e6 data_size 0x2f(0x6ada bytes) 2024-09-14T06:17:30.136 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:30.134+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:30.141 INFO:teuthology.orchestra.run.smithi039.stdout:5129 punch from 0x7e497d4 to 0x7e55192, (0xb9be bytes) 2024-09-14T06:17:30.142 INFO:teuthology.orchestra.run.smithi039.stdout:5130 read 0x74cfc8 thru 0x753a39 (0x6a72 bytes) 2024-09-14T06:17:30.142 INFO:teuthology.orchestra.run.smithi039.stdout:5131 writesame 0xc1accbc thru 0xc1b4584 data_size 0xb(0x78c9 bytes) 2024-09-14T06:17:30.148 INFO:teuthology.orchestra.run.smithi039.stdout:5132 read 0x56294ab thru 0x5636417 (0xcf6d bytes) 2024-09-14T06:17:30.154 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:30.154 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:30.167 INFO:teuthology.orchestra.run.smithi039.stdout:5133 trunc from 0xc8b1409 to 0x28de096 2024-09-14T06:17:30.177 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:17:30.186 INFO:teuthology.orchestra.run.smithi039.stdout:5135 punch from 0x26e51d7 to 0x26ec55d, (0x7386 bytes) 2024-09-14T06:17:30.186 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:30.214 INFO:teuthology.orchestra.run.smithi039.stdout:5139 writesame 0xbb38c2d thru 0xbb3aede data_size 0x1(0x22b2 bytes) 2024-09-14T06:17:30.215 INFO:teuthology.orchestra.run.smithi039.stdout:5140 write 0x1e2d901 thru 0x1e2fad9 (0x21d9 bytes) 2024-09-14T06:17:30.219 INFO:teuthology.orchestra.run.smithi039.stdout:5141 read 0x4f4f5aa thru 0x4f54f90 (0x59e7 bytes) 2024-09-14T06:17:30.219 INFO:teuthology.orchestra.run.smithi039.stdout:5142 read 0x12d43be thru 0x12dd45e (0x90a1 bytes) 2024-09-14T06:17:30.219 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:30.219 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:30.219 INFO:teuthology.orchestra.run.smithi039.stdout:5145 read 0x5691352 thru 0x569eb6a (0xd819 bytes) 2024-09-14T06:17:30.220 INFO:teuthology.orchestra.run.smithi039.stdout:5146 writesame 0x404d9c2 thru 0x405a3ee data_size 0x1(0xca2d bytes) 2024-09-14T06:17:30.237 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:30.233+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:30.291 INFO:teuthology.orchestra.run.smithi039.stdout:5147 read 0xbee172 thru 0xbf37a6 (0x5635 bytes) 2024-09-14T06:17:30.291 INFO:teuthology.orchestra.run.smithi039.stdout:5148 punch from 0x4549e39 to 0x454d8ad, (0x3a74 bytes) 2024-09-14T06:17:30.291 INFO:teuthology.orchestra.run.smithi039.stdout:5149 punch from 0xaedd59b to 0xaeebd45, (0xe7aa bytes) 2024-09-14T06:17:30.291 INFO:teuthology.orchestra.run.smithi039.stdout:5150 write 0x5c83068 thru 0x5c8434c (0x12e5 bytes) 2024-09-14T06:17:30.297 INFO:teuthology.orchestra.run.smithi039.stdout:5151 read 0x300f3c6 thru 0x301a487 (0xb0c2 bytes) 2024-09-14T06:17:30.298 INFO:teuthology.orchestra.run.smithi039.stdout:5152 writesame 0x5b484d4 thru 0x5b57d9f data_size 0x2(0xf8cc bytes) 2024-09-14T06:17:30.310 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:30.310 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:30.324 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:17:30.340 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:17:30.341 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T06:17:30.348 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:30.348 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:30.348 INFO:teuthology.orchestra.run.smithi039.stdout:5156 punch from 0x23361e3 to 0x233e859, (0x8676 bytes) 2024-09-14T06:17:30.348 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:30.348 INFO:teuthology.orchestra.run.smithi039.stdout:5158 write 0xb44ecb3 thru 0xb457946 (0x8c94 bytes) 2024-09-14T06:17:30.348 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:30.346+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:30.353 INFO:teuthology.orchestra.run.smithi039.stdout:5160 write 0xe36771 thru 0xe3ecc6 (0x8556 bytes) 2024-09-14T06:17:30.360 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:30.360 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:30.360 INFO:teuthology.orchestra.run.smithi039.stdout:5163 read 0xd66f47 thru 0xd74d2b (0xdde5 bytes) 2024-09-14T06:17:30.361 INFO:teuthology.orchestra.run.smithi039.stdout:5164 read 0x2c57db1 thru 0x2c61662 (0x98b2 bytes) 2024-09-14T06:17:30.361 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:30.376 INFO:teuthology.orchestra.run.smithi039.stdout:5166 writesame 0xed89965 thru 0xed97b29 data_size 0x1(0xe1c5 bytes) 2024-09-14T06:17:30.376 INFO:teuthology.orchestra.run.smithi039.stdout:5167 read 0xabd2d40 thru 0xabddab0 (0xad71 bytes) 2024-09-14T06:17:30.376 INFO:teuthology.orchestra.run.smithi039.stdout:5168 write 0x7255c42 thru 0x725d8d2 (0x7c91 bytes) 2024-09-14T06:17:30.380 INFO:teuthology.orchestra.run.smithi039.stdout:5169 read 0x78f5dc8 thru 0x78ffb3f (0x9d78 bytes) 2024-09-14T06:17:30.380 INFO:teuthology.orchestra.run.smithi039.stdout:5170 read 0x459fdf9 thru 0x45a0b1b (0xd23 bytes) 2024-09-14T06:17:30.393 INFO:teuthology.orchestra.run.smithi039.stdout:5173 trunc from 0xed97b2a to 0x6ab51c0 2024-09-14T06:17:30.408 INFO:teuthology.orchestra.run.smithi039.stdout:5174 write 0xfa6d1 thru 0x103aa2 (0x93d2 bytes) 2024-09-14T06:17:30.427 INFO:teuthology.orchestra.run.smithi039.stdout:5175 write 0xac800e7 thru 0xac8479a (0x46b4 bytes) 2024-09-14T06:17:30.439 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:30.437+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:30.444 INFO:teuthology.orchestra.run.smithi039.stdout:5176 write 0xc97700a thru 0xc978dd6 (0x1dcd bytes) 2024-09-14T06:17:30.450 INFO:teuthology.orchestra.run.smithi039.stdout:5179 read 0xbbe9b26 thru 0xbbee6a8 (0x4b83 bytes) 2024-09-14T06:17:30.450 INFO:teuthology.orchestra.run.smithi039.stdout:5181 punch from 0x4c4bfd2 to 0x4c516f1, (0x571f bytes) 2024-09-14T06:17:30.450 INFO:teuthology.orchestra.run.smithi039.stdout:5182 punch from 0x1a7d7bd to 0x1a8bff4, (0xe837 bytes) 2024-09-14T06:17:30.450 INFO:teuthology.orchestra.run.smithi039.stdout:5183 read 0x1a13011 thru 0x1a1bb82 (0x8b72 bytes) 2024-09-14T06:17:30.450 INFO:teuthology.orchestra.run.smithi039.stdout:5184 write 0x95f2aba thru 0x9600cd4 (0xe21b bytes) 2024-09-14T06:17:30.472 INFO:teuthology.orchestra.run.smithi039.stdout:5185 trunc from 0xc978dd7 to 0x24f3ee4 2024-09-14T06:17:30.489 INFO:teuthology.orchestra.run.smithi039.stdout:5187 read 0x15fc184 thru 0x1601f01 (0x5d7e bytes) 2024-09-14T06:17:30.489 INFO:teuthology.orchestra.run.smithi039.stdout:5188 read 0x1a14a0c thru 0x1a17120 (0x2715 bytes) 2024-09-14T06:17:30.489 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:30.489 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:30.497 INFO:teuthology.orchestra.run.smithi039.stdout:5189 trunc from 0x24f3ee4 to 0x7621ba8 2024-09-14T06:17:30.515 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:17:30.521 INFO:teuthology.orchestra.run.smithi039.stdout:5190 write 0xe06946f thru 0xe078b23 (0xf6b5 bytes) 2024-09-14T06:17:30.525 INFO:teuthology.orchestra.run.smithi039.stdout:5191 read 0x3e3f502 thru 0x3e41695 (0x2194 bytes) 2024-09-14T06:17:30.525 INFO:teuthology.orchestra.run.smithi039.stdout:5192 read 0x1ddbd2a thru 0x1de0583 (0x485a bytes) 2024-09-14T06:17:30.526 INFO:teuthology.orchestra.run.smithi039.stdout:5193 write 0x6002308 thru 0x6011521 (0xf21a bytes) 2024-09-14T06:17:30.531 INFO:teuthology.orchestra.run.smithi039.stdout:5194 trunc from 0xe078b24 to 0xe67a377 2024-09-14T06:17:30.538 INFO:teuthology.orchestra.run.smithi039.stdout:5196 punch from 0x2e39100 to 0x2e4041f, (0x731f bytes) 2024-09-14T06:17:30.538 INFO:teuthology.orchestra.run.smithi039.stdout:5197 read 0xd1bf7c5 thru 0xd1bfca4 (0x4e0 bytes) 2024-09-14T06:17:30.539 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:30.538+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:30.558 INFO:teuthology.orchestra.run.smithi039.stdout:5198 trunc from 0xe67a377 to 0x8e9033 2024-09-14T06:17:30.589 INFO:teuthology.orchestra.run.smithi039.stdout:5199 write 0x8c15742 thru 0x8c1d8b5 (0x8174 bytes) 2024-09-14T06:17:30.594 INFO:teuthology.orchestra.run.smithi039.stdout:5202 writesame 0x8bfee33 thru 0x8c0997a data_size 0x1c(0xab48 bytes) 2024-09-14T06:17:30.599 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:30.599 INFO:teuthology.orchestra.run.smithi039.stdout:5206 punch from 0x89ba0f3 to 0x89c1cfd, (0x7c0a bytes) 2024-09-14T06:17:30.608 INFO:teuthology.orchestra.run.smithi039.stdout:5207 write 0xa656cfa thru 0xa65b57a (0x4881 bytes) 2024-09-14T06:17:30.612 INFO:teuthology.orchestra.run.smithi039.stdout:5208 read 0x574fd74 thru 0x575e6a7 (0xe934 bytes) 2024-09-14T06:17:30.612 INFO:teuthology.orchestra.run.smithi039.stdout:5209 writesame 0x26ee73b thru 0x26f9458 data_size 0x1(0xad1e bytes) 2024-09-14T06:17:30.640 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:30.638+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:30.653 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:30.653 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:30.668 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:17:30.670 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:30.670 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:30.680 INFO:teuthology.orchestra.run.smithi039.stdout:5213 write 0xa91c1d0 thru 0xa926a41 (0xa872 bytes) 2024-09-14T06:17:30.699 INFO:teuthology.orchestra.run.smithi039.stdout:5214 write 0xb9e50a0 thru 0xb9ecab0 (0x7a11 bytes) 2024-09-14T06:17:30.703 INFO:teuthology.orchestra.run.smithi039.stdout:5216 punch from 0xa307696 to 0xa309612, (0x1f7c bytes) 2024-09-14T06:17:30.703 INFO:teuthology.orchestra.run.smithi039.stdout:5218 trunc from 0xb9ecab1 to 0xbbc7b0b 2024-09-14T06:17:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:5220 write 0x4d920a1 thru 0x4d9610f (0x406f bytes) 2024-09-14T06:17:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:5221 trunc from 0xbbc7b0b to 0x2508f9a 2024-09-14T06:17:30.741 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:30.737+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:30.768 INFO:teuthology.orchestra.run.smithi039.stdout:5222 write 0x80fe0e6 thru 0x810a9bb (0xc8d6 bytes) 2024-09-14T06:17:30.774 INFO:teuthology.orchestra.run.smithi039.stdout:5223 punch from 0x866965 to 0x86ae9f, (0x453a bytes) 2024-09-14T06:17:30.774 INFO:teuthology.orchestra.run.smithi039.stdout:5225 write 0x167ff08 thru 0x16836cb (0x37c4 bytes) 2024-09-14T06:17:30.783 INFO:teuthology.orchestra.run.smithi039.stdout:5226 trunc from 0x810a9bc to 0x9f87f74 2024-09-14T06:17:30.789 INFO:teuthology.orchestra.run.smithi039.stdout:5227 read 0xb843bb thru 0xb8547b (0x10c1 bytes) 2024-09-14T06:17:30.789 INFO:teuthology.orchestra.run.smithi039.stdout:5228 read 0x75523a2 thru 0x75542fd (0x1f5c bytes) 2024-09-14T06:17:30.789 INFO:teuthology.orchestra.run.smithi039.stdout:5229 punch from 0x410a0f8 to 0x41104d9, (0x63e1 bytes) 2024-09-14T06:17:30.789 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:30.789 INFO:teuthology.orchestra.run.smithi039.stdout:5231 writesame 0x41d70e0 thru 0x41dc8ad data_size 0x1(0x57ce bytes) 2024-09-14T06:17:30.803 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:30.803 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:30.817 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:17:30.818 INFO:teuthology.orchestra.run.smithi039.stdout:5232 write 0x1c76da5 thru 0x1c7e203 (0x745f bytes) 2024-09-14T06:17:30.821 INFO:teuthology.orchestra.run.smithi039.stdout:5233 writesame 0x217f320 thru 0x218ea9b data_size 0x5e(0xf77c bytes) 2024-09-14T06:17:30.828 INFO:teuthology.orchestra.run.smithi039.stdout:5236 write 0x6c20afc thru 0x6c2270a (0x1c0f bytes) 2024-09-14T06:17:30.832 INFO:teuthology.orchestra.run.smithi039.stdout:5238 read 0x1477725 thru 0x1482625 (0xaf01 bytes) 2024-09-14T06:17:30.833 INFO:teuthology.orchestra.run.smithi039.stdout:5239 punch from 0x89a3999 to 0x89b1096, (0xd6fd bytes) 2024-09-14T06:17:30.841 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:30.838+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:30.846 INFO:teuthology.orchestra.run.smithi039.stdout:5240 write 0xe11defa thru 0xe122053 (0x415a bytes) 2024-09-14T06:17:30.852 INFO:teuthology.orchestra.run.smithi039.stdout:5241 write 0x21b9436 thru 0x21c4f1f (0xbaea bytes) 2024-09-14T06:17:30.856 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:30.856 INFO:teuthology.orchestra.run.smithi039.stdout:5243 read 0xdf8f9b7 thru 0xdf8fbd9 (0x223 bytes) 2024-09-14T06:17:30.857 INFO:teuthology.orchestra.run.smithi039.stdout:5244 read 0x6535aec thru 0x653d5b0 (0x7ac5 bytes) 2024-09-14T06:17:30.857 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:30.872 INFO:teuthology.orchestra.run.smithi039.stdout:5247 trunc from 0xe122054 to 0x40719f0 2024-09-14T06:17:30.889 INFO:teuthology.orchestra.run.smithi039.stdout:5248 writesame 0x14d4af6 thru 0x14d66e2 data_size 0x1(0x1bed bytes) 2024-09-14T06:17:30.901 INFO:teuthology.orchestra.run.smithi039.stdout:5251 read 0x37cc6c8 thru 0x37d2fa7 (0x68e0 bytes) 2024-09-14T06:17:30.901 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:30.901 INFO:teuthology.orchestra.run.smithi039.stdout:5254 punch from 0x183b300 to 0x1840e3a, (0x5b3a bytes) 2024-09-14T06:17:30.920 INFO:teuthology.orchestra.run.smithi039.stdout:5255 write 0xcacad23 thru 0xcacefde (0x42bc bytes) 2024-09-14T06:17:30.937 INFO:teuthology.orchestra.run.smithi039.stdout:5256 trunc from 0xcacefdf to 0x5e18658 2024-09-14T06:17:30.942 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:30.937+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:30.956 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:30.956 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:30.961 INFO:teuthology.orchestra.run.smithi039.stdout:5257 trunc from 0x5e18658 to 0x7e6764c 2024-09-14T06:17:30.974 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:17:30.997 INFO:teuthology.orchestra.run.smithi039.stdout:5259 writesame 0xdb37306 thru 0xdb38868 data_size 0x19(0x1563 bytes) 2024-09-14T06:17:30.997 INFO:teuthology.orchestra.run.smithi039.stdout:5260 write 0x31faba9 thru 0x31fe7ba (0x3c12 bytes) 2024-09-14T06:17:31.005 INFO:teuthology.orchestra.run.smithi039.stdout:5261 write 0xa73fcf1 thru 0xa749f9e (0xa2ae bytes) 2024-09-14T06:17:31.014 INFO:teuthology.orchestra.run.smithi039.stdout:5262 writesame 0xc5ba8bb thru 0xc5c627d data_size 0x1(0xb9c3 bytes) 2024-09-14T06:17:31.043 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:31.042+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:5263 punch from 0x66f57e7 to 0x66ff064, (0x987d bytes) 2024-09-14T06:17:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:5264 read 0x493cb44 thru 0x494b957 (0xee14 bytes) 2024-09-14T06:17:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:5265 punch from 0x7a7b32 to 0x7af158, (0x7626 bytes) 2024-09-14T06:17:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:5266 write 0x406d2a2 thru 0x406e247 (0xfa6 bytes) 2024-09-14T06:17:31.055 INFO:teuthology.orchestra.run.smithi039.stdout:5267 write 0x2359630 thru 0x236872c (0xf0fd bytes) 2024-09-14T06:17:31.076 INFO:teuthology.orchestra.run.smithi039.stdout:5270 trunc from 0xdb38869 to 0x38723d1 2024-09-14T06:17:31.099 INFO:teuthology.orchestra.run.smithi039.stdout:5271 trunc from 0x38723d1 to 0x6a67fc 2024-09-14T06:17:31.120 INFO:teuthology.orchestra.run.smithi039.stdout:5272 punch from 0xc74da to 0xc95b3, (0x20d9 bytes) 2024-09-14T06:17:31.120 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:31.120 INFO:teuthology.orchestra.run.smithi039.stdout:5274 punch from 0x3b4a20 to 0x3bce0b, (0x83eb bytes) 2024-09-14T06:17:31.120 INFO:teuthology.orchestra.run.smithi039.stdout:5275 read 0x9dde3 thru 0xa70b3 (0x92d1 bytes) 2024-09-14T06:17:31.121 INFO:teuthology.orchestra.run.smithi039.stdout:5276 read 0x26a5dd thru 0x26d0fa (0x2b1e bytes) 2024-09-14T06:17:31.121 INFO:teuthology.orchestra.run.smithi039.stdout:5278 punch from 0x223f42 to 0x227df0, (0x3eae bytes) 2024-09-14T06:17:31.122 INFO:teuthology.orchestra.run.smithi039.stdout:5279 read 0x4a930a thru 0x4a9c14 (0x90b bytes) 2024-09-14T06:17:31.122 INFO:teuthology.orchestra.run.smithi039.stdout:5281 read 0x30e573 thru 0x3147ef (0x627d bytes) 2024-09-14T06:17:31.140 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:31.141 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:31.143 INFO:teuthology.orchestra.run.smithi039.stdout:5282 trunc from 0x6a67fc to 0x9a40750 2024-09-14T06:17:31.143 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:31.142+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:31.154 INFO:teuthology.orchestra.run.smithi039.stdout:5283 read 0x91d2f23 thru 0x91dcb2d (0x9c0b bytes) 2024-09-14T06:17:31.159 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:17:31.164 INFO:teuthology.orchestra.run.smithi039.stdout:5285 write 0xa3d51e6 thru 0xa3dd05d (0x7e78 bytes) 2024-09-14T06:17:31.171 INFO:teuthology.orchestra.run.smithi039.stdout:5286 write 0x1fb19e thru 0x2018d2 (0x6735 bytes) 2024-09-14T06:17:31.175 INFO:teuthology.orchestra.run.smithi039.stdout:5287 write 0x8ab2865 thru 0x8ab924d (0x69e9 bytes) 2024-09-14T06:17:31.181 INFO:teuthology.orchestra.run.smithi039.stdout:5288 read 0x53aa05b thru 0x53abfa4 (0x1f4a bytes) 2024-09-14T06:17:31.181 INFO:teuthology.orchestra.run.smithi039.stdout:5289 writesame 0x1e43100 thru 0x1e434a1 data_size 0xf(0x3a2 bytes) 2024-09-14T06:17:31.185 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:31.185 INFO:teuthology.orchestra.run.smithi039.stdout:5291 punch from 0x878a114 to 0x878b3e5, (0x12d1 bytes) 2024-09-14T06:17:31.198 INFO:teuthology.orchestra.run.smithi039.stdout:5292 writesame 0xd1960fa thru 0xd1a12c8 data_size 0x1(0xb1cf bytes) 2024-09-14T06:17:31.198 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:31.198 INFO:teuthology.orchestra.run.smithi039.stdout:5297 write 0x14b94b9 thru 0x14b9ded (0x935 bytes) 2024-09-14T06:17:31.201 INFO:teuthology.orchestra.run.smithi039.stdout:5298 punch from 0xc29ca1a to 0xc2a1532, (0x4b18 bytes) 2024-09-14T06:17:31.202 INFO:teuthology.orchestra.run.smithi039.stdout:5299 read 0xaa6b3a7 thru 0xaa6c752 (0x13ac bytes) 2024-09-14T06:17:31.211 INFO:teuthology.orchestra.run.smithi039.stdout:5300 writesame 0xe77fbc7 thru 0xe788a4d data_size 0x1f(0x8e87 bytes) 2024-09-14T06:17:31.220 INFO:teuthology.orchestra.run.smithi039.stdout:5301 trunc from 0xe788a4e to 0x9499743 2024-09-14T06:17:31.244 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:31.241+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:31.252 INFO:teuthology.orchestra.run.smithi039.stdout:5302 write 0xeb66f15 thru 0xeb7276f (0xb85b bytes) 2024-09-14T06:17:31.258 INFO:teuthology.orchestra.run.smithi039.stdout:5303 punch from 0xc101116 to 0xc10740f, (0x62f9 bytes) 2024-09-14T06:17:31.258 INFO:teuthology.orchestra.run.smithi039.stdout:5305 write 0x502fe93 thru 0x5036c0d (0x6d7b bytes) 2024-09-14T06:17:31.264 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:31.264 INFO:teuthology.orchestra.run.smithi039.stdout:5307 write 0x2b84179 thru 0x2b842b8 (0x140 bytes) 2024-09-14T06:17:31.270 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:31.270 INFO:teuthology.orchestra.run.smithi039.stdout:5309 punch from 0x3eaca0c to 0x3ebb00a, (0xe5fe bytes) 2024-09-14T06:17:31.270 INFO:teuthology.orchestra.run.smithi039.stdout:5310 writesame 0xa4164a1 thru 0xa41bb97 data_size 0x29(0x56f7 bytes) 2024-09-14T06:17:31.274 INFO:teuthology.orchestra.run.smithi039.stdout:5315 read 0x92fd67e thru 0x930c61d (0xefa0 bytes) 2024-09-14T06:17:31.274 INFO:teuthology.orchestra.run.smithi039.stdout:5317 write 0x9e14ca thru 0x9e5114 (0x3c4b bytes) 2024-09-14T06:17:31.280 INFO:teuthology.orchestra.run.smithi039.stdout:5318 write 0xb5ef8ce thru 0xb5fefb1 (0xf6e4 bytes) 2024-09-14T06:17:31.285 INFO:teuthology.orchestra.run.smithi039.stdout:5319 write 0xcda09d1 thru 0xcdaee03 (0xe433 bytes) 2024-09-14T06:17:31.291 INFO:teuthology.orchestra.run.smithi039.stdout:5320 writesame 0xcddb366 thru 0xcde0c01 data_size 0x35(0x589c bytes) 2024-09-14T06:17:31.296 INFO:teuthology.orchestra.run.smithi039.stdout:5321 write 0x59548eb thru 0x5956eef (0x2605 bytes) 2024-09-14T06:17:31.301 INFO:teuthology.orchestra.run.smithi039.stdout:5322 write 0x618788b thru 0x619203f (0xa7b5 bytes) 2024-09-14T06:17:31.306 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:31.306 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:31.315 INFO:teuthology.orchestra.run.smithi039.stdout:5323 trunc from 0xeb72770 to 0xa2a69ca 2024-09-14T06:17:31.329 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:17:31.346 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:31.341+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:31.349 INFO:teuthology.orchestra.run.smithi039.stdout:5325 trunc from 0xa2a69ca to 0x838991 2024-09-14T06:17:31.395 INFO:teuthology.orchestra.run.smithi039.stdout:5326 write 0xc1d9b93 thru 0xc1da9a0 (0xe0e bytes) 2024-09-14T06:17:31.399 INFO:teuthology.orchestra.run.smithi039.stdout:5329 punch from 0x3dccf0c to 0x3dcf826, (0x291a bytes) 2024-09-14T06:17:31.400 INFO:teuthology.orchestra.run.smithi039.stdout:5331 punch from 0x8b620ae to 0x8b680b7, (0x6009 bytes) 2024-09-14T06:17:31.400 INFO:teuthology.orchestra.run.smithi039.stdout:5335 writesame 0x568986e thru 0x5697292 data_size 0x99(0xda25 bytes) 2024-09-14T06:17:31.404 INFO:teuthology.orchestra.run.smithi039.stdout:5336 read 0x36b26d3 thru 0x36b3f1e (0x184c bytes) 2024-09-14T06:17:31.405 INFO:teuthology.orchestra.run.smithi039.stdout:5338 read 0xac2ab2e thru 0xac32a79 (0x7f4c bytes) 2024-09-14T06:17:31.405 INFO:teuthology.orchestra.run.smithi039.stdout:5339 read 0x1d2a39c thru 0x1d3600d (0xbc72 bytes) 2024-09-14T06:17:31.405 INFO:teuthology.orchestra.run.smithi039.stdout:5340 write 0xb84bc00 thru 0xb84f49a (0x389b bytes) 2024-09-14T06:17:31.410 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:31.410 INFO:teuthology.orchestra.run.smithi039.stdout:5343 write 0x2a94c09 thru 0x2a9f300 (0xa6f8 bytes) 2024-09-14T06:17:31.414 INFO:teuthology.orchestra.run.smithi039.stdout:5344 punch from 0x6a3ced4 to 0x6a3dd3b, (0xe67 bytes) 2024-09-14T06:17:31.414 INFO:teuthology.orchestra.run.smithi039.stdout:5345 writesame 0x3b89cbf thru 0x3b98241 data_size 0x5(0xe583 bytes) 2024-09-14T06:17:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:5347 punch from 0x813f893 to 0x8144280, (0x49ed bytes) 2024-09-14T06:17:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:5348 read 0x51a2d39 thru 0x51b04d0 (0xd798 bytes) 2024-09-14T06:17:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:5349 read 0x4eeaa37 thru 0x4ef1bd3 (0x719d bytes) 2024-09-14T06:17:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:5351 read 0xa773db7 thru 0xa77c600 (0x884a bytes) 2024-09-14T06:17:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:5352 punch from 0x717e7f0 to 0x71823fc, (0x3c0c bytes) 2024-09-14T06:17:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:5353 write 0x77fe67 thru 0x788aa8 (0x8c42 bytes) 2024-09-14T06:17:31.440 INFO:teuthology.orchestra.run.smithi039.stdout:5354 read 0x4aa83fa thru 0x4ab80f0 (0xfcf7 bytes) 2024-09-14T06:17:31.440 INFO:teuthology.orchestra.run.smithi039.stdout:5355 punch from 0x6ee3b98 to 0x6ef0fec, (0xd454 bytes) 2024-09-14T06:17:31.440 INFO:teuthology.orchestra.run.smithi039.stdout:5356 write 0x4f384d thru 0x4f43ff (0xbb3 bytes) 2024-09-14T06:17:31.442 INFO:teuthology.orchestra.run.smithi039.stdout:5357 punch from 0x2fe055d to 0x2fec820, (0xc2c3 bytes) 2024-09-14T06:17:31.442 INFO:teuthology.orchestra.run.smithi039.stdout:5359 punch from 0x72d8599 to 0x72dd7b4, (0x521b bytes) 2024-09-14T06:17:31.442 INFO:teuthology.orchestra.run.smithi039.stdout:5360 punch from 0x459c5e1 to 0x45ac29e, (0xfcbd bytes) 2024-09-14T06:17:31.446 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:31.442+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:31.448 INFO:teuthology.orchestra.run.smithi039.stdout:5361 writesame 0xc42889b thru 0xc42f006 data_size 0x2(0x676c bytes) 2024-09-14T06:17:31.449 INFO:teuthology.orchestra.run.smithi039.stdout:5362 read 0x43e5862 thru 0x43f326a (0xda09 bytes) 2024-09-14T06:17:31.449 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:31.449 INFO:teuthology.orchestra.run.smithi039.stdout:5366 punch from 0x3069ef2 to 0x306c576, (0x2684 bytes) 2024-09-14T06:17:31.449 INFO:teuthology.orchestra.run.smithi039.stdout:5367 read 0x69109c thru 0x694250 (0x31b5 bytes) 2024-09-14T06:17:31.450 INFO:teuthology.orchestra.run.smithi039.stdout:5368 read 0x1016c88 thru 0x101bd26 (0x509f bytes) 2024-09-14T06:17:31.450 INFO:teuthology.orchestra.run.smithi039.stdout:5371 writesame 0x69b726e thru 0x69bbafb data_size 0x25(0x488e bytes) 2024-09-14T06:17:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:5372 punch from 0xa0a791e to 0xa0ab0d5, (0x37b7 bytes) 2024-09-14T06:17:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:5373 read 0x1f88946 thru 0x1f90bbf (0x827a bytes) 2024-09-14T06:17:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:5374 punch from 0x6fd2fe2 to 0x6fd5be0, (0x2bfe bytes) 2024-09-14T06:17:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:5375 write 0x8470a1b thru 0x847ab68 (0xa14e bytes) 2024-09-14T06:17:31.463 INFO:teuthology.orchestra.run.smithi039.stdout:5376 read 0x66b22a2 thru 0x66b4de7 (0x2b46 bytes) 2024-09-14T06:17:31.470 INFO:teuthology.orchestra.run.smithi039.stdout:5377 trunc from 0xc42f007 to 0x8d39dd4 2024-09-14T06:17:31.489 INFO:teuthology.orchestra.run.smithi039.stdout:5380 writesame 0x89bda34 thru 0x89bebb4 data_size 0x1(0x1181 bytes) 2024-09-14T06:17:31.494 INFO:teuthology.orchestra.run.smithi039.stdout:5383 read 0x227b4cf thru 0x228b32e (0xfe60 bytes) 2024-09-14T06:17:31.494 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:31.495 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:31.509 INFO:teuthology.orchestra.run.smithi039.stdout:5385 write 0xace4bf5 thru 0xace901e (0x442a bytes) 2024-09-14T06:17:31.513 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:17:31.516 INFO:teuthology.orchestra.run.smithi039.stdout:5386 writesame 0x5b7891d thru 0x5b87529 data_size 0x1(0xec0d bytes) 2024-09-14T06:17:31.549 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T06:17:31.558 INFO:teuthology.orchestra.run.smithi039.stdout:5387 read 0x65ebb4d thru 0x65eee5f (0x3313 bytes) 2024-09-14T06:17:31.558 INFO:teuthology.orchestra.run.smithi039.stdout:5389 write 0x3264ca0 thru 0x32742f3 (0xf654 bytes) 2024-09-14T06:17:31.559 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:31.557+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:31.564 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:17:31.565 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:31.566 INFO:teuthology.orchestra.run.smithi039.stdout:5392 punch from 0x982bc27 to 0x983902e, (0xd407 bytes) 2024-09-14T06:17:31.566 INFO:teuthology.orchestra.run.smithi039.stdout:5393 read 0x149e99b thru 0x14a8e63 (0xa4c9 bytes) 2024-09-14T06:17:31.575 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T06:17:31.576 INFO:teuthology.orchestra.run.smithi039.stdout:5396 write 0xbc30def thru 0xbc3d69e (0xc8b0 bytes) 2024-09-14T06:17:31.582 INFO:teuthology.orchestra.run.smithi039.stdout:5397 writesame 0x49d6a8f thru 0x49e3767 data_size 0x1(0xccd9 bytes) 2024-09-14T06:17:31.621 INFO:teuthology.orchestra.run.smithi039.stdout:5398 write 0x5e1220e thru 0x5e1c154 (0x9f47 bytes) 2024-09-14T06:17:31.623 INFO:teuthology.orchestra.run.smithi039.stdout:5400 read 0x5430bbf thru 0x5439998 (0x8dda bytes) 2024-09-14T06:17:31.623 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:31.624 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:31.639 INFO:teuthology.orchestra.run.smithi039.stdout:5404 trunc from 0xbc3d69f to 0x228d6c7 2024-09-14T06:17:31.648 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:31.646+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:31.655 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:31.655 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:31.657 INFO:teuthology.orchestra.run.smithi039.stdout:5405 write 0xaa0fc0 thru 0xab040d (0xf44e bytes) 2024-09-14T06:17:31.660 INFO:teuthology.orchestra.run.smithi039.stdout:5407 writesame 0x1c7ab6 thru 0x1d081e data_size 0xb(0x8d69 bytes) 2024-09-14T06:17:31.670 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:17:31.672 INFO:teuthology.orchestra.run.smithi039.stdout:5409 writesame 0x2a728ba thru 0x2a799b4 data_size 0x1f(0x70fb bytes) 2024-09-14T06:17:31.688 INFO:teuthology.orchestra.run.smithi039.stdout:5410 write 0x7b0970f thru 0x7b11857 (0x8149 bytes) 2024-09-14T06:17:31.693 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:31.693 INFO:teuthology.orchestra.run.smithi039.stdout:5412 read 0x134988d thru 0x13555f6 (0xbd6a bytes) 2024-09-14T06:17:31.705 INFO:teuthology.orchestra.run.smithi039.stdout:5413 trunc from 0x7b11858 to 0x61b638 2024-09-14T06:17:31.746 INFO:teuthology.orchestra.run.smithi039.stdout:5415 write 0x8322c40 thru 0x832cfb4 (0xa375 bytes) 2024-09-14T06:17:31.749 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:31.746+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:31.759 INFO:teuthology.orchestra.run.smithi039.stdout:5416 writesame 0xa709a2d thru 0xa70d264 data_size 0x1c(0x3838 bytes) 2024-09-14T06:17:31.765 INFO:teuthology.orchestra.run.smithi039.stdout:5417 trunc from 0xa70d265 to 0x6a615d4 2024-09-14T06:17:31.777 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:31.789 INFO:teuthology.orchestra.run.smithi039.stdout:5420 write 0xa5459c0 thru 0xa548ff3 (0x3634 bytes) 2024-09-14T06:17:31.793 INFO:teuthology.orchestra.run.smithi039.stdout:5421 read 0x8cab12e thru 0x8cba78a (0xf65d bytes) 2024-09-14T06:17:31.793 INFO:teuthology.orchestra.run.smithi039.stdout:5422 writesame 0x39a0c2c thru 0x39a2654 data_size 0x1(0x1a29 bytes) 2024-09-14T06:17:31.810 INFO:teuthology.orchestra.run.smithi039.stdout:5423 trunc from 0xa548ff4 to 0xebb000c 2024-09-14T06:17:31.817 INFO:teuthology.orchestra.run.smithi039.stdout:5424 write 0xc002345 thru 0xc00c50f (0xa1cb bytes) 2024-09-14T06:17:31.822 INFO:teuthology.orchestra.run.smithi039.stdout:5425 write 0x14804bf thru 0x14845ad (0x40ef bytes) 2024-09-14T06:17:31.828 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:31.828 INFO:teuthology.orchestra.run.smithi039.stdout:5428 punch from 0x69bc91e to 0x69bea5d, (0x213f bytes) 2024-09-14T06:17:31.828 INFO:teuthology.orchestra.run.smithi039.stdout:5429 writesame 0x12e6912 thru 0x12efedb data_size 0x2a(0x95ca bytes) 2024-09-14T06:17:31.834 INFO:teuthology.orchestra.run.smithi039.stdout:5433 punch from 0xcbfbc96 to 0xcc0bb99, (0xff03 bytes) 2024-09-14T06:17:31.834 INFO:teuthology.orchestra.run.smithi039.stdout:5434 write 0x9b84d8 thru 0x9bdf97 (0x5ac0 bytes) 2024-09-14T06:17:31.837 INFO:teuthology.orchestra.run.smithi039.stdout:5435 write 0xc2dbbc thru 0xc399df (0xbe24 bytes) 2024-09-14T06:17:31.840 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:31.840 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:31.841 INFO:teuthology.orchestra.run.smithi039.stdout:5437 write 0x60b740c thru 0x60c5e62 (0xea57 bytes) 2024-09-14T06:17:31.847 INFO:teuthology.orchestra.run.smithi039.stdout:5438 write 0x2e13dc8 thru 0x2e238c2 (0xfafb bytes) 2024-09-14T06:17:31.849 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:31.846+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:31.854 INFO:teuthology.orchestra.run.smithi039.stdout:5439 write 0xd184e8c thru 0xd18bebf (0x7034 bytes) 2024-09-14T06:17:31.855 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:17:31.858 INFO:teuthology.orchestra.run.smithi039.stdout:5440 write 0x5b87e17 thru 0x5b8c36f (0x4559 bytes) 2024-09-14T06:17:31.862 INFO:teuthology.orchestra.run.smithi039.stdout:5441 read 0xb966098 thru 0xb96d1c1 (0x712a bytes) 2024-09-14T06:17:31.862 INFO:teuthology.orchestra.run.smithi039.stdout:5443 punch from 0x367dee7 to 0x367faa7, (0x1bc0 bytes) 2024-09-14T06:17:31.862 INFO:teuthology.orchestra.run.smithi039.stdout:5444 read 0xc65d10d thru 0xc65fcf4 (0x2be8 bytes) 2024-09-14T06:17:31.862 INFO:teuthology.orchestra.run.smithi039.stdout:5445 writesame 0xa5f0bc6 thru 0xa5fc0d0 data_size 0x1(0xb50b bytes) 2024-09-14T06:17:31.883 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:17:31.892 INFO:teuthology.orchestra.run.smithi039.stdout:5446 read 0x7589ab6 thru 0x758f894 (0x5ddf bytes) 2024-09-14T06:17:31.893 INFO:teuthology.orchestra.run.smithi039.stdout:5447 read 0xbe2bb3d thru 0xbe2fd20 (0x41e4 bytes) 2024-09-14T06:17:31.893 INFO:teuthology.orchestra.run.smithi039.stdout:5448 writesame 0x16098d0 thru 0x1613565 data_size 0x83(0x9c96 bytes) 2024-09-14T06:17:31.894 INFO:teuthology.orchestra.run.smithi039.stdout:5449 read 0xd5d9de3 thru 0xd5e5840 (0xba5e bytes) 2024-09-14T06:17:31.895 INFO:teuthology.orchestra.run.smithi039.stdout:5452 write 0xbe4e2c6 thru 0xbe5e018 (0xfd53 bytes) 2024-09-14T06:17:31.899 INFO:teuthology.orchestra.run.smithi039.stdout:5453 read 0x6af629f thru 0x6af964b (0x33ad bytes) 2024-09-14T06:17:31.918 INFO:teuthology.orchestra.run.smithi039.stdout:5456 trunc from 0xebb000c to 0x3c6f382 2024-09-14T06:17:31.931 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:31.942 INFO:teuthology.orchestra.run.smithi039.stdout:5458 writesame 0x6a056d1 thru 0x6a0f421 data_size 0x17(0x9d51 bytes) 2024-09-14T06:17:31.943 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:31.943 INFO:teuthology.orchestra.run.smithi039.stdout:5460 read 0x3bcd9d8 thru 0x3bd9ac8 (0xc0f1 bytes) 2024-09-14T06:17:31.943 INFO:teuthology.orchestra.run.smithi039.stdout:5463 write 0x5ca3e7b thru 0x5cae27d (0xa403 bytes) 2024-09-14T06:17:31.946 INFO:teuthology.orchestra.run.smithi039.stdout:5464 punch from 0x1c89a48 to 0x1c976e1, (0xdc99 bytes) 2024-09-14T06:17:31.946 INFO:teuthology.orchestra.run.smithi039.stdout:5465 read 0x5df16a thru 0x5e379a (0x4631 bytes) 2024-09-14T06:17:31.947 INFO:teuthology.orchestra.run.smithi039.stdout:5466 read 0x42b36b2 thru 0x42bd0e8 (0x9a37 bytes) 2024-09-14T06:17:31.947 INFO:teuthology.orchestra.run.smithi039.stdout:5467 writesame 0xdaf24d thru 0xdafa7e data_size 0x1(0x832 bytes) 2024-09-14T06:17:31.950 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:31.946+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:31.961 INFO:teuthology.orchestra.run.smithi039.stdout:5468 write 0xa7d3982 thru 0xa7e0777 (0xcdf6 bytes) 2024-09-14T06:17:31.965 INFO:teuthology.orchestra.run.smithi039.stdout:5471 punch from 0x77aa553 to 0x77b1f54, (0x7a01 bytes) 2024-09-14T06:17:31.973 INFO:teuthology.orchestra.run.smithi039.stdout:5472 trunc from 0xa7e0778 to 0x617bae0 2024-09-14T06:17:31.978 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:31.978 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:31.993 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:17:32.005 INFO:teuthology.orchestra.run.smithi039.stdout:5473 writesame 0xeb300a8 thru 0xeb309d7 data_size 0x1c(0x930 bytes) 2024-09-14T06:17:32.022 INFO:teuthology.orchestra.run.smithi039.stdout:5474 trunc from 0xeb309d8 to 0x30a5018 2024-09-14T06:17:32.037 INFO:teuthology.orchestra.run.smithi039.stdout:5475 read 0xcebff2 thru 0xcf080b (0x481a bytes) 2024-09-14T06:17:32.049 INFO:teuthology.orchestra.run.smithi039.stdout:5477 write 0x6d89c2a thru 0x6d97361 (0xd738 bytes) 2024-09-14T06:17:32.051 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:32.050+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:32.053 INFO:teuthology.orchestra.run.smithi039.stdout:5478 writesame 0x488e595 thru 0x4895e44 data_size 0x8(0x78b0 bytes) 2024-09-14T06:17:32.057 INFO:teuthology.orchestra.run.smithi039.stdout:5479 read 0x5704c8a thru 0x5714723 (0xfa9a bytes) 2024-09-14T06:17:32.058 INFO:teuthology.orchestra.run.smithi039.stdout:5480 read 0x1281c6b thru 0x128fc82 (0xe018 bytes) 2024-09-14T06:17:32.062 INFO:teuthology.orchestra.run.smithi039.stdout:5481 trunc from 0x6d97362 to 0x424a096 2024-09-14T06:17:32.066 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:32.067 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:32.073 INFO:teuthology.orchestra.run.smithi039.stdout:5482 punch from 0xa7efc5 to 0xa88fea, (0xa025 bytes) 2024-09-14T06:17:32.074 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:32.074 INFO:teuthology.orchestra.run.smithi039.stdout:5485 writesame 0x19113e4 thru 0x1911919 data_size 0x1(0x536 bytes) 2024-09-14T06:17:32.077 INFO:teuthology.orchestra.run.smithi039.stdout:5486 write 0x3541a2a thru 0x35469a4 (0x4f7b bytes) 2024-09-14T06:17:32.086 INFO:teuthology.orchestra.run.smithi039.stdout:5488 trunc from 0x424a096 to 0x11a6a65 2024-09-14T06:17:32.086 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:32.087 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:32.101 INFO:teuthology.orchestra.run.smithi039.stdout:5489 read 0xcbd0de thru 0xcbe1e1 (0x1104 bytes) 2024-09-14T06:17:32.102 INFO:teuthology.orchestra.run.smithi039.stdout:5490 trunc from 0x11a6a65 to 0x36b4b31 2024-09-14T06:17:32.105 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:32.106 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:32.115 INFO:teuthology.orchestra.run.smithi039.stdout:5491 write 0x3d39e64 thru 0x3d3f4b2 (0x564f bytes) 2024-09-14T06:17:32.118 INFO:teuthology.orchestra.run.smithi039.stdout:5493 punch from 0x2ed0567 to 0x2ed2c7a, (0x2713 bytes) 2024-09-14T06:17:32.118 INFO:teuthology.orchestra.run.smithi039.stdout:5494 write 0xe70608 thru 0xe7caf4 (0xc4ed bytes) 2024-09-14T06:17:32.120 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:32.120 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:32.120 INFO:teuthology.orchestra.run.smithi039.stdout:5495 punch from 0x3573b8b to 0x3574e2a, (0x129f bytes) 2024-09-14T06:17:32.121 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:32.121 INFO:teuthology.orchestra.run.smithi039.stdout:5497 read 0x1aa5445 thru 0x1aa65e3 (0x119f bytes) 2024-09-14T06:17:32.126 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:32.126 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:32.132 INFO:teuthology.orchestra.run.smithi039.stdout:5498 writesame 0x74b7387 thru 0x74ba276 data_size 0x8(0x2ef0 bytes) 2024-09-14T06:17:32.134 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:17:32.144 INFO:teuthology.orchestra.run.smithi039.stdout:5500 write 0xbc72462 thru 0xbc7dd58 (0xb8f7 bytes) 2024-09-14T06:17:32.147 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:32.147 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:32.148 INFO:teuthology.orchestra.run.smithi039.stdout:5501 read 0xb78ea83 thru 0xb79e1e9 (0xf767 bytes) 2024-09-14T06:17:32.148 INFO:teuthology.orchestra.run.smithi039.stdout:5502 read 0x870e512 thru 0x871ac15 (0xc704 bytes) 2024-09-14T06:17:32.148 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:32.148 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:32.148 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:32.148 INFO:teuthology.orchestra.run.smithi039.stdout:5507 read 0x4005d61 thru 0x400a6cc (0x496c bytes) 2024-09-14T06:17:32.152 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:32.150+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:32.157 INFO:teuthology.orchestra.run.smithi039.stdout:5508 write 0xdbc0634 thru 0xdbc0f6e (0x93b bytes) 2024-09-14T06:17:32.161 INFO:teuthology.orchestra.run.smithi039.stdout:5509 read 0x4264186 thru 0x4265aad (0x1928 bytes) 2024-09-14T06:17:32.161 INFO:teuthology.orchestra.run.smithi039.stdout:5511 read 0x2aa1b32 thru 0x2aa9b6d (0x803c bytes) 2024-09-14T06:17:32.166 INFO:teuthology.orchestra.run.smithi039.stdout:5512 write 0xdd731e4 thru 0xdd8290a (0xf727 bytes) 2024-09-14T06:17:32.167 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:32.167 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:32.171 INFO:teuthology.orchestra.run.smithi039.stdout:5513 write 0x39d588 thru 0x3aaa41 (0xd4ba bytes) 2024-09-14T06:17:32.173 INFO:teuthology.orchestra.run.smithi039.stdout:5515 read 0x2f8072d thru 0x2f82095 (0x1969 bytes) 2024-09-14T06:17:32.181 INFO:teuthology.orchestra.run.smithi039.stdout:5517 trunc from 0xdd8290b to 0x850a72f 2024-09-14T06:17:32.186 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:32.187 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:32.193 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:32.207 INFO:teuthology.orchestra.run.smithi039.stdout:5519 write 0xd9091d2 thru 0xd913619 (0xa448 bytes) 2024-09-14T06:17:32.208 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:32.208 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:32.211 INFO:teuthology.orchestra.run.smithi039.stdout:5520 read 0x5d7f096 thru 0x5d802a5 (0x1210 bytes) 2024-09-14T06:17:32.211 INFO:teuthology.orchestra.run.smithi039.stdout:5521 punch from 0x9818bdf to 0x982379d, (0xabbe bytes) 2024-09-14T06:17:32.212 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:32.212 INFO:teuthology.orchestra.run.smithi039.stdout:5523 write 0x68d2dac thru 0x68dafba (0x820f bytes) 2024-09-14T06:17:32.215 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:32.215 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:32.221 INFO:teuthology.orchestra.run.smithi039.stdout:5527 trunc from 0xd91361a to 0x9e5cab3 2024-09-14T06:17:32.232 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:32.246 INFO:teuthology.orchestra.run.smithi039.stdout:5531 write 0xeb1aad4 thru 0xeb214a1 (0x69ce bytes) 2024-09-14T06:17:32.251 INFO:teuthology.orchestra.run.smithi039.stdout:5532 punch from 0xdfc4762 to 0xdfc78fe, (0x319c bytes) 2024-09-14T06:17:32.251 INFO:teuthology.orchestra.run.smithi039.stdout:5533 clone 30 order 20 su 262144 sc 5 2024-09-14T06:17:32.253 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:32.249+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:32.259 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:32.259 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:32.259 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:32.259 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:32.260 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:32.273 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:17:32.363 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:32.361+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:32.440 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:32.440 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:32.440 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:32.440 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:32.440 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.440 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:30547 3.1f 3:f84dfb53:::rbd_data.11529eb229bd.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write 16226533~50642] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T06:17:32.440 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:32.440 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 33.247862452, 2024-09-14T06:17:32.440 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0079770020000001, 2024-09-14T06:17:32.440 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.440 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.441 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:32.441 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.441 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.441 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.441 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 30547 2024-09-14T06:17:32.441 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.442 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.442 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.442 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.442 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:32.442 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.442 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.442 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.442 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.442 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:17:32.442 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.442 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.443 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.443 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.443 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189811+0000", 2024-09-14T06:17:32.443 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2060000000000001e-06 2024-09-14T06:17:32.443 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.443 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.443 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.443 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189871+0000", 2024-09-14T06:17:32.443 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9862999999999999e-05 2024-09-14T06:17:32.443 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.444 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.444 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.444 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189899+0000", 2024-09-14T06:17:32.444 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7849000000000001e-05 2024-09-14T06:17:32.444 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.444 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.444 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.444 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189915+0000", 2024-09-14T06:17:32.445 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:17:32.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.445 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.445 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190024+0000", 2024-09-14T06:17:32.445 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000109202 2024-09-14T06:17:32.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.445 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.445 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:17:32.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190041+0000", 2024-09-14T06:17:32.446 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7665000000000001e-05 2024-09-14T06:17:32.446 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.446 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.446 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.446 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188512+0000", 2024-09-14T06:17:32.446 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.99847100700000002 2024-09-14T06:17:32.446 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.446 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.446 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for missing object", 2024-09-14T06:17:32.446 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188646+0000", 2024-09-14T06:17:32.446 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000133986 2024-09-14T06:17:32.446 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.446 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.446 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.447 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193097+0000", 2024-09-14T06:17:32.447 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044507059999999996 2024-09-14T06:17:32.447 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.447 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.447 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.447 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193183+0000", 2024-09-14T06:17:32.447 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5496999999999998e-05 2024-09-14T06:17:32.447 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.447 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.447 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:17:32.447 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193307+0000", 2024-09-14T06:17:32.447 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000124611 2024-09-14T06:17:32.447 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.448 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.448 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:32.448 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.194318+0000", 2024-09-14T06:17:32.448 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0010106 2024-09-14T06:17:32.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.448 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.448 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:32.448 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197670+0000", 2024-09-14T06:17:32.448 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033523170000000001 2024-09-14T06:17:32.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.449 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:32.449 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197712+0000", 2024-09-14T06:17:32.449 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1681000000000003e-05 2024-09-14T06:17:32.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.449 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.449 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197786+0000", 2024-09-14T06:17:32.449 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4148999999999998e-05 2024-09-14T06:17:32.449 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.450 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.450 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.450 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.450 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.450 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32022 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:32.450 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:32.450 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 28.569482240999999, 2024-09-14T06:17:32.450 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095598104000000003, 2024-09-14T06:17:32.450 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.450 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.450 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.450 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.450 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.450 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.450 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32022 2024-09-14T06:17:32.451 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.451 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.451 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.451 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.451 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:32.451 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.451 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.451 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.451 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.451 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:17:32.451 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.451 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.451 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.451 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.451 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868190+0000", 2024-09-14T06:17:32.451 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6000000000000003e-07 2024-09-14T06:17:32.452 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.452 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.452 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.452 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868243+0000", 2024-09-14T06:17:32.452 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3443999999999999e-05 2024-09-14T06:17:32.452 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.452 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.452 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.452 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868260+0000", 2024-09-14T06:17:32.452 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6799e-05 2024-09-14T06:17:32.452 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.452 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.452 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.452 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868271+0000", 2024-09-14T06:17:32.452 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0523000000000001e-05 2024-09-14T06:17:32.453 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.453 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.453 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.453 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963502+0000", 2024-09-14T06:17:32.453 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095231466000000001 2024-09-14T06:17:32.453 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.453 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.453 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.453 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963636+0000", 2024-09-14T06:17:32.453 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013388699999999999 2024-09-14T06:17:32.453 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.453 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.453 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.453 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963787+0000", 2024-09-14T06:17:32.454 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000151225 2024-09-14T06:17:32.454 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.454 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.454 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.454 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.454 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.454 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32023 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5816320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:32.454 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:32.454 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 28.569387115000001, 2024-09-14T06:17:32.454 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095879753999999998, 2024-09-14T06:17:32.454 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.454 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.454 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.455 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.455 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.455 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.455 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32023 2024-09-14T06:17:32.455 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.455 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.455 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.455 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.455 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:32.455 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.455 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.455 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.456 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.456 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:17:32.456 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.456 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.456 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.456 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.456 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868285+0000", 2024-09-14T06:17:32.456 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6299999999999993e-07 2024-09-14T06:17:32.456 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.456 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.456 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.456 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868288+0000", 2024-09-14T06:17:32.456 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.18e-06 2024-09-14T06:17:32.456 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.456 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.457 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.457 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868298+0000", 2024-09-14T06:17:32.457 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0029e-05 2024-09-14T06:17:32.457 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.457 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.457 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.457 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868307+0000", 2024-09-14T06:17:32.457 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7229999999999993e-06 2024-09-14T06:17:32.457 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.457 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.457 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963871+0000", 2024-09-14T06:17:32.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095563876000000006 2024-09-14T06:17:32.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.458 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.458 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964015+0000", 2024-09-14T06:17:32.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143573 2024-09-14T06:17:32.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.458 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.458 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.458 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964164+0000", 2024-09-14T06:17:32.458 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014940999999999999 2024-09-14T06:17:32.458 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.458 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.459 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.459 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.459 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.459 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32024 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:32.459 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:32.459 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 28.569353970999998, 2024-09-14T06:17:32.459 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096230231999999999, 2024-09-14T06:17:32.459 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.459 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.459 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.459 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.459 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.459 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.460 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32024 2024-09-14T06:17:32.460 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.460 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.460 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.460 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.460 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:32.460 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.460 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.460 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.460 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.460 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:17:32.460 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.460 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868318+0000", 2024-09-14T06:17:32.461 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1699999999999997e-07 2024-09-14T06:17:32.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868321+0000", 2024-09-14T06:17:32.461 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6769999999999999e-06 2024-09-14T06:17:32.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868334+0000", 2024-09-14T06:17:32.461 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2928999999999999e-05 2024-09-14T06:17:32.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.462 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.462 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868341+0000", 2024-09-14T06:17:32.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9800000000000001e-06 2024-09-14T06:17:32.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.462 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.462 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964258+0000", 2024-09-14T06:17:32.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095917099000000006 2024-09-14T06:17:32.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.462 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.462 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964392+0000", 2024-09-14T06:17:32.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013396999999999999 2024-09-14T06:17:32.463 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.463 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.463 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.463 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964548+0000", 2024-09-14T06:17:32.463 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015585999999999999 2024-09-14T06:17:32.463 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.463 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.463 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.463 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.463 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.463 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32025 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:32.463 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:32.463 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 28.569321315, 2024-09-14T06:17:32.463 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096576591000000003, 2024-09-14T06:17:32.464 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.464 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.464 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.464 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.464 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.464 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.464 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32025 2024-09-14T06:17:32.464 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.464 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.464 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.464 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.464 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:32.464 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.464 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.464 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.465 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.465 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:17:32.465 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.465 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.465 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.465 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.465 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868351+0000", 2024-09-14T06:17:32.465 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1600000000000001e-07 2024-09-14T06:17:32.465 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.465 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.465 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.465 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868353+0000", 2024-09-14T06:17:32.465 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.627e-06 2024-09-14T06:17:32.466 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.466 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.466 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.466 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868364+0000", 2024-09-14T06:17:32.466 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0893000000000001e-05 2024-09-14T06:17:32.466 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.466 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.466 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.466 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868381+0000", 2024-09-14T06:17:32.466 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6769000000000002e-05 2024-09-14T06:17:32.466 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.466 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.466 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.466 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964632+0000", 2024-09-14T06:17:32.467 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096250589999999997 2024-09-14T06:17:32.467 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.467 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.467 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.467 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964772+0000", 2024-09-14T06:17:32.467 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014045299999999999 2024-09-14T06:17:32.467 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.467 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.467 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.467 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964927+0000", 2024-09-14T06:17:32.467 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015454299999999999 2024-09-14T06:17:32.467 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.467 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.467 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.467 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.467 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.468 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32026 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:32.468 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:32.468 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 28.569280364000001, 2024-09-14T06:17:32.468 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096901622000000007, 2024-09-14T06:17:32.468 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.468 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.468 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.468 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.468 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.468 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.468 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32026 2024-09-14T06:17:32.468 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.468 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.468 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.468 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.469 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:32.469 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.469 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.469 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.469 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.469 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:17:32.469 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.469 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.469 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.469 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.469 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868392+0000", 2024-09-14T06:17:32.469 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0699999999999996e-07 2024-09-14T06:17:32.469 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.469 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.470 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.470 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868395+0000", 2024-09-14T06:17:32.470 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9859999999999999e-06 2024-09-14T06:17:32.470 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.470 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.470 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.470 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868405+0000", 2024-09-14T06:17:32.470 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9229999999999997e-06 2024-09-14T06:17:32.470 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.470 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.470 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.470 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868412+0000", 2024-09-14T06:17:32.470 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9670000000000004e-06 2024-09-14T06:17:32.471 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.471 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.471 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.471 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965010+0000", 2024-09-14T06:17:32.471 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096598562999999998 2024-09-14T06:17:32.471 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.471 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.471 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.471 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965143+0000", 2024-09-14T06:17:32.471 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013247700000000001 2024-09-14T06:17:32.471 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.471 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.471 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.471 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965293+0000", 2024-09-14T06:17:32.472 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000149899 2024-09-14T06:17:32.472 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.472 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.472 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.472 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.472 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32027 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:32.472 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:32.473 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 28.569212026999999, 2024-09-14T06:17:32.473 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097211306999999997, 2024-09-14T06:17:32.473 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.473 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.473 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.473 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.473 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.474 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.474 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32027 2024-09-14T06:17:32.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.474 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.474 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.474 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.474 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:32.474 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.474 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.474 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.474 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:17:32.474 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.475 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.475 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.475 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868461+0000", 2024-09-14T06:17:32.475 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.296e-06 2024-09-14T06:17:32.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.475 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.475 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.476 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868465+0000", 2024-09-14T06:17:32.476 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9570000000000004e-06 2024-09-14T06:17:32.476 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.476 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.476 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.477 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868476+0000", 2024-09-14T06:17:32.477 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1365999999999999e-05 2024-09-14T06:17:32.477 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.477 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.477 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.478 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868483+0000", 2024-09-14T06:17:32.478 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2529999999999998e-06 2024-09-14T06:17:32.478 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.478 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.478 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.478 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965375+0000", 2024-09-14T06:17:32.478 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096891973000000006 2024-09-14T06:17:32.478 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.478 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.478 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.479 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965518+0000", 2024-09-14T06:17:32.479 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143017 2024-09-14T06:17:32.479 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.479 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.479 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.479 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965671+0000", 2024-09-14T06:17:32.480 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015244499999999999 2024-09-14T06:17:32.480 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.480 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.480 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.480 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.480 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.480 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32028 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:32.480 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:32.480 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 28.569177198999999, 2024-09-14T06:17:32.480 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097841480999999994, 2024-09-14T06:17:32.480 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.480 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.480 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.480 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.481 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.481 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.481 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32028 2024-09-14T06:17:32.481 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.481 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.481 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.481 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.481 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:32.481 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.481 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.481 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.482 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.482 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:17:32.482 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.482 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.482 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.482 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868495+0000", 2024-09-14T06:17:32.482 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6400000000000001e-07 2024-09-14T06:17:32.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.482 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.482 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.482 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868498+0000", 2024-09-14T06:17:32.482 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7760000000000002e-06 2024-09-14T06:17:32.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.483 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.483 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.483 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868507+0000", 2024-09-14T06:17:32.483 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1530000000000003e-06 2024-09-14T06:17:32.483 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.483 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.483 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.483 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868516+0000", 2024-09-14T06:17:32.483 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8929999999999994e-06 2024-09-14T06:17:32.483 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.483 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.483 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.483 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965753+0000", 2024-09-14T06:17:32.483 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097236970000000006 2024-09-14T06:17:32.484 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.484 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.484 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.484 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965890+0000", 2024-09-14T06:17:32.484 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013677599999999999 2024-09-14T06:17:32.484 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.484 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.484 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.484 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966336+0000", 2024-09-14T06:17:32.484 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00044614900000000001 2024-09-14T06:17:32.484 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.484 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.484 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.484 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.485 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.485 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32029 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:32.485 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:32.485 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 28.569146124, 2024-09-14T06:17:32.485 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098577779000000004, 2024-09-14T06:17:32.485 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.485 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.485 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.485 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.485 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.485 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.485 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32029 2024-09-14T06:17:32.485 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.486 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:32.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:17:32.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868526+0000", 2024-09-14T06:17:32.487 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9300000000000006e-07 2024-09-14T06:17:32.487 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.487 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.487 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.487 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868529+0000", 2024-09-14T06:17:32.487 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5969999999999999e-06 2024-09-14T06:17:32.487 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.487 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.487 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.487 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868538+0000", 2024-09-14T06:17:32.487 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1700000000000003e-06 2024-09-14T06:17:32.487 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.487 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.487 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.487 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868560+0000", 2024-09-14T06:17:32.488 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1596000000000001e-05 2024-09-14T06:17:32.488 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.488 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.488 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.488 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966423+0000", 2024-09-14T06:17:32.488 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097863396000000005 2024-09-14T06:17:32.488 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.488 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.488 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.488 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966578+0000", 2024-09-14T06:17:32.488 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000154792 2024-09-14T06:17:32.488 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.488 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.488 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.489 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967103+0000", 2024-09-14T06:17:32.489 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00052523499999999998 2024-09-14T06:17:32.489 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.489 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.489 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.489 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.489 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32030 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:32.489 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:32.489 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 28.569101205999999, 2024-09-14T06:17:32.489 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099324030999999993, 2024-09-14T06:17:32.489 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.489 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.490 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.490 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.490 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.490 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.490 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32030 2024-09-14T06:17:32.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.490 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.490 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.490 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:32.490 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.490 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.491 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:17:32.491 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.491 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.491 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.491 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868571+0000", 2024-09-14T06:17:32.491 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9699999999999995e-07 2024-09-14T06:17:32.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.491 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.491 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.491 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868574+0000", 2024-09-14T06:17:32.491 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9529999999999999e-06 2024-09-14T06:17:32.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.491 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.491 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.491 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868584+0000", 2024-09-14T06:17:32.492 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7860000000000008e-06 2024-09-14T06:17:32.492 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.492 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.492 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.492 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868591+0000", 2024-09-14T06:17:32.492 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0500000000000003e-06 2024-09-14T06:17:32.492 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.492 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.492 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.493 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967191+0000", 2024-09-14T06:17:32.493 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098600035000000003 2024-09-14T06:17:32.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.493 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.493 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.493 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967333+0000", 2024-09-14T06:17:32.493 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014190599999999999 2024-09-14T06:17:32.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.493 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.493 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.494 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967894+0000", 2024-09-14T06:17:32.494 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000561504 2024-09-14T06:17:32.494 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.494 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.494 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.494 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.494 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.494 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32031 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:32.494 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:32.494 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 28.569071644000001, 2024-09-14T06:17:32.494 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.10010279, 2024-09-14T06:17:32.494 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.494 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.494 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.495 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.495 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.495 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.495 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32031 2024-09-14T06:17:32.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.495 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:32.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:32.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:17:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:17:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868603+0000", 2024-09-14T06:17:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.599e-06 2024-09-14T06:17:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868612+0000", 2024-09-14T06:17:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.986e-06 2024-09-14T06:17:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868619+0000", 2024-09-14T06:17:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8900000000000001e-06 2024-09-14T06:17:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967983+0000", 2024-09-14T06:17:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099363612000000004 2024-09-14T06:17:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968140+0000", 2024-09-14T06:17:32.498 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015788600000000001 2024-09-14T06:17:32.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.498 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968703+0000", 2024-09-14T06:17:32.498 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00056208300000000001 2024-09-14T06:17:32.498 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.498 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.498 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.498 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32032 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6258688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:32.498 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:32.498 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 28.569043169, 2024-09-14T06:17:32.499 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100488088, 2024-09-14T06:17:32.499 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.499 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.499 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.499 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.499 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.499 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.499 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32032 2024-09-14T06:17:32.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.499 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.500 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.500 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:32.500 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.500 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.500 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:17:32.500 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.501 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.501 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868629+0000", 2024-09-14T06:17:32.501 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:17:32.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.501 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.501 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868631+0000", 2024-09-14T06:17:32.501 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4099999999999998e-06 2024-09-14T06:17:32.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.501 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.501 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868640+0000", 2024-09-14T06:17:32.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0939999999999997e-06 2024-09-14T06:17:32.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868647+0000", 2024-09-14T06:17:32.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9460000000000001e-06 2024-09-14T06:17:32.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968797+0000", 2024-09-14T06:17:32.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100149867 2024-09-14T06:17:32.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.503 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968937+0000", 2024-09-14T06:17:32.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013981399999999999 2024-09-14T06:17:32.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.503 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.503 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969116+0000", 2024-09-14T06:17:32.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017927399999999999 2024-09-14T06:17:32.503 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.503 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.503 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.504 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32044 3.1e 3:7e6619d4:::rbd_data.115274ed0aa8.0000000000000068:head [set-alloc-hint object_size 1048576 write_size 1048576,write 835584~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:17:32.504 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:32.504 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 28.561354086000001, 2024-09-14T06:17:32.504 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096039367, 2024-09-14T06:17:32.504 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.504 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.504 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:32.504 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.504 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.504 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.504 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32044 2024-09-14T06:17:32.504 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.504 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.505 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.505 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.505 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:32.505 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.505 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.505 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.505 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.505 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:17:32.505 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.505 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.505 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.505 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.506 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876320+0000", 2024-09-14T06:17:32.506 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8270000000000002e-06 2024-09-14T06:17:32.506 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.506 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.506 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.506 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876397+0000", 2024-09-14T06:17:32.506 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6985999999999998e-05 2024-09-14T06:17:32.506 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.506 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.506 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.506 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876431+0000", 2024-09-14T06:17:32.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3719e-05 2024-09-14T06:17:32.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876452+0000", 2024-09-14T06:17:32.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0891999999999999e-05 2024-09-14T06:17:32.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969186+0000", 2024-09-14T06:17:32.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.092734681999999999 2024-09-14T06:17:32.509 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.509 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.509 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969550+0000", 2024-09-14T06:17:32.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036402199999999998 2024-09-14T06:17:32.509 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.509 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:17:32.509 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969987+0000", 2024-09-14T06:17:32.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000436759 2024-09-14T06:17:32.509 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:32.510 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971569+0000", 2024-09-14T06:17:32.510 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001581821 2024-09-14T06:17:32.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:32.510 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971947+0000", 2024-09-14T06:17:32.510 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378446 2024-09-14T06:17:32.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:32.510 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972146+0000", 2024-09-14T06:17:32.510 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019819 2024-09-14T06:17:32.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.511 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.511 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972357+0000", 2024-09-14T06:17:32.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021102299999999999 2024-09-14T06:17:32.511 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.511 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.511 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.511 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40252 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 827392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:32.512 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:32.512 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 22.72071798, 2024-09-14T06:17:32.512 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095633474999999996, 2024-09-14T06:17:32.512 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.512 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.512 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.512 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.512 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.512 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.512 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40252 2024-09-14T06:17:32.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.512 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.512 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.513 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:32.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.513 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.513 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:17:32.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.513 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.513 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717376+0000", 2024-09-14T06:17:32.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000422216 2024-09-14T06:17:32.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.514 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.514 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717386+0000", 2024-09-14T06:17:32.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0396e-05 2024-09-14T06:17:32.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.514 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.514 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717395+0000", 2024-09-14T06:17:32.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7819999999999999e-06 2024-09-14T06:17:32.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.514 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.514 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717400+0000", 2024-09-14T06:17:32.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1599999999999997e-06 2024-09-14T06:17:32.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.811922+0000", 2024-09-14T06:17:32.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.094522018999999999 2024-09-14T06:17:32.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812007+0000", 2024-09-14T06:17:32.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5328999999999996e-05 2024-09-14T06:17:32.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.516 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812587+0000", 2024-09-14T06:17:32.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000579573 2024-09-14T06:17:32.516 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.516 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.516 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.516 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40254 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:32.516 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:32.516 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 22.720259807000001, 2024-09-14T06:17:32.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095562731999999997, 2024-09-14T06:17:32.516 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.516 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.516 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.516 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.517 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.517 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.517 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40254 2024-09-14T06:17:32.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.517 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.517 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.517 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.517 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:32.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.517 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.517 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.517 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:32.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.518 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.518 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:17:32.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1699999999999999e-07 2024-09-14T06:17:32.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.518 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.518 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717414+0000", 2024-09-14T06:17:32.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:17:32.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.518 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717419+0000", 2024-09-14T06:17:32.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2839999999999997e-06 2024-09-14T06:17:32.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717424+0000", 2024-09-14T06:17:32.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1549999999999998e-06 2024-09-14T06:17:32.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812619+0000", 2024-09-14T06:17:32.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095194940000000006 2024-09-14T06:17:32.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.520 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812717+0000", 2024-09-14T06:17:32.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7647999999999997e-05 2024-09-14T06:17:32.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.520 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.520 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.520 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812974+0000", 2024-09-14T06:17:32.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025752099999999998 2024-09-14T06:17:32.520 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.520 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.520 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.520 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.520 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40256 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 843776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:32.520 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:32.520 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 22.720237203, 2024-09-14T06:17:32.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095932066999999996, 2024-09-14T06:17:32.521 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.521 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.521 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.521 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.521 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.521 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.521 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40256 2024-09-14T06:17:32.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.521 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:32.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:17:32.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717628+0000", 2024-09-14T06:17:32.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019339499999999999 2024-09-14T06:17:32.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717632+0000", 2024-09-14T06:17:32.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2230000000000001e-06 2024-09-14T06:17:32.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.523 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.523 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717641+0000", 2024-09-14T06:17:32.523 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6309999999999998e-06 2024-09-14T06:17:32.523 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.523 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.523 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717656+0000", 2024-09-14T06:17:32.523 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4724e-05 2024-09-14T06:17:32.523 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.523 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.523 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813006+0000", 2024-09-14T06:17:32.523 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095349801999999997 2024-09-14T06:17:32.523 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.524 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.524 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813107+0000", 2024-09-14T06:17:32.524 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000100973 2024-09-14T06:17:32.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.524 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.524 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813366+0000", 2024-09-14T06:17:32.524 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025931899999999999 2024-09-14T06:17:32.524 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.524 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.524 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.525 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40258 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 851968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:32.525 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:32.525 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 22.720001103000001, 2024-09-14T06:17:32.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096062543, 2024-09-14T06:17:32.525 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.525 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.525 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.525 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.525 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.525 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.525 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40258 2024-09-14T06:17:32.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.525 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:32.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:17:32.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717671+0000", 2024-09-14T06:17:32.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8299999999999997e-07 2024-09-14T06:17:32.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717674+0000", 2024-09-14T06:17:32.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7080000000000002e-06 2024-09-14T06:17:32.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.527 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717681+0000", 2024-09-14T06:17:32.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7889999999999995e-06 2024-09-14T06:17:32.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.527 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717687+0000", 2024-09-14T06:17:32.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7529999999999998e-06 2024-09-14T06:17:32.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.527 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813408+0000", 2024-09-14T06:17:32.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095720451999999998 2024-09-14T06:17:32.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813499+0000", 2024-09-14T06:17:32.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0933000000000005e-05 2024-09-14T06:17:32.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813733+0000", 2024-09-14T06:17:32.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00023432499999999999 2024-09-14T06:17:32.529 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.529 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.529 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.529 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40260 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 860160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:32.529 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:32.529 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 22.719975254000001, 2024-09-14T06:17:32.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096337381999999999, 2024-09-14T06:17:32.529 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.529 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.529 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.529 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.530 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.530 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.530 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40260 2024-09-14T06:17:32.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.530 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.530 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.530 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.530 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:32.530 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.530 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.530 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.530 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:17:32.530 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718008+0000", 2024-09-14T06:17:32.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00031192699999999998 2024-09-14T06:17:32.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718013+0000", 2024-09-14T06:17:32.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1830000000000002e-06 2024-09-14T06:17:32.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718036+0000", 2024-09-14T06:17:32.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2721000000000002e-05 2024-09-14T06:17:32.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718053+0000", 2024-09-14T06:17:32.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6736e-05 2024-09-14T06:17:32.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813767+0000", 2024-09-14T06:17:32.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095714658999999994 2024-09-14T06:17:32.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.533 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.533 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813816+0000", 2024-09-14T06:17:32.533 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8848999999999999e-05 2024-09-14T06:17:32.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.533 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.533 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.533 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814034+0000", 2024-09-14T06:17:32.533 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000217307 2024-09-14T06:17:32.533 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.533 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.533 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.534 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40262 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 868352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:32.534 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:32.534 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 22.719575571, 2024-09-14T06:17:32.534 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096354165000000006, 2024-09-14T06:17:32.534 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.534 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.534 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.534 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.534 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.534 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.534 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40262 2024-09-14T06:17:32.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.535 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.535 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:32.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.535 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:17:32.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718281+0000", 2024-09-14T06:17:32.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000185069 2024-09-14T06:17:32.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718286+0000", 2024-09-14T06:17:32.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.118e-06 2024-09-14T06:17:32.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718295+0000", 2024-09-14T06:17:32.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0429999999999996e-06 2024-09-14T06:17:32.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718302+0000", 2024-09-14T06:17:32.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2790000000000001e-06 2024-09-14T06:17:32.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814076+0000", 2024-09-14T06:17:32.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095774076999999999 2024-09-14T06:17:32.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814191+0000", 2024-09-14T06:17:32.538 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00011500700000000001 2024-09-14T06:17:32.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.538 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.538 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814450+0000", 2024-09-14T06:17:32.538 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025857199999999999 2024-09-14T06:17:32.538 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.538 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.538 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.538 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40264 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 876544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:32.538 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:32.539 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 22.719218503, 2024-09-14T06:17:32.539 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096372478999999997, 2024-09-14T06:17:32.539 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:32.539 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:32.539 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:32.539 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:32.539 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:32.539 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:32.539 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40264 2024-09-14T06:17:32.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.539 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:32.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:32.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:32.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:32.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:17:32.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:32.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:32.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718793+0000", 2024-09-14T06:17:32.541 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00033991099999999998 2024-09-14T06:17:32.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.541 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.541 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:32.541 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718805+0000", 2024-09-14T06:17:32.541 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2514000000000001e-05 2024-09-14T06:17:32.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.541 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.541 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:32.541 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718815+0000", 2024-09-14T06:17:32.541 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9590000000000007e-06 2024-09-14T06:17:32.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.541 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.541 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:32.542 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718822+0000", 2024-09-14T06:17:32.542 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2030000000000003e-06 2024-09-14T06:17:32.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.542 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:32.542 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814490+0000", 2024-09-14T06:17:32.542 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095667346 2024-09-14T06:17:32.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:32.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814577+0000", 2024-09-14T06:17:32.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6920000000000001e-05 2024-09-14T06:17:32.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:32.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:32.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:32.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814825+0000", 2024-09-14T06:17:32.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000248626 2024-09-14T06:17:32.544 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.544 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.544 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.544 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:32.544 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:32.544 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:32.546 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:17:32.546 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:32.541+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:32.560 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-09-14T06:17:32.560 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T06:17:32.592 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T06:17:32.647 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:32.645+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:32.718 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:32.718 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:32.718 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:32.718 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:32.732 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:17:32.748 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:32.746+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:32.822 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-14T06:17:32.850 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:32.846+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:32.903 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:32.903 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:32.904 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:32.904 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:32.904 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:32.917 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:17:32.951 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:32.946+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:33.041 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:33.041 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:33.041 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:33.041 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:33.041 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.041 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:3595 3.1 3:811c1c24:::rbd_data.1152bd92e04f.000000000000006e:head [set-alloc-hint object_size 2097152 write_size 2097152,write 440183~2185] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T06:17:33.041 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:33.041 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.817363220000004, 2024-09-14T06:17:33.041 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0365806689999999, 2024-09-14T06:17:33.042 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.042 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.042 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.042 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.042 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.042 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.042 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 3595 2024-09-14T06:17:33.042 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.042 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.042 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.042 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.042 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:33.042 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.042 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.043 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.043 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.043 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:33.043 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.043 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.043 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.043 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.043 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222823+0000", 2024-09-14T06:17:33.043 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8219999999999998e-06 2024-09-14T06:17:33.043 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.043 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.043 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.043 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222835+0000", 2024-09-14T06:17:33.044 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2052000000000001e-05 2024-09-14T06:17:33.044 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.044 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.044 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.044 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222859+0000", 2024-09-14T06:17:33.044 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3694e-05 2024-09-14T06:17:33.044 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.044 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.044 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.044 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222872+0000", 2024-09-14T06:17:33.044 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3203999999999999e-05 2024-09-14T06:17:33.044 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.044 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.044 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.045 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223068+0000", 2024-09-14T06:17:33.045 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000195995 2024-09-14T06:17:33.045 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.045 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.045 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.045 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223312+0000", 2024-09-14T06:17:33.045 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024424199999999998 2024-09-14T06:17:33.045 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.045 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.045 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:33.045 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223786+0000", 2024-09-14T06:17:33.045 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00047356600000000003 2024-09-14T06:17:33.045 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.045 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.046 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:33.046 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.225374+0000", 2024-09-14T06:17:33.046 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015875150000000001 2024-09-14T06:17:33.046 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.046 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.046 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.046 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.233122+0000", 2024-09-14T06:17:33.046 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0077480700000000001 2024-09-14T06:17:33.046 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.046 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.046 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:17:33.046 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.233157+0000", 2024-09-14T06:17:33.047 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5113000000000001e-05 2024-09-14T06:17:33.047 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.047 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.047 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.047 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.252998+0000", 2024-09-14T06:17:33.047 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.019841142 2024-09-14T06:17:33.047 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.047 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.047 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for degraded object", 2024-09-14T06:17:33.047 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.254121+0000", 2024-09-14T06:17:33.047 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0011227979999999999 2024-09-14T06:17:33.047 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.047 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.047 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.047 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.259263+0000", 2024-09-14T06:17:33.048 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0051426700000000002 2024-09-14T06:17:33.048 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.048 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.048 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.048 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.259401+0000", 2024-09-14T06:17:33.048 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013778600000000001 2024-09-14T06:17:33.048 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.048 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.048 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.048 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.048 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.048 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11421 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.048 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:33.048 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.054468034000003, 2024-09-14T06:17:33.049 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078553170000000005, 2024-09-14T06:17:33.049 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.049 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.049 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.049 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.049 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.049 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.049 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11421 2024-09-14T06:17:33.049 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.049 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.049 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.049 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.049 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:33.050 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.050 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.050 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.050 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.050 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:33.050 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.050 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.050 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.050 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.050 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985718+0000", 2024-09-14T06:17:33.050 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8840000000000001e-06 2024-09-14T06:17:33.050 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.050 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.050 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.050 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985723+0000", 2024-09-14T06:17:33.051 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8799999999999996e-06 2024-09-14T06:17:33.051 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.051 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.051 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.051 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985742+0000", 2024-09-14T06:17:33.051 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8776000000000001e-05 2024-09-14T06:17:33.051 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.051 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.051 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.051 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985759+0000", 2024-09-14T06:17:33.051 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7119999999999999e-05 2024-09-14T06:17:33.051 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.051 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.051 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.051 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064259+0000", 2024-09-14T06:17:33.052 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078499418000000001 2024-09-14T06:17:33.052 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.052 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.052 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.052 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064269+0000", 2024-09-14T06:17:33.052 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:17:33.052 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.052 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.052 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.052 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.052 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.052 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11422 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.052 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:33.053 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.054398067000001, 2024-09-14T06:17:33.053 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078521540000000001, 2024-09-14T06:17:33.053 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.053 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.053 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.053 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.053 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.053 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.053 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11422 2024-09-14T06:17:33.053 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.053 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.053 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.053 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.053 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:33.053 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.054 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.054 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.054 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.054 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:33.054 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.054 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.054 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.054 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.054 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985787+0000", 2024-09-14T06:17:33.054 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7630000000000001e-06 2024-09-14T06:17:33.054 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.054 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.055 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.055 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985842+0000", 2024-09-14T06:17:33.055 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4555999999999997e-05 2024-09-14T06:17:33.055 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.055 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.055 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.055 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985864+0000", 2024-09-14T06:17:33.055 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2189999999999999e-05 2024-09-14T06:17:33.055 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.055 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.055 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.055 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985879+0000", 2024-09-14T06:17:33.055 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4382999999999999e-05 2024-09-14T06:17:33.055 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.055 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.056 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.056 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064298+0000", 2024-09-14T06:17:33.056 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078419121999999994 2024-09-14T06:17:33.056 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.056 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.056 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.056 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064307+0000", 2024-09-14T06:17:33.056 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5259999999999999e-06 2024-09-14T06:17:33.056 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.056 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.056 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.056 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.056 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.056 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11423 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.057 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:33.057 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.054280695000003, 2024-09-14T06:17:33.057 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078437175999999997, 2024-09-14T06:17:33.057 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.057 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.057 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.057 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.057 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.057 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.057 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11423 2024-09-14T06:17:33.057 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.057 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.057 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.057 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.058 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:33.058 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.058 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.058 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.058 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.058 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:33.058 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.058 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.058 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.058 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.058 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985905+0000", 2024-09-14T06:17:33.058 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8139999999999999e-06 2024-09-14T06:17:33.058 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.058 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.059 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.059 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985911+0000", 2024-09-14T06:17:33.059 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7230000000000001e-06 2024-09-14T06:17:33.059 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.059 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.059 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.059 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985930+0000", 2024-09-14T06:17:33.059 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9360000000000001e-05 2024-09-14T06:17:33.059 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.059 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.059 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.059 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985944+0000", 2024-09-14T06:17:33.059 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4479e-05 2024-09-14T06:17:33.059 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.060 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.060 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.060 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064332+0000", 2024-09-14T06:17:33.060 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078387686999999998 2024-09-14T06:17:33.060 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.060 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.060 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.060 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064340+0000", 2024-09-14T06:17:33.060 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1130000000000001e-06 2024-09-14T06:17:33.060 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.060 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.060 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.060 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.061 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.061 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11480 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.061 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:33.061 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.052930218999997, 2024-09-14T06:17:33.061 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077122298000000006, 2024-09-14T06:17:33.061 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.061 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.061 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.061 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.061 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.061 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.061 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11480 2024-09-14T06:17:33.061 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.062 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.062 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.062 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.062 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:33.062 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.062 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.062 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.062 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.062 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:33.062 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.062 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.062 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.062 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.063 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987255+0000", 2024-09-14T06:17:33.063 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:17:33.063 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.063 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.063 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.063 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987263+0000", 2024-09-14T06:17:33.063 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6499999999999996e-06 2024-09-14T06:17:33.063 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.063 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.063 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.063 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987285+0000", 2024-09-14T06:17:33.063 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1860000000000001e-05 2024-09-14T06:17:33.063 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.063 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.064 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.064 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987341+0000", 2024-09-14T06:17:33.064 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6648999999999999e-05 2024-09-14T06:17:33.064 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.064 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.064 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.064 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064367+0000", 2024-09-14T06:17:33.064 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077025622000000002 2024-09-14T06:17:33.064 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.064 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.064 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.064 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064376+0000", 2024-09-14T06:17:33.064 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7499999999999992e-06 2024-09-14T06:17:33.064 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.065 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.065 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.065 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.065 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.065 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11481 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.065 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:33.065 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.052794517000002, 2024-09-14T06:17:33.065 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077022057000000005, 2024-09-14T06:17:33.065 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.065 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.065 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.065 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.065 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.065 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.065 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11481 2024-09-14T06:17:33.066 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.066 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.066 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.066 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.066 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:33.066 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.066 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.066 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.066 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.066 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:33.066 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.066 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.066 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.066 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.066 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987391+0000", 2024-09-14T06:17:33.067 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4839999999999999e-06 2024-09-14T06:17:33.067 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.067 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.067 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.067 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987399+0000", 2024-09-14T06:17:33.067 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8660000000000006e-06 2024-09-14T06:17:33.067 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.067 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.067 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.067 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987419+0000", 2024-09-14T06:17:33.067 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9984e-05 2024-09-14T06:17:33.067 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.067 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.067 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.068 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987434+0000", 2024-09-14T06:17:33.068 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5396e-05 2024-09-14T06:17:33.068 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.068 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.068 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.068 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064403+0000", 2024-09-14T06:17:33.068 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076969212999999995 2024-09-14T06:17:33.068 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.068 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.068 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.068 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064411+0000", 2024-09-14T06:17:33.068 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1140000000000006e-06 2024-09-14T06:17:33.068 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.068 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.068 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.069 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.069 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.069 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11482 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.069 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:33.069 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.052709657000001, 2024-09-14T06:17:33.069 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076971172000000004, 2024-09-14T06:17:33.069 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.069 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.069 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.069 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.069 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.069 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.069 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11482 2024-09-14T06:17:33.070 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.070 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.070 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.070 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.070 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:33.070 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.070 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.070 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.070 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.070 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:33.070 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.070 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.070 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.071 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.071 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987476+0000", 2024-09-14T06:17:33.071 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.793e-06 2024-09-14T06:17:33.071 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.071 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.071 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.071 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987482+0000", 2024-09-14T06:17:33.071 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8270000000000003e-06 2024-09-14T06:17:33.071 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.071 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.071 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.071 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987502+0000", 2024-09-14T06:17:33.071 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0656e-05 2024-09-14T06:17:33.071 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.072 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.072 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.072 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987518+0000", 2024-09-14T06:17:33.072 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:17:33.072 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.072 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.072 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.072 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064438+0000", 2024-09-14T06:17:33.072 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076919623000000006 2024-09-14T06:17:33.072 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.072 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.072 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.072 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064445+0000", 2024-09-14T06:17:33.073 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.61e-06 2024-09-14T06:17:33.073 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.073 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.073 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.073 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.073 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.073 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11483 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.073 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:33.073 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.052641807999997, 2024-09-14T06:17:33.073 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076936562999999999, 2024-09-14T06:17:33.073 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.073 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.073 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.073 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.074 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.074 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.074 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11483 2024-09-14T06:17:33.074 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.074 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.074 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.074 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.074 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:33.074 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.074 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.074 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.074 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.074 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:33.074 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.074 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.075 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.075 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.075 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987544+0000", 2024-09-14T06:17:33.075 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.654e-06 2024-09-14T06:17:33.075 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.075 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.075 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.075 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987550+0000", 2024-09-14T06:17:33.075 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0299999999999999e-06 2024-09-14T06:17:33.075 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.075 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.075 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.075 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987570+0000", 2024-09-14T06:17:33.075 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0080000000000001e-05 2024-09-14T06:17:33.076 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.076 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.076 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.076 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987619+0000", 2024-09-14T06:17:33.076 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9228999999999999e-05 2024-09-14T06:17:33.076 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.076 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.076 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.076 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064471+0000", 2024-09-14T06:17:33.076 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076852086999999999 2024-09-14T06:17:33.076 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.076 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.076 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.076 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064478+0000", 2024-09-14T06:17:33.077 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4830000000000002e-06 2024-09-14T06:17:33.077 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.077 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.077 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.077 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.077 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.077 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11485 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.077 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:33.077 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.052543315000001, 2024-09-14T06:17:33.077 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076868463999999997, 2024-09-14T06:17:33.077 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.077 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.077 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.077 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.078 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.078 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.078 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11485 2024-09-14T06:17:33.078 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.078 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.078 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.078 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.078 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:33.078 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.078 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.078 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.078 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.078 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:33.078 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.078 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.079 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.079 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.079 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987642+0000", 2024-09-14T06:17:33.079 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4100000000000001e-06 2024-09-14T06:17:33.079 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.079 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.079 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.079 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987648+0000", 2024-09-14T06:17:33.079 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2500000000000003e-06 2024-09-14T06:17:33.079 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.079 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.079 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.079 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987683+0000", 2024-09-14T06:17:33.079 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5193e-05 2024-09-14T06:17:33.079 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.080 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.080 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.080 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987698+0000", 2024-09-14T06:17:33.080 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4893e-05 2024-09-14T06:17:33.080 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.080 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.080 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.080 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064501+0000", 2024-09-14T06:17:33.080 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076803263999999996 2024-09-14T06:17:33.080 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.080 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.080 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.080 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064509+0000", 2024-09-14T06:17:33.080 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4540000000000001e-06 2024-09-14T06:17:33.081 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.081 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.081 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.081 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.081 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.081 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11490 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.081 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:33.081 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.052393746, 2024-09-14T06:17:33.081 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076755043999999994, 2024-09-14T06:17:33.081 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.081 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.081 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.081 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.081 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.081 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.082 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11490 2024-09-14T06:17:33.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.082 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.082 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.082 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.082 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:33.082 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.082 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.082 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.082 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:33.082 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.082 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.082 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.083 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987791+0000", 2024-09-14T06:17:33.083 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:17:33.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.083 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.083 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.083 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987797+0000", 2024-09-14T06:17:33.083 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7969999999999997e-06 2024-09-14T06:17:33.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.083 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.083 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.083 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987817+0000", 2024-09-14T06:17:33.083 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9746e-05 2024-09-14T06:17:33.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.083 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.083 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.084 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987832+0000", 2024-09-14T06:17:33.084 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.452e-05 2024-09-14T06:17:33.084 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.084 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.084 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.084 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064535+0000", 2024-09-14T06:17:33.084 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076703478000000005 2024-09-14T06:17:33.084 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.084 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.084 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.084 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064545+0000", 2024-09-14T06:17:33.084 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9799999999999993e-06 2024-09-14T06:17:33.084 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.084 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.084 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.085 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.085 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.085 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11492 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.085 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:33.085 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.052266881000001, 2024-09-14T06:17:33.085 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076661759999999995, 2024-09-14T06:17:33.085 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.085 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.085 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.085 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.085 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.085 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.085 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11492 2024-09-14T06:17:33.085 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.086 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.086 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.086 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.086 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:33.086 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.086 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.086 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.086 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.086 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:33.086 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.086 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.086 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.086 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.086 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987918+0000", 2024-09-14T06:17:33.086 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3239999999999999e-06 2024-09-14T06:17:33.087 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.087 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.087 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.087 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987924+0000", 2024-09-14T06:17:33.087 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6930000000000004e-06 2024-09-14T06:17:33.087 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.087 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.087 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.087 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987943+0000", 2024-09-14T06:17:33.087 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9487000000000002e-05 2024-09-14T06:17:33.087 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.087 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.087 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.087 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987958+0000", 2024-09-14T06:17:33.087 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4453e-05 2024-09-14T06:17:33.088 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.088 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.088 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.088 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064571+0000", 2024-09-14T06:17:33.088 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076612988000000007 2024-09-14T06:17:33.088 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.088 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.088 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.088 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064579+0000", 2024-09-14T06:17:33.088 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8150000000000006e-06 2024-09-14T06:17:33.088 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.088 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.088 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.089 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.089 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.089 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11495 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.089 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:33.089 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.052098192000003, 2024-09-14T06:17:33.089 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076524746000000005, 2024-09-14T06:17:33.089 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.089 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.089 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.089 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.089 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.089 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.089 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11495 2024-09-14T06:17:33.089 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.090 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.090 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.090 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.090 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:33.090 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.090 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.090 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.090 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.090 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:33.090 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.090 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.090 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.090 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.091 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988087+0000", 2024-09-14T06:17:33.091 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0499999999999999e-06 2024-09-14T06:17:33.091 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.091 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.091 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.091 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988118+0000", 2024-09-14T06:17:33.091 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1900000000000003e-05 2024-09-14T06:17:33.091 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.091 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.091 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.091 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988138+0000", 2024-09-14T06:17:33.091 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.933e-05 2024-09-14T06:17:33.091 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.091 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.091 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.092 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988152+0000", 2024-09-14T06:17:33.092 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3889e-05 2024-09-14T06:17:33.092 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.092 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.092 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.092 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064603+0000", 2024-09-14T06:17:33.092 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076451082000000004 2024-09-14T06:17:33.092 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.092 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.092 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.092 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064610+0000", 2024-09-14T06:17:33.092 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4950000000000002e-06 2024-09-14T06:17:33.092 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.092 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.092 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.093 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.093 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.093 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11498 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.093 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:33.093 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.052012116, 2024-09-14T06:17:33.093 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076471883000000004, 2024-09-14T06:17:33.093 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.093 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.093 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.093 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.093 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.093 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.093 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11498 2024-09-14T06:17:33.094 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.094 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.094 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.094 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.094 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:33.094 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.094 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.094 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.094 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.094 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:33.094 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.094 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.094 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.094 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.094 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988173+0000", 2024-09-14T06:17:33.095 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2300000000000001e-06 2024-09-14T06:17:33.095 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.095 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.095 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.095 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988178+0000", 2024-09-14T06:17:33.095 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4229999999999996e-06 2024-09-14T06:17:33.095 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.095 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.095 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.095 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988235+0000", 2024-09-14T06:17:33.095 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6616999999999997e-05 2024-09-14T06:17:33.095 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.095 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.095 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.095 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988264+0000", 2024-09-14T06:17:33.096 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9189000000000001e-05 2024-09-14T06:17:33.096 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.096 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.096 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.096 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064636+0000", 2024-09-14T06:17:33.096 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076371573999999998 2024-09-14T06:17:33.096 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.096 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.096 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.096 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064643+0000", 2024-09-14T06:17:33.096 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8499999999999994e-06 2024-09-14T06:17:33.096 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.096 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.096 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.097 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.097 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.097 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11523 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.097 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:33.097 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.051291968000001, 2024-09-14T06:17:33.097 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076062454000000002, 2024-09-14T06:17:33.097 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.097 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.097 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.097 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.097 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.097 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.097 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11523 2024-09-14T06:17:33.098 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.098 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.098 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.098 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.098 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:33.098 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.098 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.098 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.098 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.098 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:33.098 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.098 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.098 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.098 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.098 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988894+0000", 2024-09-14T06:17:33.099 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.843e-06 2024-09-14T06:17:33.099 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.099 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.099 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.099 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988903+0000", 2024-09-14T06:17:33.099 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0440000000000001e-06 2024-09-14T06:17:33.099 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.099 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.099 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.099 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988929+0000", 2024-09-14T06:17:33.099 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6679e-05 2024-09-14T06:17:33.099 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.099 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.099 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.100 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988967+0000", 2024-09-14T06:17:33.100 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7453000000000002e-05 2024-09-14T06:17:33.100 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.100 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.100 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.100 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064668+0000", 2024-09-14T06:17:33.100 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075701389999999993 2024-09-14T06:17:33.100 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.100 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.100 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.100 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064954+0000", 2024-09-14T06:17:33.100 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028604500000000002 2024-09-14T06:17:33.100 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.101 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.101 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.101 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.101 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.101 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11530 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.101 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:33.101 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.050801417999999, 2024-09-14T06:17:33.101 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075620674999999998, 2024-09-14T06:17:33.101 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.101 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.101 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.101 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.101 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11530 2024-09-14T06:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989384+0000", 2024-09-14T06:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6160000000000001e-06 2024-09-14T06:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989390+0000", 2024-09-14T06:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7039999999999999e-06 2024-09-14T06:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989409+0000", 2024-09-14T06:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.95e-05 2024-09-14T06:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989429+0000", 2024-09-14T06:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0173000000000002e-05 2024-09-14T06:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064992+0000", 2024-09-14T06:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075562764000000004 2024-09-14T06:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065003+0000", 2024-09-14T06:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0917999999999999e-05 2024-09-14T06:17:33.105 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.105 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.105 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.105 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.105 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.105 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11532 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.105 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:33.105 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.050731022000001, 2024-09-14T06:17:33.105 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075604656000000006, 2024-09-14T06:17:33.105 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.105 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.105 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.105 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.105 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11532 2024-09-14T06:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.107 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989454+0000", 2024-09-14T06:17:33.107 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6500000000000001e-06 2024-09-14T06:17:33.107 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.107 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.107 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.107 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989460+0000", 2024-09-14T06:17:33.107 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7459999999999997e-06 2024-09-14T06:17:33.107 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.107 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.107 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.107 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989480+0000", 2024-09-14T06:17:33.107 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.948e-05 2024-09-14T06:17:33.107 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.107 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.107 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.108 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989497+0000", 2024-09-14T06:17:33.108 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7153e-05 2024-09-14T06:17:33.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.108 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.108 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.108 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065046+0000", 2024-09-14T06:17:33.108 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075548905999999999 2024-09-14T06:17:33.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.108 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.108 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.108 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065057+0000", 2024-09-14T06:17:33.108 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1721e-05 2024-09-14T06:17:33.108 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.108 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.108 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.109 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.109 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.109 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11534 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.109 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:33.109 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.050664535999999, 2024-09-14T06:17:33.109 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075574350999999998, 2024-09-14T06:17:33.109 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.109 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.109 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.109 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.109 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.109 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.109 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11534 2024-09-14T06:17:33.109 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.109 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.110 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.110 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.110 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:33.110 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.110 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.110 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.110 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.110 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:33.110 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.110 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.110 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.110 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.110 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989521+0000", 2024-09-14T06:17:33.110 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.533e-06 2024-09-14T06:17:33.110 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.111 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.111 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.111 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989527+0000", 2024-09-14T06:17:33.111 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0700000000000003e-06 2024-09-14T06:17:33.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.111 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.111 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.111 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989546+0000", 2024-09-14T06:17:33.111 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.948e-05 2024-09-14T06:17:33.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.111 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.111 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.111 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989561+0000", 2024-09-14T06:17:33.111 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4827000000000001e-05 2024-09-14T06:17:33.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.112 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.112 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.112 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065086+0000", 2024-09-14T06:17:33.112 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075524542 2024-09-14T06:17:33.112 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.112 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.112 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.112 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065094+0000", 2024-09-14T06:17:33.112 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8990000000000001e-06 2024-09-14T06:17:33.112 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.112 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.112 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.112 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.112 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.112 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11535 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.113 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:33.113 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.050597173, 2024-09-14T06:17:33.113 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075542466000000003, 2024-09-14T06:17:33.113 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.113 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.113 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.113 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.113 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.113 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.113 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11535 2024-09-14T06:17:33.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.113 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.113 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.113 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.114 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:33.114 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.114 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.114 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:33.114 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.114 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.114 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989588+0000", 2024-09-14T06:17:33.114 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1659999999999999e-06 2024-09-14T06:17:33.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.114 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.114 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989593+0000", 2024-09-14T06:17:33.115 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6230000000000002e-06 2024-09-14T06:17:33.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.115 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.115 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.115 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989613+0000", 2024-09-14T06:17:33.115 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9604000000000001e-05 2024-09-14T06:17:33.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.115 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.115 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.115 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989643+0000", 2024-09-14T06:17:33.115 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0450000000000002e-05 2024-09-14T06:17:33.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.115 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.115 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.115 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065119+0000", 2024-09-14T06:17:33.115 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075475930999999996 2024-09-14T06:17:33.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.116 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.116 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.116 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065129+0000", 2024-09-14T06:17:33.116 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6919999999999997e-06 2024-09-14T06:17:33.116 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.116 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.116 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.116 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.116 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11537 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.116 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:33.116 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.050519250000001, 2024-09-14T06:17:33.117 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075497868999999995, 2024-09-14T06:17:33.117 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.117 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.117 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.117 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.117 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.117 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.117 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11537 2024-09-14T06:17:33.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.117 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.117 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.117 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.117 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:33.117 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.118 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:33.118 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.118 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989666+0000", 2024-09-14T06:17:33.118 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.48e-06 2024-09-14T06:17:33.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.118 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989672+0000", 2024-09-14T06:17:33.118 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7960000000000001e-06 2024-09-14T06:17:33.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.119 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.119 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.119 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989690+0000", 2024-09-14T06:17:33.119 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8743e-05 2024-09-14T06:17:33.119 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.119 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.119 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.119 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989704+0000", 2024-09-14T06:17:33.119 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3913e-05 2024-09-14T06:17:33.119 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.119 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.119 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.119 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065155+0000", 2024-09-14T06:17:33.119 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075450351999999998 2024-09-14T06:17:33.119 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.120 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.120 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.120 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065162+0000", 2024-09-14T06:17:33.120 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5850000000000002e-06 2024-09-14T06:17:33.120 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.120 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.120 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.120 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.120 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.120 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11543 3.12 3.4d02a8b2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.120 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:33.120 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.050241686, 2024-09-14T06:17:33.120 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074335152000000002, 2024-09-14T06:17:33.120 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.121 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.121 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:33.121 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.121 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.121 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.121 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11543 2024-09-14T06:17:33.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.121 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.121 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.121 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:33.121 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.121 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.122 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:33.122 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.122 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.122 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.122 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989944+0000", 2024-09-14T06:17:33.122 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.694e-06 2024-09-14T06:17:33.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.122 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.122 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.122 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989950+0000", 2024-09-14T06:17:33.122 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3389999999999998e-06 2024-09-14T06:17:33.124 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.124 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.124 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.124 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989969+0000", 2024-09-14T06:17:33.124 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8963000000000001e-05 2024-09-14T06:17:33.124 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.124 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.124 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.124 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989985+0000", 2024-09-14T06:17:33.124 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5586999999999998e-05 2024-09-14T06:17:33.124 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.124 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.124 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.125 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064270+0000", 2024-09-14T06:17:33.125 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074285552000000005 2024-09-14T06:17:33.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.125 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.125 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.125 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064277+0000", 2024-09-14T06:17:33.125 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0169999999999999e-06 2024-09-14T06:17:33.125 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.125 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.125 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.125 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.125 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.125 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:33.127 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:17:33.127 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:33.122+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:33.227 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:33.226+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:33.252 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:33.253 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:33.253 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:33.253 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:33.265 INFO:teuthology.orchestra.run.smithi039.stdout:5534 punch from 0x1b58d44 to 0x1b5d1c3, (0x447f bytes) 2024-09-14T06:17:33.266 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:17:33.273 INFO:teuthology.orchestra.run.smithi039.stdout:5535 write 0x92fec97 thru 0x930aac9 (0xbe33 bytes) 2024-09-14T06:17:33.281 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:33.281 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:33.289 INFO:teuthology.orchestra.run.smithi039.stdout:5541 write 0xeb47cd9 thru 0xeb4f321 (0x7649 bytes) 2024-09-14T06:17:33.294 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:33.294 INFO:teuthology.orchestra.run.smithi039.stdout:5543 write 0x58696af thru 0x5872ba8 (0x94fa bytes) 2024-09-14T06:17:33.300 INFO:teuthology.orchestra.run.smithi039.stdout:5545 read 0x48b2749 thru 0x48bc537 (0x9def bytes) 2024-09-14T06:17:33.300 INFO:teuthology.orchestra.run.smithi039.stdout:5546 write 0x8396a3f thru 0x83a6091 (0xf653 bytes) 2024-09-14T06:17:33.308 INFO:teuthology.orchestra.run.smithi039.stdout:5549 writesame 0xa332e10 thru 0xa335bab data_size 0x2a(0x2d9c bytes) 2024-09-14T06:17:33.314 INFO:teuthology.orchestra.run.smithi039.stdout:5550 writesame 0xabe89ea thru 0xabf0870 data_size 0x3d(0x7e87 bytes) 2024-09-14T06:17:33.328 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:33.325+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:33.346 INFO:teuthology.orchestra.run.smithi039.stdout:5551 trunc from 0xeb4f322 to 0x28f8308 2024-09-14T06:17:33.396 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:33.396 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:33.396 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:33.396 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:33.396 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:33.411 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:17:33.411 INFO:teuthology.orchestra.run.smithi039.stdout:5552 write 0xa4331f6 thru 0xa433941 (0x74c bytes) 2024-09-14T06:17:33.417 INFO:teuthology.orchestra.run.smithi039.stdout:5553 read 0xb5692a thru 0xb5f546 (0x8c1d bytes) 2024-09-14T06:17:33.418 INFO:teuthology.orchestra.run.smithi039.stdout:5556 read 0xa94fc4 thru 0xa9e266 (0x92a3 bytes) 2024-09-14T06:17:33.419 INFO:teuthology.orchestra.run.smithi039.stdout:5558 read 0x8ef521b thru 0x8efc276 (0x705c bytes) 2024-09-14T06:17:33.420 INFO:teuthology.orchestra.run.smithi039.stdout:5561 writesame 0x589dfa thru 0x595a9b data_size 0x37(0xbca2 bytes) 2024-09-14T06:17:33.430 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:33.430 INFO:teuthology.orchestra.run.smithi039.stdout:5563 write 0x14be2fa thru 0x14c458a (0x6291 bytes) 2024-09-14T06:17:33.441 INFO:teuthology.orchestra.run.smithi039.stdout:5564 read 0x915d196 thru 0x91660d1 (0x8f3c bytes) 2024-09-14T06:17:33.441 INFO:teuthology.orchestra.run.smithi039.stdout:5565 read 0x5cd2390 thru 0x5cd2a60 (0x6d1 bytes) 2024-09-14T06:17:33.453 INFO:teuthology.orchestra.run.smithi039.stdout:5566 writesame 0xb21ba41 thru 0xb224174 data_size 0x22(0x8734 bytes) 2024-09-14T06:17:33.453 INFO:teuthology.orchestra.run.smithi039.stdout:5569 writesame 0x75698f thru 0x75d6bf data_size 0x1(0x6d31 bytes) 2024-09-14T06:17:33.459 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:33.457+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:33.478 INFO:teuthology.orchestra.run.smithi039.stdout:5571 punch from 0x70c3eec to 0x70c6435, (0x2549 bytes) 2024-09-14T06:17:33.478 INFO:teuthology.orchestra.run.smithi039.stdout:5573 punch from 0x1354140 to 0x13573ea, (0x32aa bytes) 2024-09-14T06:17:33.493 INFO:teuthology.orchestra.run.smithi039.stdout:5574 write 0xb8fa0a3 thru 0xb8fae65 (0xdc3 bytes) 2024-09-14T06:17:33.501 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:33.501 INFO:teuthology.orchestra.run.smithi039.stdout:5580 punch from 0xdd82f6 to 0xde192a, (0x9634 bytes) 2024-09-14T06:17:33.501 INFO:teuthology.orchestra.run.smithi039.stdout:5582 read 0x5c7ad3c thru 0x5c7c5e1 (0x18a6 bytes) 2024-09-14T06:17:33.513 INFO:teuthology.orchestra.run.smithi039.stdout:5583 trunc from 0xb8fae66 to 0x74ab8a0 2024-09-14T06:17:33.530 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:33.525+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:33.537 INFO:teuthology.orchestra.run.smithi039.stdout:5584 punch from 0x3b55bfc to 0x3b61610, (0xba14 bytes) 2024-09-14T06:17:33.537 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:33.542 INFO:teuthology.orchestra.run.smithi039.stdout:5587 trunc from 0x74ab8a0 to 0x8f5a831 2024-09-14T06:17:33.557 INFO:teuthology.orchestra.run.smithi039.stdout:5589 read 0x7387f35 thru 0x738eb7b (0x6c47 bytes) 2024-09-14T06:17:33.563 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T06:17:33.574 INFO:teuthology.orchestra.run.smithi039.stdout:5590 write 0x99096ff thru 0x990b47a (0x1d7c bytes) 2024-09-14T06:17:33.588 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:33.589 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:33.589 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:33.589 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:33.589 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.589 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12584 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.589 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.039784+0000", 2024-09-14T06:17:33.589 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.547408761, 2024-09-14T06:17:33.589 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.024433808000000001, 2024-09-14T06:17:33.589 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.589 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.589 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.589 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.590 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.590 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.590 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12584 2024-09-14T06:17:33.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.590 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.590 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.590 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.590 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039784+0000", 2024-09-14T06:17:33.590 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.590 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.590 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.590 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039784+0000", 2024-09-14T06:17:33.590 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.591 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.591 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.591 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.591 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039784+0000", 2024-09-14T06:17:33.591 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1999999999999999e-07 2024-09-14T06:17:33.591 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.591 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.591 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.591 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039788+0000", 2024-09-14T06:17:33.591 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.41e-06 2024-09-14T06:17:33.591 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.591 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.591 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.591 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039797+0000", 2024-09-14T06:17:33.592 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7969999999999995e-06 2024-09-14T06:17:33.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.592 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.592 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.592 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039804+0000", 2024-09-14T06:17:33.592 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7870000000000004e-06 2024-09-14T06:17:33.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.592 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.592 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.592 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.048222+0000", 2024-09-14T06:17:33.592 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0084172770000000008 2024-09-14T06:17:33.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.592 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.592 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.593 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.048334+0000", 2024-09-14T06:17:33.593 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000112895 2024-09-14T06:17:33.593 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.593 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.593 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.593 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064217+0000", 2024-09-14T06:17:33.593 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015882922000000001 2024-09-14T06:17:33.593 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.593 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.593 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.593 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.593 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.593 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12661 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.593 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.040519+0000", 2024-09-14T06:17:33.594 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.546672981, 2024-09-14T06:17:33.594 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014404233000000001, 2024-09-14T06:17:33.594 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.594 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.594 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.594 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.594 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.594 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.594 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12661 2024-09-14T06:17:33.594 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.594 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.594 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.594 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.594 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040519+0000", 2024-09-14T06:17:33.595 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.595 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.595 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.595 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040519+0000", 2024-09-14T06:17:33.595 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.595 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.595 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.595 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040520+0000", 2024-09-14T06:17:33.595 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6899999999999996e-07 2024-09-14T06:17:33.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.595 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.595 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.596 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040530+0000", 2024-09-14T06:17:33.596 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6409999999999997e-06 2024-09-14T06:17:33.596 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.596 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.596 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.596 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040540+0000", 2024-09-14T06:17:33.596 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0247e-05 2024-09-14T06:17:33.596 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.596 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.596 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.596 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040547+0000", 2024-09-14T06:17:33.596 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3830000000000003e-06 2024-09-14T06:17:33.596 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.596 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.597 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.597 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.054677+0000", 2024-09-14T06:17:33.597 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014130095000000001 2024-09-14T06:17:33.597 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.597 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.597 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.597 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.054786+0000", 2024-09-14T06:17:33.597 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000108441 2024-09-14T06:17:33.597 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.597 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.597 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.597 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.054924+0000", 2024-09-14T06:17:33.597 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013765699999999999 2024-09-14T06:17:33.597 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.598 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.598 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.598 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.598 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12662 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.598 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.040559+0000", 2024-09-14T06:17:33.598 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.546633644000003, 2024-09-14T06:17:33.598 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014659604999999999, 2024-09-14T06:17:33.598 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.598 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.598 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.598 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.598 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.598 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.599 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12662 2024-09-14T06:17:33.599 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.599 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.599 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.599 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.599 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040559+0000", 2024-09-14T06:17:33.599 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.599 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.599 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.599 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.599 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040559+0000", 2024-09-14T06:17:33.599 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.599 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.599 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.600 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.600 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040559+0000", 2024-09-14T06:17:33.600 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5799999999999998e-07 2024-09-14T06:17:33.600 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.600 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.600 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.600 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040562+0000", 2024-09-14T06:17:33.600 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9349999999999999e-06 2024-09-14T06:17:33.600 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.600 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.600 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.600 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040572+0000", 2024-09-14T06:17:33.600 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9480000000000003e-06 2024-09-14T06:17:33.600 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.601 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.601 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040579+0000", 2024-09-14T06:17:33.601 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1459999999999999e-06 2024-09-14T06:17:33.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.601 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.601 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.054963+0000", 2024-09-14T06:17:33.601 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014383986 2024-09-14T06:17:33.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.601 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.601 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055084+0000", 2024-09-14T06:17:33.601 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000120822 2024-09-14T06:17:33.602 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.602 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.602 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.602 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055218+0000", 2024-09-14T06:17:33.602 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013401 2024-09-14T06:17:33.602 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.602 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.602 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.602 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.602 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.602 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12666 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.602 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.040590+0000", 2024-09-14T06:17:33.602 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.546602495999998, 2024-09-14T06:17:33.602 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014895584999999999, 2024-09-14T06:17:33.602 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.602 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.603 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.603 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.603 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.603 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.603 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12666 2024-09-14T06:17:33.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.603 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.603 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.603 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040590+0000", 2024-09-14T06:17:33.603 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.603 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.603 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040590+0000", 2024-09-14T06:17:33.604 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.604 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.604 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.604 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040590+0000", 2024-09-14T06:17:33.604 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:17:33.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.604 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.604 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.604 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040593+0000", 2024-09-14T06:17:33.604 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0189999999999998e-06 2024-09-14T06:17:33.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.604 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.604 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.604 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040603+0000", 2024-09-14T06:17:33.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2820000000000002e-06 2024-09-14T06:17:33.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.605 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.605 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040610+0000", 2024-09-14T06:17:33.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1219999999999997e-06 2024-09-14T06:17:33.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.605 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.605 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055257+0000", 2024-09-14T06:17:33.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014646762000000001 2024-09-14T06:17:33.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.605 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.605 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055379+0000", 2024-09-14T06:17:33.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012186899999999999 2024-09-14T06:17:33.606 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.606 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.606 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.606 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055485+0000", 2024-09-14T06:17:33.606 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000106848 2024-09-14T06:17:33.606 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.606 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.606 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.606 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.606 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.606 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12672 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.606 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.040620+0000", 2024-09-14T06:17:33.606 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.546572013000002, 2024-09-14T06:17:33.606 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015140836, 2024-09-14T06:17:33.606 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.607 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.607 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.607 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.607 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.607 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.607 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12672 2024-09-14T06:17:33.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.607 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.607 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.607 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.607 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040620+0000", 2024-09-14T06:17:33.607 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.607 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040620+0000", 2024-09-14T06:17:33.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040621+0000", 2024-09-14T06:17:33.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9699999999999995e-07 2024-09-14T06:17:33.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040632+0000", 2024-09-14T06:17:33.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1012999999999999e-05 2024-09-14T06:17:33.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.609 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040642+0000", 2024-09-14T06:17:33.609 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0054999999999999e-05 2024-09-14T06:17:33.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.609 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.609 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.609 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040649+0000", 2024-09-14T06:17:33.609 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3769999999999998e-06 2024-09-14T06:17:33.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.609 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.609 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.609 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055521+0000", 2024-09-14T06:17:33.609 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014871383 2024-09-14T06:17:33.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.609 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.610 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.610 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055634+0000", 2024-09-14T06:17:33.610 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000112757 2024-09-14T06:17:33.610 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.610 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.610 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.610 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055761+0000", 2024-09-14T06:17:33.610 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012755399999999999 2024-09-14T06:17:33.610 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.610 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.610 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.610 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.610 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.610 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12673 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 385024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.611 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.041758+0000", 2024-09-14T06:17:33.611 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.545433799000001, 2024-09-14T06:17:33.611 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014243184000000001, 2024-09-14T06:17:33.611 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.611 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.611 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.611 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.611 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.611 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.611 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12673 2024-09-14T06:17:33.611 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.611 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.611 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.611 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.611 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041758+0000", 2024-09-14T06:17:33.611 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.612 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.612 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.612 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.612 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041758+0000", 2024-09-14T06:17:33.612 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.612 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.612 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.612 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.612 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041760+0000", 2024-09-14T06:17:33.612 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0860000000000001e-06 2024-09-14T06:17:33.612 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.612 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.612 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.612 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041766+0000", 2024-09-14T06:17:33.612 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-06 2024-09-14T06:17:33.612 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.613 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.613 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.613 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041794+0000", 2024-09-14T06:17:33.613 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7993000000000001e-05 2024-09-14T06:17:33.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.613 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.613 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.613 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041813+0000", 2024-09-14T06:17:33.613 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.823e-05 2024-09-14T06:17:33.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.613 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.613 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.613 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055797+0000", 2024-09-14T06:17:33.613 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013984612 2024-09-14T06:17:33.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.614 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.614 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.614 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055900+0000", 2024-09-14T06:17:33.614 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000102464 2024-09-14T06:17:33.614 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.614 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.614 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.614 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.056002+0000", 2024-09-14T06:17:33.614 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000101869 2024-09-14T06:17:33.614 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.614 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.614 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.614 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.614 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.614 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12674 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.614 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.041833+0000", 2024-09-14T06:17:33.615 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.545359269999999, 2024-09-14T06:17:33.615 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022511527, 2024-09-14T06:17:33.615 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.615 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.615 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.615 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.615 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.615 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.615 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12674 2024-09-14T06:17:33.615 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.615 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.615 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.615 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.615 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041833+0000", 2024-09-14T06:17:33.615 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.615 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.616 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.616 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.616 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041833+0000", 2024-09-14T06:17:33.616 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.616 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.616 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.616 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041834+0000", 2024-09-14T06:17:33.616 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7999999999999995e-07 2024-09-14T06:17:33.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.616 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.616 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.616 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041869+0000", 2024-09-14T06:17:33.616 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5766000000000003e-05 2024-09-14T06:17:33.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.617 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.617 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.617 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041922+0000", 2024-09-14T06:17:33.617 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2086000000000002e-05 2024-09-14T06:17:33.617 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.617 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.617 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.617 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041931+0000", 2024-09-14T06:17:33.617 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6160000000000008e-06 2024-09-14T06:17:33.617 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.617 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.617 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.617 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.056038+0000", 2024-09-14T06:17:33.617 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01410729 2024-09-14T06:17:33.617 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.618 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.618 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.618 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.056143+0000", 2024-09-14T06:17:33.618 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000104697 2024-09-14T06:17:33.618 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.618 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.618 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.618 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064345+0000", 2024-09-14T06:17:33.618 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0082013920000000001 2024-09-14T06:17:33.618 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.618 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.618 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.619 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.619 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.619 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12677 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.619 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.041952+0000", 2024-09-14T06:17:33.619 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.545239852999998, 2024-09-14T06:17:33.619 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022599910000000001, 2024-09-14T06:17:33.619 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.619 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.619 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.619 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.619 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.619 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.619 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12677 2024-09-14T06:17:33.619 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.619 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.620 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.620 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.620 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041952+0000", 2024-09-14T06:17:33.620 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.620 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.620 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.620 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.620 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041952+0000", 2024-09-14T06:17:33.620 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.620 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.620 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.620 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.620 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041953+0000", 2024-09-14T06:17:33.620 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1100000000000005e-07 2024-09-14T06:17:33.621 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.621 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.621 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.621 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041958+0000", 2024-09-14T06:17:33.621 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3839999999999999e-06 2024-09-14T06:17:33.621 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.621 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.621 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.621 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042800+0000", 2024-09-14T06:17:33.621 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00084272399999999999 2024-09-14T06:17:33.621 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.621 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.621 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.621 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042817+0000", 2024-09-14T06:17:33.621 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6507e-05 2024-09-14T06:17:33.622 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.622 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.622 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.622 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064375+0000", 2024-09-14T06:17:33.622 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021558165000000001 2024-09-14T06:17:33.622 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.622 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.622 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.622 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064458+0000", 2024-09-14T06:17:33.622 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3205e-05 2024-09-14T06:17:33.622 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.622 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.622 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.622 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064552+0000", 2024-09-14T06:17:33.622 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.4114e-05 2024-09-14T06:17:33.622 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.623 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.623 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.623 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.623 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.623 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12678 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.623 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.042838+0000", 2024-09-14T06:17:33.623 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.544354224999999, 2024-09-14T06:17:33.623 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021916718000000002, 2024-09-14T06:17:33.623 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.623 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.623 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.623 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.623 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.623 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.623 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12678 2024-09-14T06:17:33.623 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.624 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.624 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.624 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.624 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042838+0000", 2024-09-14T06:17:33.624 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.624 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.624 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.624 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.624 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042838+0000", 2024-09-14T06:17:33.624 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.624 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.624 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.624 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.624 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042839+0000", 2024-09-14T06:17:33.624 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1510000000000001e-06 2024-09-14T06:17:33.625 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.625 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.625 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.625 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042845+0000", 2024-09-14T06:17:33.625 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9190000000000004e-06 2024-09-14T06:17:33.625 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.625 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.625 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.625 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042859+0000", 2024-09-14T06:17:33.625 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4022000000000001e-05 2024-09-14T06:17:33.625 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.625 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.625 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.625 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042867+0000", 2024-09-14T06:17:33.625 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9549999999999992e-06 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064581+0000", 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021713542999999998 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064656+0000", 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5564999999999999e-05 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064755+0000", 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.8562999999999998e-05 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.626 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.627 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.627 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.627 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.627 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12687 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.627 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.042878+0000", 2024-09-14T06:17:33.627 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.544314430999997, 2024-09-14T06:17:33.627 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022072561000000001, 2024-09-14T06:17:33.627 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.627 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.627 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.627 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.627 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.627 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.627 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12687 2024-09-14T06:17:33.627 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.627 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.628 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.628 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.628 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042878+0000", 2024-09-14T06:17:33.628 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.628 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.628 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.628 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.628 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042878+0000", 2024-09-14T06:17:33.628 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.628 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.628 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.628 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.628 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042879+0000", 2024-09-14T06:17:33.628 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9999999999999997e-07 2024-09-14T06:17:33.628 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.628 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.629 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.629 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042882+0000", 2024-09-14T06:17:33.629 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.608e-06 2024-09-14T06:17:33.629 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.629 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.629 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.629 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042892+0000", 2024-09-14T06:17:33.629 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9729999999999992e-06 2024-09-14T06:17:33.629 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.629 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.629 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.629 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042911+0000", 2024-09-14T06:17:33.629 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9369999999999999e-05 2024-09-14T06:17:33.629 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.629 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.630 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.630 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064785+0000", 2024-09-14T06:17:33.630 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021873182000000001 2024-09-14T06:17:33.630 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.630 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.630 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.630 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064860+0000", 2024-09-14T06:17:33.630 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5403999999999998e-05 2024-09-14T06:17:33.630 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.630 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.630 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.630 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064950+0000", 2024-09-14T06:17:33.630 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0323999999999998e-05 2024-09-14T06:17:33.630 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.631 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.631 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.631 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.631 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.631 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12688 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.631 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.042922+0000", 2024-09-14T06:17:33.631 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.544270107999999, 2024-09-14T06:17:33.631 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022202118, 2024-09-14T06:17:33.631 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.631 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.631 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.631 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.631 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.631 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.631 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12688 2024-09-14T06:17:33.632 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.632 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.632 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.632 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.632 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042922+0000", 2024-09-14T06:17:33.632 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.632 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.632 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.632 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.632 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042922+0000", 2024-09-14T06:17:33.632 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.632 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.632 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.632 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.632 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042923+0000", 2024-09-14T06:17:33.633 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:17:33.633 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.633 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.633 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.633 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042937+0000", 2024-09-14T06:17:33.633 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4337e-05 2024-09-14T06:17:33.633 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.633 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.633 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.633 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042948+0000", 2024-09-14T06:17:33.633 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0569e-05 2024-09-14T06:17:33.633 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.633 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.633 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.634 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042955+0000", 2024-09-14T06:17:33.634 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3250000000000002e-06 2024-09-14T06:17:33.634 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.634 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.634 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.634 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064981+0000", 2024-09-14T06:17:33.634 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022026046000000001 2024-09-14T06:17:33.634 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.634 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.634 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.634 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065055+0000", 2024-09-14T06:17:33.634 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4016000000000004e-05 2024-09-14T06:17:33.634 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.634 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.634 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.635 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065124+0000", 2024-09-14T06:17:33.635 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9141999999999996e-05 2024-09-14T06:17:33.635 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.635 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.635 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.635 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.635 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.635 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12690 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.635 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044605+0000", 2024-09-14T06:17:33.635 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.542587068000003, 2024-09-14T06:17:33.635 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020723715, 2024-09-14T06:17:33.635 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.635 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.635 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.635 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.636 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.636 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.636 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12690 2024-09-14T06:17:33.636 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.636 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.636 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.636 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.636 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044605+0000", 2024-09-14T06:17:33.636 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.636 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.636 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.636 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.636 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044605+0000", 2024-09-14T06:17:33.637 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.637 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.637 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.637 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.637 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044606+0000", 2024-09-14T06:17:33.637 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2020000000000001e-06 2024-09-14T06:17:33.637 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.637 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.637 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.637 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044612+0000", 2024-09-14T06:17:33.637 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9830000000000001e-06 2024-09-14T06:17:33.637 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.637 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.638 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.638 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044631+0000", 2024-09-14T06:17:33.638 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8280000000000001e-05 2024-09-14T06:17:33.638 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.638 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.638 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.638 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044642+0000", 2024-09-14T06:17:33.638 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1018e-05 2024-09-14T06:17:33.638 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.638 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.638 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.638 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065153+0000", 2024-09-14T06:17:33.638 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020511709999999999 2024-09-14T06:17:33.638 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.638 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.639 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.639 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065230+0000", 2024-09-14T06:17:33.639 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7003999999999997e-05 2024-09-14T06:17:33.639 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.639 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.639 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.639 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065329+0000", 2024-09-14T06:17:33.639 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.8517999999999994e-05 2024-09-14T06:17:33.639 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.639 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.639 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.639 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.639 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.639 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12692 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.640 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044659+0000", 2024-09-14T06:17:33.640 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.542533239000001, 2024-09-14T06:17:33.640 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020896346999999999, 2024-09-14T06:17:33.640 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.640 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.640 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.640 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.640 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.640 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.640 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12692 2024-09-14T06:17:33.640 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.640 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.641 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.641 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.641 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044659+0000", 2024-09-14T06:17:33.641 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.641 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.641 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.641 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.641 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044659+0000", 2024-09-14T06:17:33.641 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.641 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.641 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.641 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.641 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044660+0000", 2024-09-14T06:17:33.641 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0200000000000001e-07 2024-09-14T06:17:33.642 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.642 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.642 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.642 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044663+0000", 2024-09-14T06:17:33.642 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:17:33.642 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.642 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.642 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.642 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044674+0000", 2024-09-14T06:17:33.642 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.096e-05 2024-09-14T06:17:33.642 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.642 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.642 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.642 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044682+0000", 2024-09-14T06:17:33.643 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7149999999999998e-06 2024-09-14T06:17:33.643 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.643 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.643 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.643 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065357+0000", 2024-09-14T06:17:33.643 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020674794999999999 2024-09-14T06:17:33.643 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.643 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.643 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.643 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065435+0000", 2024-09-14T06:17:33.643 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8170999999999994e-05 2024-09-14T06:17:33.643 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.643 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.643 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.643 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065555+0000", 2024-09-14T06:17:33.644 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000120609 2024-09-14T06:17:33.644 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.644 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.644 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.644 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.644 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.644 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12694 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.644 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044692+0000", 2024-09-14T06:17:33.644 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.542499964000001, 2024-09-14T06:17:33.644 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021050059999999999, 2024-09-14T06:17:33.644 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.644 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.644 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.644 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.645 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.645 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.645 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12694 2024-09-14T06:17:33.645 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.645 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.645 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.645 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.645 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044692+0000", 2024-09-14T06:17:33.645 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.645 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.645 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.645 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.645 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044692+0000", 2024-09-14T06:17:33.645 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.646 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.646 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.646 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.646 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044693+0000", 2024-09-14T06:17:33.646 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8800000000000002e-07 2024-09-14T06:17:33.646 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.646 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.646 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.646 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044705+0000", 2024-09-14T06:17:33.646 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2252000000000001e-05 2024-09-14T06:17:33.646 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.646 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.646 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.646 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044716+0000", 2024-09-14T06:17:33.646 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1206000000000001e-05 2024-09-14T06:17:33.647 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.647 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.647 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.647 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044724+0000", 2024-09-14T06:17:33.647 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7360000000000001e-06 2024-09-14T06:17:33.647 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.647 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.647 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.647 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065584+0000", 2024-09-14T06:17:33.647 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020859459 2024-09-14T06:17:33.647 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.647 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.647 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.647 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065657+0000", 2024-09-14T06:17:33.648 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3509000000000004e-05 2024-09-14T06:17:33.648 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.648 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.648 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.648 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065742+0000", 2024-09-14T06:17:33.648 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5210000000000006e-05 2024-09-14T06:17:33.648 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.648 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.648 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.648 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.648 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.648 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12702 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.648 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044734+0000", 2024-09-14T06:17:33.649 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.542457974000001, 2024-09-14T06:17:33.649 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021236848999999999, 2024-09-14T06:17:33.649 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.649 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.649 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.649 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.649 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.649 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.649 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12702 2024-09-14T06:17:33.649 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.649 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.649 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.649 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.649 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044734+0000", 2024-09-14T06:17:33.649 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.649 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.650 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.650 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.650 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044734+0000", 2024-09-14T06:17:33.650 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.650 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.650 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.650 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.650 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044735+0000", 2024-09-14T06:17:33.650 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3600000000000003e-07 2024-09-14T06:17:33.650 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.650 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.650 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.650 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044738+0000", 2024-09-14T06:17:33.651 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9189999999999999e-06 2024-09-14T06:17:33.651 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.651 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.651 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.651 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044748+0000", 2024-09-14T06:17:33.651 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0111e-05 2024-09-14T06:17:33.651 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.651 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.651 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.651 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044766+0000", 2024-09-14T06:17:33.651 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8199999999999999e-05 2024-09-14T06:17:33.651 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.651 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.651 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065771+0000", 2024-09-14T06:17:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021004683 2024-09-14T06:17:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065849+0000", 2024-09-14T06:17:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7684999999999999e-05 2024-09-14T06:17:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065971+0000", 2024-09-14T06:17:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012251499999999999 2024-09-14T06:17:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.652 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12703 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 401408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044777+0000", 2024-09-14T06:17:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.542415347999999, 2024-09-14T06:17:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021382453999999999, 2024-09-14T06:17:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12703 2024-09-14T06:17:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.653 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044777+0000", 2024-09-14T06:17:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044777+0000", 2024-09-14T06:17:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044778+0000", 2024-09-14T06:17:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2999999999999999e-07 2024-09-14T06:17:33.655 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.655 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.655 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.655 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044787+0000", 2024-09-14T06:17:33.655 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2429999999999994e-06 2024-09-14T06:17:33.655 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.655 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.655 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.655 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044798+0000", 2024-09-14T06:17:33.655 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0447e-05 2024-09-14T06:17:33.655 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.655 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.655 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.655 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044805+0000", 2024-09-14T06:17:33.655 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.182e-06 2024-09-14T06:17:33.656 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.656 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.656 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.656 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066001+0000", 2024-09-14T06:17:33.656 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021196079 2024-09-14T06:17:33.656 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.656 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.656 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.656 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066082+0000", 2024-09-14T06:17:33.656 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1011000000000001e-05 2024-09-14T06:17:33.656 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.656 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.656 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.656 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066159+0000", 2024-09-14T06:17:33.656 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7562e-05 2024-09-14T06:17:33.657 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.657 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.657 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.657 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.657 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.657 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12705 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.657 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044815+0000", 2024-09-14T06:17:33.657 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.542377508000001, 2024-09-14T06:17:33.657 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021573917000000001, 2024-09-14T06:17:33.657 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.657 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.657 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.657 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.657 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.657 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.658 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12705 2024-09-14T06:17:33.658 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.658 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.658 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.658 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.658 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044815+0000", 2024-09-14T06:17:33.658 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.658 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.658 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.658 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.658 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044815+0000", 2024-09-14T06:17:33.658 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.658 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.658 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.658 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.659 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044815+0000", 2024-09-14T06:17:33.659 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9999999999999997e-07 2024-09-14T06:17:33.659 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.659 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.659 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.659 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044818+0000", 2024-09-14T06:17:33.659 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7779999999999999e-06 2024-09-14T06:17:33.659 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.659 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.659 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.659 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044828+0000", 2024-09-14T06:17:33.659 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0095000000000001e-05 2024-09-14T06:17:33.659 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.659 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.659 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.660 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044835+0000", 2024-09-14T06:17:33.660 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9419999999999998e-06 2024-09-14T06:17:33.660 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.660 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.660 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.660 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066188+0000", 2024-09-14T06:17:33.660 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021352560999999999 2024-09-14T06:17:33.660 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.660 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.660 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.660 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066275+0000", 2024-09-14T06:17:33.660 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7020000000000004e-05 2024-09-14T06:17:33.660 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.660 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.661 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.661 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066389+0000", 2024-09-14T06:17:33.661 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000113921 2024-09-14T06:17:33.661 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.661 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.661 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.661 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.661 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.661 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12707 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:33.661 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044844+0000", 2024-09-14T06:17:33.661 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.542347939000003, 2024-09-14T06:17:33.661 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021756622, 2024-09-14T06:17:33.661 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.661 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.661 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:33.662 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.662 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.662 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.662 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12707 2024-09-14T06:17:33.662 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.662 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.662 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.662 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.662 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044844+0000", 2024-09-14T06:17:33.662 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.662 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.662 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.662 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.663 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044844+0000", 2024-09-14T06:17:33.663 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.663 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.663 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.663 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.663 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044845+0000", 2024-09-14T06:17:33.663 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3600000000000003e-07 2024-09-14T06:17:33.663 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.663 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.663 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.663 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044848+0000", 2024-09-14T06:17:33.663 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4710000000000001e-06 2024-09-14T06:17:33.663 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.663 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.664 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.664 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044857+0000", 2024-09-14T06:17:33.664 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5219999999999996e-06 2024-09-14T06:17:33.664 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.664 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.664 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.664 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044864+0000", 2024-09-14T06:17:33.664 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:17:33.664 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.664 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.664 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.664 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066424+0000", 2024-09-14T06:17:33.664 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021560461999999999 2024-09-14T06:17:33.664 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.664 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.665 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.665 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066511+0000", 2024-09-14T06:17:33.665 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6298999999999995e-05 2024-09-14T06:17:33.665 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.665 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.665 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.665 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066601+0000", 2024-09-14T06:17:33.665 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0252000000000003e-05 2024-09-14T06:17:33.665 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.665 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.665 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.665 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.665 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.666 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:24231 3.c e47/23 3:335dc9f4:::rbd_data.1152db28e106.0000000000000002:head v 47'36, mlcod=46'34)", 2024-09-14T06:17:33.666 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:51.909218+0000", 2024-09-14T06:17:33.666 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 41.677974534999997, 2024-09-14T06:17:33.666 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.018625953000000001, 2024-09-14T06:17:33.666 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.666 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.666 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:33.666 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.666 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.666 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.666 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.909218+0000", 2024-09-14T06:17:33.666 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.666 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.666 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.666 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.667 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.909218+0000", 2024-09-14T06:17:33.667 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.667 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.667 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.667 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.667 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.909221+0000", 2024-09-14T06:17:33.667 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:17:33.667 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.667 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.667 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.667 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.921926+0000", 2024-09-14T06:17:33.667 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.012705602 2024-09-14T06:17:33.667 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.667 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.667 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.667 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.921949+0000", 2024-09-14T06:17:33.668 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2824e-05 2024-09-14T06:17:33.668 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.668 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.668 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.668 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.921960+0000", 2024-09-14T06:17:33.668 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0626e-05 2024-09-14T06:17:33.668 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.668 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.668 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.668 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.922035+0000", 2024-09-14T06:17:33.668 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4938999999999995e-05 2024-09-14T06:17:33.668 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.668 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.668 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.668 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.922058+0000", 2024-09-14T06:17:33.669 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2847e-05 2024-09-14T06:17:33.669 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.669 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.669 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:33.669 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.927752+0000", 2024-09-14T06:17:33.669 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056944810000000004 2024-09-14T06:17:33.669 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.669 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.669 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.669 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.927844+0000", 2024-09-14T06:17:33.669 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1524e-05 2024-09-14T06:17:33.669 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.669 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.669 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.670 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.670 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.670 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:27637 3.b 3:d7b02ca9:::rbd_data.1152ff705991.00000000000000a5:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:17:33.670 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:55.303878+0000", 2024-09-14T06:17:33.670 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 38.283314605000001, 2024-09-14T06:17:33.670 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.027469031000000001, 2024-09-14T06:17:33.670 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:33.670 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:33.670 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:33.670 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:33.670 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:33.670 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:33.670 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 27637 2024-09-14T06:17:33.671 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.671 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:33.671 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.671 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:33.671 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303878+0000", 2024-09-14T06:17:33.671 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.671 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.671 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.671 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:33.671 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303878+0000", 2024-09-14T06:17:33.671 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:33.671 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.671 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.671 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:33.671 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303881+0000", 2024-09-14T06:17:33.671 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8430000000000001e-06 2024-09-14T06:17:33.672 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.672 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.672 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:33.672 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303894+0000", 2024-09-14T06:17:33.672 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3376e-05 2024-09-14T06:17:33.672 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.672 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.672 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:33.672 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303926+0000", 2024-09-14T06:17:33.672 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1825000000000001e-05 2024-09-14T06:17:33.672 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.672 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.672 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:33.673 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303941+0000", 2024-09-14T06:17:33.673 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5048999999999999e-05 2024-09-14T06:17:33.673 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.673 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.673 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:33.673 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.304106+0000", 2024-09-14T06:17:33.673 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00016504700000000001 2024-09-14T06:17:33.673 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.673 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.673 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:33.673 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.304357+0000", 2024-09-14T06:17:33.673 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025158899999999998 2024-09-14T06:17:33.673 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.673 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.673 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 6", 2024-09-14T06:17:33.674 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.308743+0000", 2024-09-14T06:17:33.674 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043854419999999998 2024-09-14T06:17:33.674 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.674 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.674 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:33.674 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.331026+0000", 2024-09-14T06:17:33.674 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022282999000000001 2024-09-14T06:17:33.674 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.674 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.674 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:33.674 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.331072+0000", 2024-09-14T06:17:33.674 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5694e-05 2024-09-14T06:17:33.674 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.674 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.674 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:33.675 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.331229+0000", 2024-09-14T06:17:33.675 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015776399999999999 2024-09-14T06:17:33.675 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:33.675 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:33.675 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:33.675 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.331347+0000", 2024-09-14T06:17:33.675 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000117403 2024-09-14T06:17:33.675 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.675 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.675 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.675 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:33.675 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:33.675 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:33.677 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T06:17:33.677 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:33.674+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:33.678 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:17:33.778 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:33.773+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:33.864 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:33.864 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:33.864 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:33.864 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:33.879 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:17:33.879 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:33.874+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:33.979 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:33.977+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:34.045 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:34.045 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:34.059 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:17:34.080 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:34.077+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:34.177 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T06:17:34.181 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:34.177+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:34.184 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40393 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 868352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.708703+0000", 2024-09-14T06:17:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.473036993000001, 2024-09-14T06:17:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12669228199999999, 2024-09-14T06:17:34.184 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40393 2024-09-14T06:17:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708703+0000", 2024-09-14T06:17:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708703+0000", 2024-09-14T06:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708881+0000", 2024-09-14T06:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00017825 2024-09-14T06:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708885+0000", 2024-09-14T06:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.019e-06 2024-09-14T06:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708894+0000", 2024-09-14T06:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5469999999999997e-06 2024-09-14T06:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708900+0000", 2024-09-14T06:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.421e-06 2024-09-14T06:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835012+0000", 2024-09-14T06:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12611203400000001 2024-09-14T06:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835095+0000", 2024-09-14T06:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.229e-05 2024-09-14T06:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835395+0000", 2024-09-14T06:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000300721 2024-09-14T06:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40395 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 876544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.708914+0000", 2024-09-14T06:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.472826360999999, 2024-09-14T06:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12688781199999999, 2024-09-14T06:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40395 2024-09-14T06:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708914+0000", 2024-09-14T06:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708914+0000", 2024-09-14T06:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709267+0000", 2024-09-14T06:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00035336999999999999 2024-09-14T06:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709272+0000", 2024-09-14T06:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4430000000000002e-06 2024-09-14T06:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709283+0000", 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1034e-05 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709291+0000", 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4739999999999999e-06 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835428+0000", 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126137052 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835506+0000", 2024-09-14T06:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8492000000000007e-05 2024-09-14T06:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835801+0000", 2024-09-14T06:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00029494699999999999 2024-09-14T06:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40397 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.709307+0000", 2024-09-14T06:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.472433257999999, 2024-09-14T06:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126901031, 2024-09-14T06:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40397 2024-09-14T06:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709307+0000", 2024-09-14T06:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709307+0000", 2024-09-14T06:17:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709590+0000", 2024-09-14T06:17:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00028368499999999999 2024-09-14T06:17:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709596+0000", 2024-09-14T06:17:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3940000000000004e-06 2024-09-14T06:17:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709605+0000", 2024-09-14T06:17:34.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8650000000000007e-06 2024-09-14T06:17:34.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.195 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.195 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709610+0000", 2024-09-14T06:17:34.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1139999999999997e-06 2024-09-14T06:17:34.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.195 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.195 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835834+0000", 2024-09-14T06:17:34.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12622423199999999 2024-09-14T06:17:34.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.195 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.195 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835915+0000", 2024-09-14T06:17:34.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0859999999999995e-05 2024-09-14T06:17:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836208+0000", 2024-09-14T06:17:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00029288099999999999 2024-09-14T06:17:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40399 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.709624+0000", 2024-09-14T06:17:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.472115461000001, 2024-09-14T06:17:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.127024368, 2024-09-14T06:17:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.197 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.197 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.197 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.197 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.197 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.197 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40399 2024-09-14T06:17:34.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.197 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709624+0000", 2024-09-14T06:17:34.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709624+0000", 2024-09-14T06:17:34.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710144+0000", 2024-09-14T06:17:34.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00051909700000000005 2024-09-14T06:17:34.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710156+0000", 2024-09-14T06:17:34.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2846e-05 2024-09-14T06:17:34.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.199 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710166+0000", 2024-09-14T06:17:34.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:17:34.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.199 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710174+0000", 2024-09-14T06:17:34.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:17:34.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.199 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836240+0000", 2024-09-14T06:17:34.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12606634899999999 2024-09-14T06:17:34.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836320+0000", 2024-09-14T06:17:34.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0073000000000004e-05 2024-09-14T06:17:34.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836649+0000", 2024-09-14T06:17:34.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032834999999999999 2024-09-14T06:17:34.200 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.200 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.200 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.201 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40401 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.201 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.710189+0000", 2024-09-14T06:17:34.201 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.471551294000001, 2024-09-14T06:17:34.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126903455, 2024-09-14T06:17:34.201 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.201 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.201 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.201 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.201 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.201 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.201 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40401 2024-09-14T06:17:34.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.201 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710189+0000", 2024-09-14T06:17:34.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710189+0000", 2024-09-14T06:17:34.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712809+0000", 2024-09-14T06:17:34.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026201179999999998 2024-09-14T06:17:34.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712815+0000", 2024-09-14T06:17:34.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7960000000000001e-06 2024-09-14T06:17:34.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712825+0000", 2024-09-14T06:17:34.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0044000000000001e-05 2024-09-14T06:17:34.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712832+0000", 2024-09-14T06:17:34.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.328e-06 2024-09-14T06:17:34.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836682+0000", 2024-09-14T06:17:34.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.123850141 2024-09-14T06:17:34.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836762+0000", 2024-09-14T06:17:34.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0207e-05 2024-09-14T06:17:34.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.837092+0000", 2024-09-14T06:17:34.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000329821 2024-09-14T06:17:34.204 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.204 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.204 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.204 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40407 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.205 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:34.205 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.468857879000002, 2024-09-14T06:17:34.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.127190898, 2024-09-14T06:17:34.205 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.205 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.205 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.205 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.205 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.205 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.205 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40407 2024-09-14T06:17:34.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.205 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.205 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0900000000000002e-07 2024-09-14T06:17:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712884+0000", 2024-09-14T06:17:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5579999999999999e-06 2024-09-14T06:17:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712887+0000", 2024-09-14T06:17:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3119999999999998e-06 2024-09-14T06:17:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712890+0000", 2024-09-14T06:17:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7e-06 2024-09-14T06:17:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.839114+0000", 2024-09-14T06:17:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126223954 2024-09-14T06:17:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.839197+0000", 2024-09-14T06:17:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3250000000000004e-05 2024-09-14T06:17:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840073+0000", 2024-09-14T06:17:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000875715 2024-09-14T06:17:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40409 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.468846098, 2024-09-14T06:17:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12739108900000001, 2024-09-14T06:17:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40409 2024-09-14T06:17:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.65e-07 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712895+0000", 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.325e-06 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712899+0000", 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.219e-06 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712901+0000", 2024-09-14T06:17:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7310000000000002e-06 2024-09-14T06:17:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.839921+0000", 2024-09-14T06:17:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12701995199999999 2024-09-14T06:17:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840006+0000", 2024-09-14T06:17:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5063999999999997e-05 2024-09-14T06:17:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840285+0000", 2024-09-14T06:17:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027853300000000002 2024-09-14T06:17:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40411 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712909+0000", 2024-09-14T06:17:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.468830727, 2024-09-14T06:17:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.128251319, 2024-09-14T06:17:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40411 2024-09-14T06:17:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712909+0000", 2024-09-14T06:17:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712909+0000", 2024-09-14T06:17:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712910+0000", 2024-09-14T06:17:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.39e-07 2024-09-14T06:17:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712911+0000", 2024-09-14T06:17:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.37e-06 2024-09-14T06:17:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712914+0000", 2024-09-14T06:17:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4999999999999999e-06 2024-09-14T06:17:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712929+0000", 2024-09-14T06:17:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4406000000000001e-05 2024-09-14T06:17:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840856+0000", 2024-09-14T06:17:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.127927546 2024-09-14T06:17:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840942+0000", 2024-09-14T06:17:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5232e-05 2024-09-14T06:17:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841161+0000", 2024-09-14T06:17:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021892600000000001 2024-09-14T06:17:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40413 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712938+0000", 2024-09-14T06:17:34.219 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.468801801000001, 2024-09-14T06:17:34.219 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.128612052, 2024-09-14T06:17:34.219 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.219 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.219 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.219 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.219 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.219 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.219 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40413 2024-09-14T06:17:34.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.219 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.219 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.219 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.219 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712938+0000", 2024-09-14T06:17:34.220 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.220 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.220 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.220 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712938+0000", 2024-09-14T06:17:34.220 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.220 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.220 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.220 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712939+0000", 2024-09-14T06:17:34.220 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1099999999999996e-07 2024-09-14T06:17:34.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.220 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.220 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.221 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712942+0000", 2024-09-14T06:17:34.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0920000000000002e-06 2024-09-14T06:17:34.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.221 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.221 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.221 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712949+0000", 2024-09-14T06:17:34.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.075e-06 2024-09-14T06:17:34.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.221 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.221 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.221 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712954+0000", 2024-09-14T06:17:34.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2229999999999998e-06 2024-09-14T06:17:34.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.222 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.222 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841192+0000", 2024-09-14T06:17:34.222 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12823817100000001 2024-09-14T06:17:34.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.222 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.222 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841280+0000", 2024-09-14T06:17:34.222 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8190999999999996e-05 2024-09-14T06:17:34.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.222 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.222 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841550+0000", 2024-09-14T06:17:34.222 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00026978899999999999 2024-09-14T06:17:34.223 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.223 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.223 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.223 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40415 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.223 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712962+0000", 2024-09-14T06:17:34.223 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.468777524, 2024-09-14T06:17:34.223 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12902585499999999, 2024-09-14T06:17:34.223 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.223 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.223 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.223 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.223 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.223 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.224 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40415 2024-09-14T06:17:34.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.224 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.224 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.224 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712962+0000", 2024-09-14T06:17:34.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.224 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.224 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712962+0000", 2024-09-14T06:17:34.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.225 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.225 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712963+0000", 2024-09-14T06:17:34.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:17:34.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.225 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.225 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.225 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712966+0000", 2024-09-14T06:17:34.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6649999999999999e-06 2024-09-14T06:17:34.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.225 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.225 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.225 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712972+0000", 2024-09-14T06:17:34.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4819999999999999e-06 2024-09-14T06:17:34.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.225 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.226 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.226 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712978+0000", 2024-09-14T06:17:34.226 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4299999999999997e-06 2024-09-14T06:17:34.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.226 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.226 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.226 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841581+0000", 2024-09-14T06:17:34.226 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12860382100000001 2024-09-14T06:17:34.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.226 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.226 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.226 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841665+0000", 2024-09-14T06:17:34.226 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3228999999999999e-05 2024-09-14T06:17:34.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.226 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.227 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841988+0000", 2024-09-14T06:17:34.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032370599999999999 2024-09-14T06:17:34.227 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.227 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.227 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.227 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40417 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.227 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712983+0000", 2024-09-14T06:17:34.227 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.46875679, 2024-09-14T06:17:34.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129525837, 2024-09-14T06:17:34.227 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.227 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.228 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.228 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.228 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.228 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.228 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40417 2024-09-14T06:17:34.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.228 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.228 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.228 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712983+0000", 2024-09-14T06:17:34.228 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.228 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.228 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712983+0000", 2024-09-14T06:17:34.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.229 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.229 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.229 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713182+0000", 2024-09-14T06:17:34.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000198569 2024-09-14T06:17:34.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.229 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.229 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.229 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713186+0000", 2024-09-14T06:17:34.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6160000000000004e-06 2024-09-14T06:17:34.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.229 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.229 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.229 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713197+0000", 2024-09-14T06:17:34.230 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.046e-05 2024-09-14T06:17:34.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.230 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.230 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713204+0000", 2024-09-14T06:17:34.230 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.515e-06 2024-09-14T06:17:34.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.230 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.230 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842021+0000", 2024-09-14T06:17:34.230 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12881625199999999 2024-09-14T06:17:34.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.230 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.230 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842158+0000", 2024-09-14T06:17:34.231 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013778099999999999 2024-09-14T06:17:34.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.231 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842509+0000", 2024-09-14T06:17:34.231 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000350644 2024-09-14T06:17:34.231 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.231 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.231 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.231 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40419 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.231 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.713217+0000", 2024-09-14T06:17:34.232 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.468523253000001, 2024-09-14T06:17:34.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12982854599999999, 2024-09-14T06:17:34.232 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.232 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.232 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.232 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.232 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.232 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.232 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40419 2024-09-14T06:17:34.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.232 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.232 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.232 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713217+0000", 2024-09-14T06:17:34.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.233 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.233 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.233 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713217+0000", 2024-09-14T06:17:34.233 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.233 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.233 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.233 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713831+0000", 2024-09-14T06:17:34.233 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061477700000000005 2024-09-14T06:17:34.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.233 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.233 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.233 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713841+0000", 2024-09-14T06:17:34.233 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3789999999999995e-06 2024-09-14T06:17:34.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.233 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.234 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.234 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713853+0000", 2024-09-14T06:17:34.234 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1952e-05 2024-09-14T06:17:34.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.234 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.234 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.234 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713861+0000", 2024-09-14T06:17:34.234 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4500000000000004e-06 2024-09-14T06:17:34.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.234 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.234 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.234 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842560+0000", 2024-09-14T06:17:34.234 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12869836900000001 2024-09-14T06:17:34.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.234 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.235 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.235 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842655+0000", 2024-09-14T06:17:34.235 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5617000000000003e-05 2024-09-14T06:17:34.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.235 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.235 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843045+0000", 2024-09-14T06:17:34.235 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00039000200000000002 2024-09-14T06:17:34.235 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.235 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.235 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.235 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40421 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.235 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.713882+0000", 2024-09-14T06:17:34.236 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.467857886000001, 2024-09-14T06:17:34.236 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12974492600000001, 2024-09-14T06:17:34.236 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.236 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.236 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.236 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.236 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.236 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.236 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40421 2024-09-14T06:17:34.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.236 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.236 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.236 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.236 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713882+0000", 2024-09-14T06:17:34.236 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.237 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.237 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713882+0000", 2024-09-14T06:17:34.237 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.237 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.237 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714044+0000", 2024-09-14T06:17:34.237 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016224999999999999 2024-09-14T06:17:34.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.237 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.237 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714049+0000", 2024-09-14T06:17:34.237 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0150000000000003e-06 2024-09-14T06:17:34.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.238 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714068+0000", 2024-09-14T06:17:34.238 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8749e-05 2024-09-14T06:17:34.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.238 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714077+0000", 2024-09-14T06:17:34.238 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4379999999999995e-06 2024-09-14T06:17:34.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.238 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843079+0000", 2024-09-14T06:17:34.239 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129002958 2024-09-14T06:17:34.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.239 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.239 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843290+0000", 2024-09-14T06:17:34.239 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021005899999999999 2024-09-14T06:17:34.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.239 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.239 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843627+0000", 2024-09-14T06:17:34.239 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00033745699999999999 2024-09-14T06:17:34.239 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.239 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.239 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.240 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40423 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.240 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.714090+0000", 2024-09-14T06:17:34.240 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.467649515000002, 2024-09-14T06:17:34.240 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129886166, 2024-09-14T06:17:34.240 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.240 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.240 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.240 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.240 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.240 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.240 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40423 2024-09-14T06:17:34.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.240 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.240 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.240 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.241 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714090+0000", 2024-09-14T06:17:34.241 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.241 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.241 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.241 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714090+0000", 2024-09-14T06:17:34.241 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.241 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.241 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.241 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714456+0000", 2024-09-14T06:17:34.241 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000366058 2024-09-14T06:17:34.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.242 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.242 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714461+0000", 2024-09-14T06:17:34.242 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.228e-06 2024-09-14T06:17:34.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.242 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.242 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714469+0000", 2024-09-14T06:17:34.242 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6589999999999996e-06 2024-09-14T06:17:34.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.242 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.242 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714476+0000", 2024-09-14T06:17:34.242 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:17:34.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843659+0000", 2024-09-14T06:17:34.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129183137 2024-09-14T06:17:34.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843745+0000", 2024-09-14T06:17:34.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5354999999999998e-05 2024-09-14T06:17:34.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843977+0000", 2024-09-14T06:17:34.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000231883 2024-09-14T06:17:34.243 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.243 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.243 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.244 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.244 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40425 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.244 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.714490+0000", 2024-09-14T06:17:34.244 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.467250320000002, 2024-09-14T06:17:34.244 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13064684600000001, 2024-09-14T06:17:34.244 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.244 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.244 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.244 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.244 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.244 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.244 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40425 2024-09-14T06:17:34.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.244 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.245 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.245 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.245 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714490+0000", 2024-09-14T06:17:34.245 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.245 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.245 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.245 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714490+0000", 2024-09-14T06:17:34.245 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.245 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.245 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.245 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714725+0000", 2024-09-14T06:17:34.245 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000235438 2024-09-14T06:17:34.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.245 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.246 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.246 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714729+0000", 2024-09-14T06:17:34.246 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8909999999999997e-06 2024-09-14T06:17:34.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.246 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.246 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.246 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714737+0000", 2024-09-14T06:17:34.246 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5329999999999994e-06 2024-09-14T06:17:34.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.246 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.246 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.246 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714742+0000", 2024-09-14T06:17:34.246 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9559999999999996e-06 2024-09-14T06:17:34.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.246 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.246 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.247 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.844184+0000", 2024-09-14T06:17:34.247 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129441954 2024-09-14T06:17:34.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.247 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.247 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.247 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.844272+0000", 2024-09-14T06:17:34.247 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7151999999999995e-05 2024-09-14T06:17:34.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.247 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.247 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.247 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845136+0000", 2024-09-14T06:17:34.247 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00086492199999999996 2024-09-14T06:17:34.247 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.247 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.247 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.248 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40426 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.248 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.714753+0000", 2024-09-14T06:17:34.248 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.466987356000001, 2024-09-14T06:17:34.248 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.130693374, 2024-09-14T06:17:34.248 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.248 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.248 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.248 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.248 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.248 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.248 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40426 2024-09-14T06:17:34.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.248 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.249 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.249 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714753+0000", 2024-09-14T06:17:34.249 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.249 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.249 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.249 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714753+0000", 2024-09-14T06:17:34.249 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.249 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.249 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.249 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715118+0000", 2024-09-14T06:17:34.249 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036512099999999999 2024-09-14T06:17:34.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.249 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.249 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.250 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715123+0000", 2024-09-14T06:17:34.250 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9910000000000002e-06 2024-09-14T06:17:34.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.250 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.250 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.250 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715141+0000", 2024-09-14T06:17:34.250 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8219999999999998e-05 2024-09-14T06:17:34.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.250 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.250 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.250 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715149+0000", 2024-09-14T06:17:34.250 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8979999999999996e-06 2024-09-14T06:17:34.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.250 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.250 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845170+0000", 2024-09-14T06:17:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13002106799999999 2024-09-14T06:17:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845249+0000", 2024-09-14T06:17:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9163000000000001e-05 2024-09-14T06:17:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845446+0000", 2024-09-14T06:17:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196913 2024-09-14T06:17:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.252 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.252 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40428 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.252 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.715166+0000", 2024-09-14T06:17:34.252 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.466574404999999, 2024-09-14T06:17:34.252 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13071775199999999, 2024-09-14T06:17:34.252 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.252 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.252 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:34.252 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.252 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.252 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.252 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40428 2024-09-14T06:17:34.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.252 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.252 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.252 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.253 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715166+0000", 2024-09-14T06:17:34.253 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.253 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.253 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715166+0000", 2024-09-14T06:17:34.253 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.253 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.253 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715361+0000", 2024-09-14T06:17:34.253 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00019536400000000001 2024-09-14T06:17:34.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.253 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.254 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715366+0000", 2024-09-14T06:17:34.254 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4589999999999997e-06 2024-09-14T06:17:34.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.254 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.254 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.254 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715387+0000", 2024-09-14T06:17:34.254 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0829e-05 2024-09-14T06:17:34.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.254 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.254 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.254 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715402+0000", 2024-09-14T06:17:34.254 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4429e-05 2024-09-14T06:17:34.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.254 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.254 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.255 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845476+0000", 2024-09-14T06:17:34.255 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13007447999999999 2024-09-14T06:17:34.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.255 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:34.255 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845557+0000", 2024-09-14T06:17:34.255 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1100000000000006e-05 2024-09-14T06:17:34.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.255 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.255 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845883+0000", 2024-09-14T06:17:34.255 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032609099999999998 2024-09-14T06:17:34.255 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.255 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.255 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.256 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.256 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40430 3.15 3.aecb2ff5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.256 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.715416+0000", 2024-09-14T06:17:34.256 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.466323939999999, 2024-09-14T06:17:34.256 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13113939699999999, 2024-09-14T06:17:34.256 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.256 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.256 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:34.256 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.256 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.256 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.256 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40430 2024-09-14T06:17:34.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.256 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.256 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.256 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.257 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715416+0000", 2024-09-14T06:17:34.257 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.257 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.257 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.257 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715416+0000", 2024-09-14T06:17:34.257 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.257 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.257 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.257 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715995+0000", 2024-09-14T06:17:34.257 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057881100000000004 2024-09-14T06:17:34.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.257 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.257 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716000+0000", 2024-09-14T06:17:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.677e-06 2024-09-14T06:17:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716013+0000", 2024-09-14T06:17:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2326e-05 2024-09-14T06:17:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716021+0000", 2024-09-14T06:17:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9750000000000007e-06 2024-09-14T06:17:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846538+0000", 2024-09-14T06:17:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13051717400000001 2024-09-14T06:17:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846555+0000", 2024-09-14T06:17:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7433999999999999e-05 2024-09-14T06:17:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40432 3.15 3.aecb2ff5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.465704300999999, 2024-09-14T06:17:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13055824499999999, 2024-09-14T06:17:34.260 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.260 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.260 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:34.260 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.260 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.260 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.260 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40432 2024-09-14T06:17:34.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.260 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.260 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.260 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.260 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:34.260 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.261 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.261 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:34.261 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.261 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.261 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:34.261 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9599999999999998e-07 2024-09-14T06:17:34.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.262 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.262 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716040+0000", 2024-09-14T06:17:34.262 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.399e-06 2024-09-14T06:17:34.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.262 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.262 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.262 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716048+0000", 2024-09-14T06:17:34.262 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2179999999999999e-06 2024-09-14T06:17:34.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.262 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.262 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.262 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716054+0000", 2024-09-14T06:17:34.262 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2110000000000003e-06 2024-09-14T06:17:34.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.262 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.263 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.263 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846586+0000", 2024-09-14T06:17:34.263 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.130531921 2024-09-14T06:17:34.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.263 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.263 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.263 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846594+0000", 2024-09-14T06:17:34.263 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7000000000000008e-06 2024-09-14T06:17:34.263 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.263 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.263 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.263 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.263 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40434 3.15 3.aecb2ff5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:34.263 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.716065+0000", 2024-09-14T06:17:34.264 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.465674474, 2024-09-14T06:17:34.264 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13055924899999999, 2024-09-14T06:17:34.264 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:34.264 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:34.264 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:34.264 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:34.264 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:34.264 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:34.264 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40434 2024-09-14T06:17:34.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.264 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:34.264 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.264 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:34.264 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716065+0000", 2024-09-14T06:17:34.264 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.265 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.265 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:34.265 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716065+0000", 2024-09-14T06:17:34.265 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:34.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.265 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.265 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:34.265 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.723984+0000", 2024-09-14T06:17:34.265 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0079189670000000007 2024-09-14T06:17:34.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.265 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.265 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:34.265 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.723991+0000", 2024-09-14T06:17:34.265 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1759999999999998e-06 2024-09-14T06:17:34.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.266 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:34.266 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.724003+0000", 2024-09-14T06:17:34.266 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1914e-05 2024-09-14T06:17:34.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.266 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:34.266 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.724013+0000", 2024-09-14T06:17:34.266 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441999999999999e-05 2024-09-14T06:17:34.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.266 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:34.266 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846617+0000", 2024-09-14T06:17:34.266 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12260452199999999 2024-09-14T06:17:34.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:34.267 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:34.267 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:34.267 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846625+0000", 2024-09-14T06:17:34.267 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2280000000000001e-06 2024-09-14T06:17:34.267 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.267 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.267 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.267 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:34.267 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:34.267 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:34.267 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:17:34.269 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:17:34.270 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T06:17:34.313 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:34.310+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:34.383 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:34.381+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:34.435 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:34.435 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:34.435 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:34.435 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:34.456 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:17:34.460 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:34.460 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:34.479 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:34.480 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:34.483 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:34.481+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:34.496 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:34.497 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:34.513 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:34.514 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:34.534 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:34.535 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:34.555 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:34.555 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:34.575 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:34.575 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:34.576 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:34.583 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:34.581+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:34.590 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:17:34.592 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:17:34.599 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:34.599 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:34.685 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:34.682+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:34.716 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:34.717 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:34.735 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:17:34.827 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:34.826+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:34.887 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:34.885+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:34.896 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:34.896 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:34.911 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:17:34.988 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:34.989+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:35.079 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:35.079 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:35.089 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:35.085+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:35.094 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:17:35.190 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:35.185+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:35.215 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T06:17:35.218 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-09-14T06:17:35.223 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:35.223 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:35.243 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:17:35.245 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T06:17:35.291 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:35.289+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:35.315 INFO:teuthology.orchestra.run.smithi039.stdout:5593 punch from 0x6542641 to 0x654ea52, (0xc411 bytes) 2024-09-14T06:17:35.315 INFO:teuthology.orchestra.run.smithi039.stdout:5594 read 0x896ae9b thru 0x896d6ab (0x2811 bytes) 2024-09-14T06:17:35.316 INFO:teuthology.orchestra.run.smithi039.stdout:5595 punch from 0x3fb41b5 to 0x3fb8148, (0x3f93 bytes) 2024-09-14T06:17:35.324 INFO:teuthology.orchestra.run.smithi039.stdout:5596 trunc from 0x990b47b to 0x43c0a5b 2024-09-14T06:17:35.349 INFO:teuthology.orchestra.run.smithi039.stdout:5597 read 0x31c50b4 thru 0x31cee33 (0x9d80 bytes) 2024-09-14T06:17:35.349 INFO:teuthology.orchestra.run.smithi039.stdout:5598 clone 31 order 25 su 1048576 sc 13 2024-09-14T06:17:35.391 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:35.389+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:35.402 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:35.402 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:35.420 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:17:35.511 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:35.510+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:35.589 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:35.589 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:35.593 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:35.590+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:35.603 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:17:35.694 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:35.690+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:35.757 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:35.757 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:35.770 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:17:35.811 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:35.810+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:35.896 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:35.893+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:35.904 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:35.904 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:35.919 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:17:35.997 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:35.994+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:36.087 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:36.087 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:36.099 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:36.097+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:36.101 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:17:36.201 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:36.197+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:36.218 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T06:17:36.246 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T06:17:36.269 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:36.269 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:36.283 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:17:36.331 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:36.330+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:36.402 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:36.398+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:36.448 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:36.448 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:36.463 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:17:36.507 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:36.505+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:36.587 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:36.587 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:36.601 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:17:36.619 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:17:36.643 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:36.641+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:36.705 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:36.701+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:36.768 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:36.769 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:36.784 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:17:36.792 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:36.792 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:36.806 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:36.802+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:36.811 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:36.811 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:36.830 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:36.830 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:36.846 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:36.846 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:36.865 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:36.865 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:36.884 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:36.884 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:36.904 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:36.905 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:36.907 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:36.905+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:36.924 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:36.925 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:36.945 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:36.945 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:36.945 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:36.945 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:36.959 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:17:37.047 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:37.045+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:37.110 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:37.106+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:37.130 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:37.130 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:37.130 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:37.130 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:37.130 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:37.144 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:17:37.211 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:37.206+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:37.230 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T06:17:37.246 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:17:37.259 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T06:17:37.320 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:37.320 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:37.320 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:37.320 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:37.320 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.320 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:3595 3.1 3:811c1c24:::rbd_data.1152bd92e04f.000000000000006e:head [set-alloc-hint object_size 2097152 write_size 2097152,write 440183~2185] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T06:17:37.320 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:37.321 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 71.095230536000003, 2024-09-14T06:17:37.321 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0365806689999999, 2024-09-14T06:17:37.321 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.321 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.321 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.321 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.321 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.321 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.321 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 3595 2024-09-14T06:17:37.321 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.321 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.322 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.322 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.322 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:37.322 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.322 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.322 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.322 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.322 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:37.322 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.322 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.322 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.322 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.322 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222823+0000", 2024-09-14T06:17:37.323 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8219999999999998e-06 2024-09-14T06:17:37.323 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.323 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.323 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.323 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222835+0000", 2024-09-14T06:17:37.323 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2052000000000001e-05 2024-09-14T06:17:37.323 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.323 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.323 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.323 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222859+0000", 2024-09-14T06:17:37.323 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3694e-05 2024-09-14T06:17:37.323 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.323 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.324 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.324 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222872+0000", 2024-09-14T06:17:37.324 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3203999999999999e-05 2024-09-14T06:17:37.324 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.324 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.324 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.324 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223068+0000", 2024-09-14T06:17:37.324 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000195995 2024-09-14T06:17:37.324 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.324 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.324 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.324 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223312+0000", 2024-09-14T06:17:37.325 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024424199999999998 2024-09-14T06:17:37.325 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.325 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.325 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:37.325 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223786+0000", 2024-09-14T06:17:37.325 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00047356600000000003 2024-09-14T06:17:37.325 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.325 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.325 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:37.325 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.225374+0000", 2024-09-14T06:17:37.325 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015875150000000001 2024-09-14T06:17:37.325 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.325 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.326 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.326 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.233122+0000", 2024-09-14T06:17:37.326 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0077480700000000001 2024-09-14T06:17:37.326 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.326 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.326 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:17:37.326 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.233157+0000", 2024-09-14T06:17:37.326 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5113000000000001e-05 2024-09-14T06:17:37.326 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.326 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.327 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.327 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.252998+0000", 2024-09-14T06:17:37.327 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.019841142 2024-09-14T06:17:37.327 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.327 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.327 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for degraded object", 2024-09-14T06:17:37.327 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.254121+0000", 2024-09-14T06:17:37.327 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0011227979999999999 2024-09-14T06:17:37.327 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.327 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.327 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.327 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.259263+0000", 2024-09-14T06:17:37.328 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0051426700000000002 2024-09-14T06:17:37.328 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.328 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.328 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.328 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.259401+0000", 2024-09-14T06:17:37.328 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013778600000000001 2024-09-14T06:17:37.328 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.328 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.328 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.328 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.328 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.328 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11421 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.329 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:37.329 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.332335350000001, 2024-09-14T06:17:37.329 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078553170000000005, 2024-09-14T06:17:37.329 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.329 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.329 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.329 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.329 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.329 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.329 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11421 2024-09-14T06:17:37.329 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.329 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.330 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.330 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.330 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:37.330 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.330 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.330 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.330 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.330 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:37.330 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.330 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.330 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.331 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.331 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985718+0000", 2024-09-14T06:17:37.331 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8840000000000001e-06 2024-09-14T06:17:37.331 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.331 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.331 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.331 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985723+0000", 2024-09-14T06:17:37.331 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8799999999999996e-06 2024-09-14T06:17:37.331 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.331 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.331 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.331 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985742+0000", 2024-09-14T06:17:37.331 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8776000000000001e-05 2024-09-14T06:17:37.332 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.332 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.332 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.332 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985759+0000", 2024-09-14T06:17:37.332 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7119999999999999e-05 2024-09-14T06:17:37.332 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.332 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.332 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.332 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064259+0000", 2024-09-14T06:17:37.332 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078499418000000001 2024-09-14T06:17:37.332 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.333 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.333 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.333 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064269+0000", 2024-09-14T06:17:37.333 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:17:37.333 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.333 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.333 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.333 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.333 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.333 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11422 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.333 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:37.333 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.332265382999999, 2024-09-14T06:17:37.334 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078521540000000001, 2024-09-14T06:17:37.334 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.334 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.334 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.334 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.334 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.334 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.334 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11422 2024-09-14T06:17:37.334 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.334 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.334 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.335 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.335 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:37.335 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.335 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.335 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.335 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.335 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:37.335 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.335 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.335 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.335 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.335 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985787+0000", 2024-09-14T06:17:37.335 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7630000000000001e-06 2024-09-14T06:17:37.336 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.336 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.336 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.336 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985842+0000", 2024-09-14T06:17:37.336 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4555999999999997e-05 2024-09-14T06:17:37.336 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.336 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.336 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.336 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985864+0000", 2024-09-14T06:17:37.336 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2189999999999999e-05 2024-09-14T06:17:37.336 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.336 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.337 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.337 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985879+0000", 2024-09-14T06:17:37.337 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4382999999999999e-05 2024-09-14T06:17:37.337 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.337 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.337 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.337 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064298+0000", 2024-09-14T06:17:37.337 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078419121999999994 2024-09-14T06:17:37.337 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.337 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.337 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.337 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064307+0000", 2024-09-14T06:17:37.338 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5259999999999999e-06 2024-09-14T06:17:37.338 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.338 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.338 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.338 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.338 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.338 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11423 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.338 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:37.338 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.332148011000001, 2024-09-14T06:17:37.338 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078437175999999997, 2024-09-14T06:17:37.338 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.338 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.338 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.339 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.339 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.339 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.339 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11423 2024-09-14T06:17:37.339 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.339 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.339 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.339 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.339 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:37.339 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.339 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.339 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.339 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.340 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:37.340 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.340 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.340 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.340 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.340 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985905+0000", 2024-09-14T06:17:37.340 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8139999999999999e-06 2024-09-14T06:17:37.340 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.340 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.340 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.340 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985911+0000", 2024-09-14T06:17:37.340 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7230000000000001e-06 2024-09-14T06:17:37.341 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.341 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.341 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.341 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985930+0000", 2024-09-14T06:17:37.341 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9360000000000001e-05 2024-09-14T06:17:37.341 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.341 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.341 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.341 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985944+0000", 2024-09-14T06:17:37.341 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4479e-05 2024-09-14T06:17:37.341 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.341 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.341 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.342 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064332+0000", 2024-09-14T06:17:37.342 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078387686999999998 2024-09-14T06:17:37.342 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.342 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.342 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.342 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064340+0000", 2024-09-14T06:17:37.342 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1130000000000001e-06 2024-09-14T06:17:37.342 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.342 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.342 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.342 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.342 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.342 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11480 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.343 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:37.343 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.330797535000002, 2024-09-14T06:17:37.343 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077122298000000006, 2024-09-14T06:17:37.343 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.343 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.343 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.343 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.343 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.343 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.343 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11480 2024-09-14T06:17:37.343 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.343 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.344 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.344 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.344 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:37.344 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.344 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.344 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.344 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.344 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:37.344 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.344 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.344 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.344 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.345 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987255+0000", 2024-09-14T06:17:37.345 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:17:37.345 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.345 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.345 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.345 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987263+0000", 2024-09-14T06:17:37.345 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6499999999999996e-06 2024-09-14T06:17:37.345 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.345 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.345 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.345 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987285+0000", 2024-09-14T06:17:37.345 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1860000000000001e-05 2024-09-14T06:17:37.346 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.346 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.346 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.346 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987341+0000", 2024-09-14T06:17:37.346 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6648999999999999e-05 2024-09-14T06:17:37.346 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.346 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.346 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.346 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064367+0000", 2024-09-14T06:17:37.346 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077025622000000002 2024-09-14T06:17:37.346 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.346 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.347 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.347 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064376+0000", 2024-09-14T06:17:37.347 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7499999999999992e-06 2024-09-14T06:17:37.347 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.347 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.347 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.347 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.347 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.347 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11481 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.347 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:37.347 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.330661833000001, 2024-09-14T06:17:37.347 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077022057000000005, 2024-09-14T06:17:37.348 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.348 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.348 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.348 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.348 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.348 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.348 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11481 2024-09-14T06:17:37.348 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.348 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.348 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.348 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.348 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:37.349 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.349 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.349 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.349 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.349 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:37.349 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.349 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.349 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.349 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.350 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987391+0000", 2024-09-14T06:17:37.350 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4839999999999999e-06 2024-09-14T06:17:37.350 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.350 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.350 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.350 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987399+0000", 2024-09-14T06:17:37.350 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8660000000000006e-06 2024-09-14T06:17:37.350 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.350 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.350 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.351 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987419+0000", 2024-09-14T06:17:37.351 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9984e-05 2024-09-14T06:17:37.351 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.351 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.351 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.351 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987434+0000", 2024-09-14T06:17:37.351 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5396e-05 2024-09-14T06:17:37.351 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.351 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.351 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.351 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064403+0000", 2024-09-14T06:17:37.351 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076969212999999995 2024-09-14T06:17:37.352 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.352 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.352 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.352 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064411+0000", 2024-09-14T06:17:37.352 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1140000000000006e-06 2024-09-14T06:17:37.352 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.352 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.352 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.352 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.352 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.352 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11482 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.352 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:37.353 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.330576972999999, 2024-09-14T06:17:37.353 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076971172000000004, 2024-09-14T06:17:37.353 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.353 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.353 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.353 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.353 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.353 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.353 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11482 2024-09-14T06:17:37.353 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.353 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.353 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.354 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.354 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:37.354 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.354 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.354 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.354 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.354 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:37.354 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.354 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.354 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.354 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.354 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987476+0000", 2024-09-14T06:17:37.354 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.793e-06 2024-09-14T06:17:37.355 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.355 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.355 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.355 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987482+0000", 2024-09-14T06:17:37.355 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8270000000000003e-06 2024-09-14T06:17:37.355 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.355 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.355 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.355 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987502+0000", 2024-09-14T06:17:37.355 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0656e-05 2024-09-14T06:17:37.355 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.355 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.355 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.355 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987518+0000", 2024-09-14T06:17:37.356 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:17:37.356 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.356 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.356 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.356 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064438+0000", 2024-09-14T06:17:37.356 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076919623000000006 2024-09-14T06:17:37.356 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.356 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.356 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.356 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064445+0000", 2024-09-14T06:17:37.357 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.61e-06 2024-09-14T06:17:37.357 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.357 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.357 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.357 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.357 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.357 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11483 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.357 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:37.357 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.330509124000002, 2024-09-14T06:17:37.357 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076936562999999999, 2024-09-14T06:17:37.357 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.357 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.358 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.358 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.358 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.358 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.358 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11483 2024-09-14T06:17:37.358 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.358 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.358 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.358 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.358 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:37.358 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.358 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.359 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.359 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.359 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:37.359 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.359 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.359 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.359 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.359 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987544+0000", 2024-09-14T06:17:37.359 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.654e-06 2024-09-14T06:17:37.359 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.359 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.359 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.360 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987550+0000", 2024-09-14T06:17:37.360 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0299999999999999e-06 2024-09-14T06:17:37.360 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.360 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.360 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.360 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987570+0000", 2024-09-14T06:17:37.360 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0080000000000001e-05 2024-09-14T06:17:37.360 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.360 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.360 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.360 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987619+0000", 2024-09-14T06:17:37.360 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9228999999999999e-05 2024-09-14T06:17:37.361 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.361 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.361 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.361 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064471+0000", 2024-09-14T06:17:37.361 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076852086999999999 2024-09-14T06:17:37.361 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.361 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.361 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.361 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064478+0000", 2024-09-14T06:17:37.361 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4830000000000002e-06 2024-09-14T06:17:37.361 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.362 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.362 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.362 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.362 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.362 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11485 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.362 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:37.362 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.330410630999999, 2024-09-14T06:17:37.362 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076868463999999997, 2024-09-14T06:17:37.362 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.362 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.362 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.362 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.362 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.363 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.363 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11485 2024-09-14T06:17:37.363 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.363 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.363 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.363 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.363 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:37.363 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.363 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.363 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.363 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.363 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:37.363 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.364 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.364 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.364 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.364 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987642+0000", 2024-09-14T06:17:37.364 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4100000000000001e-06 2024-09-14T06:17:37.364 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.364 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.365 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.365 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987648+0000", 2024-09-14T06:17:37.365 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2500000000000003e-06 2024-09-14T06:17:37.365 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.365 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.365 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.365 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987683+0000", 2024-09-14T06:17:37.365 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5193e-05 2024-09-14T06:17:37.365 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.365 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.365 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.365 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987698+0000", 2024-09-14T06:17:37.365 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4893e-05 2024-09-14T06:17:37.366 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.366 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.366 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.366 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064501+0000", 2024-09-14T06:17:37.366 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076803263999999996 2024-09-14T06:17:37.366 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.366 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.366 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.366 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064509+0000", 2024-09-14T06:17:37.366 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4540000000000001e-06 2024-09-14T06:17:37.366 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.366 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.366 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.367 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.367 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.367 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11490 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.367 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:37.367 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.330261061999998, 2024-09-14T06:17:37.367 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076755043999999994, 2024-09-14T06:17:37.367 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.367 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.367 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.367 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.367 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.368 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.368 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11490 2024-09-14T06:17:37.368 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.368 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.368 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.368 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.368 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:37.368 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.368 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.368 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.368 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.368 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:37.369 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.369 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.369 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.369 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.369 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987791+0000", 2024-09-14T06:17:37.369 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:17:37.369 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.369 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.369 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.369 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987797+0000", 2024-09-14T06:17:37.369 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7969999999999997e-06 2024-09-14T06:17:37.370 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.370 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.370 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.370 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987817+0000", 2024-09-14T06:17:37.370 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9746e-05 2024-09-14T06:17:37.370 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.370 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.370 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.370 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987832+0000", 2024-09-14T06:17:37.370 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.452e-05 2024-09-14T06:17:37.370 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.371 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.371 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.371 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064535+0000", 2024-09-14T06:17:37.371 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076703478000000005 2024-09-14T06:17:37.371 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.371 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.371 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.371 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064545+0000", 2024-09-14T06:17:37.371 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9799999999999993e-06 2024-09-14T06:17:37.371 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.371 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.371 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.372 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.372 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.372 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11492 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.372 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:37.372 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.330134197, 2024-09-14T06:17:37.372 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076661759999999995, 2024-09-14T06:17:37.372 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.372 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.372 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.372 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.372 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.373 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.373 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11492 2024-09-14T06:17:37.373 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.373 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.373 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.373 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.373 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:37.373 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.373 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.373 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.373 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.373 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:37.374 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.374 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.374 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.374 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.374 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987918+0000", 2024-09-14T06:17:37.374 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3239999999999999e-06 2024-09-14T06:17:37.374 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.374 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.374 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.374 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987924+0000", 2024-09-14T06:17:37.374 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6930000000000004e-06 2024-09-14T06:17:37.374 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.375 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.375 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.375 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987943+0000", 2024-09-14T06:17:37.375 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9487000000000002e-05 2024-09-14T06:17:37.375 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.375 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.375 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.375 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987958+0000", 2024-09-14T06:17:37.375 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4453e-05 2024-09-14T06:17:37.375 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.375 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.376 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.376 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064571+0000", 2024-09-14T06:17:37.376 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076612988000000007 2024-09-14T06:17:37.376 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.376 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.376 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.376 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064579+0000", 2024-09-14T06:17:37.376 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8150000000000006e-06 2024-09-14T06:17:37.376 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.376 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.376 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.376 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.377 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.377 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11495 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.377 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:37.377 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.329965508000001, 2024-09-14T06:17:37.377 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076524746000000005, 2024-09-14T06:17:37.377 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.377 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.377 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.377 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.377 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.377 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.378 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11495 2024-09-14T06:17:37.378 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.378 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.378 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.378 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.378 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:37.378 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.378 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.378 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.378 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.378 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:37.378 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.379 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.379 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.379 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.379 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988087+0000", 2024-09-14T06:17:37.379 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0499999999999999e-06 2024-09-14T06:17:37.379 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.379 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.379 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.379 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988118+0000", 2024-09-14T06:17:37.379 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1900000000000003e-05 2024-09-14T06:17:37.379 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.379 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.380 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.380 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988138+0000", 2024-09-14T06:17:37.380 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.933e-05 2024-09-14T06:17:37.380 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.380 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.380 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.380 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988152+0000", 2024-09-14T06:17:37.380 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3889e-05 2024-09-14T06:17:37.380 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.380 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.380 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.381 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064603+0000", 2024-09-14T06:17:37.381 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076451082000000004 2024-09-14T06:17:37.381 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.381 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.381 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.381 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064610+0000", 2024-09-14T06:17:37.381 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4950000000000002e-06 2024-09-14T06:17:37.381 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.381 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.381 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.381 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.381 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.382 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11498 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.382 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:37.382 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.329879431999998, 2024-09-14T06:17:37.382 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076471883000000004, 2024-09-14T06:17:37.382 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.382 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.382 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.382 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.382 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.382 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.382 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11498 2024-09-14T06:17:37.383 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.383 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.383 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.383 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.383 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:37.383 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.383 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.383 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.383 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.383 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:37.383 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.383 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.384 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988173+0000", 2024-09-14T06:17:37.384 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2300000000000001e-06 2024-09-14T06:17:37.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.384 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988178+0000", 2024-09-14T06:17:37.384 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4229999999999996e-06 2024-09-14T06:17:37.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.385 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988235+0000", 2024-09-14T06:17:37.385 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6616999999999997e-05 2024-09-14T06:17:37.385 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.385 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.385 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.385 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988264+0000", 2024-09-14T06:17:37.385 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9189000000000001e-05 2024-09-14T06:17:37.385 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.385 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.385 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.385 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064636+0000", 2024-09-14T06:17:37.385 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076371573999999998 2024-09-14T06:17:37.386 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.386 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.386 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.386 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064643+0000", 2024-09-14T06:17:37.386 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8499999999999994e-06 2024-09-14T06:17:37.386 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.386 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.386 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.386 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.386 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.386 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11523 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.386 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:37.386 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.329159283999999, 2024-09-14T06:17:37.387 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076062454000000002, 2024-09-14T06:17:37.387 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.387 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.387 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.387 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.387 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.387 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.387 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11523 2024-09-14T06:17:37.387 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.387 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.387 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.387 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.388 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:37.388 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.388 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.388 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.388 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:37.388 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.388 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.388 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.388 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988894+0000", 2024-09-14T06:17:37.388 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.843e-06 2024-09-14T06:17:37.389 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.389 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.389 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.389 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988903+0000", 2024-09-14T06:17:37.389 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0440000000000001e-06 2024-09-14T06:17:37.389 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.389 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.389 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.389 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988929+0000", 2024-09-14T06:17:37.389 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6679e-05 2024-09-14T06:17:37.389 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.389 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.390 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988967+0000", 2024-09-14T06:17:37.390 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7453000000000002e-05 2024-09-14T06:17:37.390 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.390 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.390 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064668+0000", 2024-09-14T06:17:37.390 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075701389999999993 2024-09-14T06:17:37.390 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.390 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.391 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064954+0000", 2024-09-14T06:17:37.391 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028604500000000002 2024-09-14T06:17:37.391 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.391 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.391 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.391 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.391 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.391 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11530 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.391 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:37.391 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.328668733999997, 2024-09-14T06:17:37.391 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075620674999999998, 2024-09-14T06:17:37.391 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.391 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.392 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.392 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.392 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.392 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.392 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11530 2024-09-14T06:17:37.392 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.392 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.392 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.392 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.392 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:37.392 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.392 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.393 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.393 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:37.393 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.393 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.393 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989384+0000", 2024-09-14T06:17:37.393 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6160000000000001e-06 2024-09-14T06:17:37.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.393 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.394 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989390+0000", 2024-09-14T06:17:37.394 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7039999999999999e-06 2024-09-14T06:17:37.394 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.394 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.394 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.394 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989409+0000", 2024-09-14T06:17:37.394 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.95e-05 2024-09-14T06:17:37.394 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.394 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.394 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.394 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989429+0000", 2024-09-14T06:17:37.394 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0173000000000002e-05 2024-09-14T06:17:37.395 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.395 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.395 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.395 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064992+0000", 2024-09-14T06:17:37.395 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075562764000000004 2024-09-14T06:17:37.395 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.395 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.395 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.395 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065003+0000", 2024-09-14T06:17:37.395 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0917999999999999e-05 2024-09-14T06:17:37.395 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.395 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.396 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.396 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.396 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11532 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.396 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:37.396 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.328598337999999, 2024-09-14T06:17:37.396 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075604656000000006, 2024-09-14T06:17:37.396 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.396 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.396 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.396 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.397 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.397 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.397 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11532 2024-09-14T06:17:37.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.397 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.397 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.397 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:37.397 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.398 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.398 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:37.398 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.398 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.398 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.398 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.398 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989454+0000", 2024-09-14T06:17:37.398 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6500000000000001e-06 2024-09-14T06:17:37.398 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.398 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.398 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.399 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989460+0000", 2024-09-14T06:17:37.399 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7459999999999997e-06 2024-09-14T06:17:37.399 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.399 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.399 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.399 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989480+0000", 2024-09-14T06:17:37.399 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.948e-05 2024-09-14T06:17:37.399 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.399 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.399 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.399 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989497+0000", 2024-09-14T06:17:37.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7153e-05 2024-09-14T06:17:37.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.400 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.400 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.400 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065046+0000", 2024-09-14T06:17:37.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075548905999999999 2024-09-14T06:17:37.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.400 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.400 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.400 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065057+0000", 2024-09-14T06:17:37.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1721e-05 2024-09-14T06:17:37.401 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.401 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.401 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.401 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.401 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.401 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11534 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.401 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:37.401 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.328531851999998, 2024-09-14T06:17:37.401 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075574350999999998, 2024-09-14T06:17:37.401 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.401 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.402 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.402 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.402 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.402 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.402 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11534 2024-09-14T06:17:37.402 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.402 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.402 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.402 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.402 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:37.402 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.402 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.403 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.403 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.403 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:37.403 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.403 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.403 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.403 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989521+0000", 2024-09-14T06:17:37.403 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.533e-06 2024-09-14T06:17:37.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.403 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.404 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.404 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989527+0000", 2024-09-14T06:17:37.404 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0700000000000003e-06 2024-09-14T06:17:37.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.404 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.404 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989546+0000", 2024-09-14T06:17:37.404 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.948e-05 2024-09-14T06:17:37.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.405 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.405 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989561+0000", 2024-09-14T06:17:37.405 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4827000000000001e-05 2024-09-14T06:17:37.405 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.405 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.405 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.405 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065086+0000", 2024-09-14T06:17:37.405 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075524542 2024-09-14T06:17:37.405 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.405 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.405 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.405 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065094+0000", 2024-09-14T06:17:37.406 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8990000000000001e-06 2024-09-14T06:17:37.406 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.406 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.406 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.406 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.406 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.406 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11535 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.406 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:37.406 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.328464488999998, 2024-09-14T06:17:37.406 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075542466000000003, 2024-09-14T06:17:37.406 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.406 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.407 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.407 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.407 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.407 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.407 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11535 2024-09-14T06:17:37.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.407 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.407 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.407 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.407 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:37.407 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.407 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.408 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.408 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:37.408 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.408 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.408 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.408 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.408 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989588+0000", 2024-09-14T06:17:37.408 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1659999999999999e-06 2024-09-14T06:17:37.408 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.408 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.408 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.409 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989593+0000", 2024-09-14T06:17:37.409 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6230000000000002e-06 2024-09-14T06:17:37.409 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.409 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.409 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.409 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989613+0000", 2024-09-14T06:17:37.409 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9604000000000001e-05 2024-09-14T06:17:37.409 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.409 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.409 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.409 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989643+0000", 2024-09-14T06:17:37.409 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0450000000000002e-05 2024-09-14T06:17:37.409 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.410 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.410 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.410 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065119+0000", 2024-09-14T06:17:37.410 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075475930999999996 2024-09-14T06:17:37.410 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.410 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.410 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.410 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065129+0000", 2024-09-14T06:17:37.410 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6919999999999997e-06 2024-09-14T06:17:37.410 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.410 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.410 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.411 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.411 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.411 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11537 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.411 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:37.411 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.328386565999999, 2024-09-14T06:17:37.411 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075497868999999995, 2024-09-14T06:17:37.411 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.411 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.411 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.411 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.411 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.411 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.412 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11537 2024-09-14T06:17:37.412 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.412 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.412 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.412 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.412 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:37.412 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.412 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.412 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.412 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.412 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:37.412 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.413 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.413 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.413 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.413 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989666+0000", 2024-09-14T06:17:37.413 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.48e-06 2024-09-14T06:17:37.413 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.413 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.413 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.413 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989672+0000", 2024-09-14T06:17:37.413 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7960000000000001e-06 2024-09-14T06:17:37.413 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.413 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.414 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.414 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989690+0000", 2024-09-14T06:17:37.414 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8743e-05 2024-09-14T06:17:37.414 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.414 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.414 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.414 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989704+0000", 2024-09-14T06:17:37.414 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3913e-05 2024-09-14T06:17:37.414 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.414 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.414 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.414 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065155+0000", 2024-09-14T06:17:37.415 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075450351999999998 2024-09-14T06:17:37.415 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.415 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.415 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.415 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065162+0000", 2024-09-14T06:17:37.415 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5850000000000002e-06 2024-09-14T06:17:37.415 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.415 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.415 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.415 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.415 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.415 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11543 3.12 3.4d02a8b2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.415 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:37.415 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.328109001999998, 2024-09-14T06:17:37.416 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074335152000000002, 2024-09-14T06:17:37.416 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.416 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.416 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:37.416 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.416 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.416 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.416 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11543 2024-09-14T06:17:37.416 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.416 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.416 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.416 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.417 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:37.417 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.417 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.417 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.417 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:37.417 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.417 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.417 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.417 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989944+0000", 2024-09-14T06:17:37.417 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.694e-06 2024-09-14T06:17:37.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.418 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.418 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.418 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989950+0000", 2024-09-14T06:17:37.418 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3389999999999998e-06 2024-09-14T06:17:37.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.418 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.418 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.418 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989969+0000", 2024-09-14T06:17:37.418 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8963000000000001e-05 2024-09-14T06:17:37.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.418 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.419 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.419 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989985+0000", 2024-09-14T06:17:37.419 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5586999999999998e-05 2024-09-14T06:17:37.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.419 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.419 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.419 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064270+0000", 2024-09-14T06:17:37.419 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074285552000000005 2024-09-14T06:17:37.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.419 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.419 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.419 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064277+0000", 2024-09-14T06:17:37.420 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0169999999999999e-06 2024-09-14T06:17:37.420 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.420 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.420 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.420 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.420 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.420 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:37.421 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:37.346+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:37.422 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:17:37.422 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:37.417+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:37.522 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:37.518+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:37.597 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:37.597 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:37.597 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:37.597 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:37.611 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:17:37.623 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:37.618+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:37.724 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:37.722+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:37.734 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:37.734 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:37.734 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:37.734 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:37.734 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:37.748 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:17:37.825 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:37.822+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:37.869 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:37.870 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:37.870 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:37.870 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:37.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.870 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12584 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.870 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.039784+0000", 2024-09-14T06:17:37.870 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.828039889999999, 2024-09-14T06:17:37.870 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.024433808000000001, 2024-09-14T06:17:37.870 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.870 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.870 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.870 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.871 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.871 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.871 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12584 2024-09-14T06:17:37.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.871 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.871 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.871 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.871 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039784+0000", 2024-09-14T06:17:37.871 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.871 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.871 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.871 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039784+0000", 2024-09-14T06:17:37.871 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.872 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.872 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.872 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.872 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039784+0000", 2024-09-14T06:17:37.872 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1999999999999999e-07 2024-09-14T06:17:37.872 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.872 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.872 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.872 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039788+0000", 2024-09-14T06:17:37.872 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.41e-06 2024-09-14T06:17:37.872 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.872 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.872 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.873 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039797+0000", 2024-09-14T06:17:37.873 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7969999999999995e-06 2024-09-14T06:17:37.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.873 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.873 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.873 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039804+0000", 2024-09-14T06:17:37.873 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7870000000000004e-06 2024-09-14T06:17:37.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.873 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.873 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.873 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.048222+0000", 2024-09-14T06:17:37.873 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0084172770000000008 2024-09-14T06:17:37.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.873 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.048334+0000", 2024-09-14T06:17:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000112895 2024-09-14T06:17:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064217+0000", 2024-09-14T06:17:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015882922000000001 2024-09-14T06:17:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.874 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12661 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.040519+0000", 2024-09-14T06:17:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.82730411, 2024-09-14T06:17:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014404233000000001, 2024-09-14T06:17:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12661 2024-09-14T06:17:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.875 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.876 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040519+0000", 2024-09-14T06:17:37.876 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.876 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.876 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.876 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040519+0000", 2024-09-14T06:17:37.876 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.876 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.876 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.876 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040520+0000", 2024-09-14T06:17:37.876 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6899999999999996e-07 2024-09-14T06:17:37.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.877 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.877 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.877 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040530+0000", 2024-09-14T06:17:37.877 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6409999999999997e-06 2024-09-14T06:17:37.877 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.877 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.877 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.877 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040540+0000", 2024-09-14T06:17:37.877 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0247e-05 2024-09-14T06:17:37.877 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.877 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.877 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.877 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040547+0000", 2024-09-14T06:17:37.878 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3830000000000003e-06 2024-09-14T06:17:37.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.878 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.878 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.878 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.054677+0000", 2024-09-14T06:17:37.878 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014130095000000001 2024-09-14T06:17:37.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.878 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.878 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.878 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.054786+0000", 2024-09-14T06:17:37.878 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000108441 2024-09-14T06:17:37.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.878 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.878 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.879 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.054924+0000", 2024-09-14T06:17:37.879 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013765699999999999 2024-09-14T06:17:37.879 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.879 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.879 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.879 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12662 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.879 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.040559+0000", 2024-09-14T06:17:37.879 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.827264773000003, 2024-09-14T06:17:37.879 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014659604999999999, 2024-09-14T06:17:37.879 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.879 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.879 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.880 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.880 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.880 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.880 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12662 2024-09-14T06:17:37.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.880 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.880 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.880 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040559+0000", 2024-09-14T06:17:37.880 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.880 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.880 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040559+0000", 2024-09-14T06:17:37.881 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.881 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.881 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.881 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040559+0000", 2024-09-14T06:17:37.881 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5799999999999998e-07 2024-09-14T06:17:37.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.881 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.881 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.881 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040562+0000", 2024-09-14T06:17:37.881 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9349999999999999e-06 2024-09-14T06:17:37.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.881 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.882 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.882 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040572+0000", 2024-09-14T06:17:37.882 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9480000000000003e-06 2024-09-14T06:17:37.882 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.882 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.882 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.882 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040579+0000", 2024-09-14T06:17:37.882 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1459999999999999e-06 2024-09-14T06:17:37.882 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.882 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.882 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.882 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.054963+0000", 2024-09-14T06:17:37.882 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014383986 2024-09-14T06:17:37.882 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.883 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.883 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.883 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055084+0000", 2024-09-14T06:17:37.883 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000120822 2024-09-14T06:17:37.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.883 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.883 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.883 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055218+0000", 2024-09-14T06:17:37.883 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013401 2024-09-14T06:17:37.883 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.883 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.883 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.884 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.884 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12666 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.884 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.040590+0000", 2024-09-14T06:17:37.884 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.827233624999998, 2024-09-14T06:17:37.884 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014895584999999999, 2024-09-14T06:17:37.884 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.884 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.884 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.884 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.884 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.884 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.884 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12666 2024-09-14T06:17:37.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.885 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040590+0000", 2024-09-14T06:17:37.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040590+0000", 2024-09-14T06:17:37.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040590+0000", 2024-09-14T06:17:37.886 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:17:37.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.886 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.886 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040593+0000", 2024-09-14T06:17:37.886 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0189999999999998e-06 2024-09-14T06:17:37.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.886 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.886 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040603+0000", 2024-09-14T06:17:37.886 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2820000000000002e-06 2024-09-14T06:17:37.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.887 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.887 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040610+0000", 2024-09-14T06:17:37.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1219999999999997e-06 2024-09-14T06:17:37.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.887 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.887 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.887 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055257+0000", 2024-09-14T06:17:37.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014646762000000001 2024-09-14T06:17:37.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.887 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.887 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.887 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055379+0000", 2024-09-14T06:17:37.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012186899999999999 2024-09-14T06:17:37.888 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.888 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.888 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.888 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055485+0000", 2024-09-14T06:17:37.888 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000106848 2024-09-14T06:17:37.888 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.888 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.888 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.888 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.888 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.888 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12672 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.888 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.040620+0000", 2024-09-14T06:17:37.888 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.827203142000002, 2024-09-14T06:17:37.888 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015140836, 2024-09-14T06:17:37.889 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.889 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.889 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.889 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.889 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.889 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.889 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12672 2024-09-14T06:17:37.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.889 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.889 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.889 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.889 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040620+0000", 2024-09-14T06:17:37.889 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.890 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.890 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.890 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.890 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040620+0000", 2024-09-14T06:17:37.890 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.890 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.890 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.890 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.890 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040621+0000", 2024-09-14T06:17:37.890 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9699999999999995e-07 2024-09-14T06:17:37.890 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.890 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.890 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.891 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040632+0000", 2024-09-14T06:17:37.891 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1012999999999999e-05 2024-09-14T06:17:37.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.891 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.891 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.891 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040642+0000", 2024-09-14T06:17:37.891 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0054999999999999e-05 2024-09-14T06:17:37.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.891 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.891 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.891 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040649+0000", 2024-09-14T06:17:37.891 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3769999999999998e-06 2024-09-14T06:17:37.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.892 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.892 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.892 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055521+0000", 2024-09-14T06:17:37.892 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014871383 2024-09-14T06:17:37.892 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.892 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.892 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.892 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055634+0000", 2024-09-14T06:17:37.892 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000112757 2024-09-14T06:17:37.892 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.892 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.892 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.892 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055761+0000", 2024-09-14T06:17:37.892 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012755399999999999 2024-09-14T06:17:37.893 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.893 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.893 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.893 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.893 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.893 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12673 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 385024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.893 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.041758+0000", 2024-09-14T06:17:37.893 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.826064928000001, 2024-09-14T06:17:37.893 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014243184000000001, 2024-09-14T06:17:37.893 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.893 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.893 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.893 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.893 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.893 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.894 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12673 2024-09-14T06:17:37.894 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.894 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.894 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.894 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.894 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041758+0000", 2024-09-14T06:17:37.894 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.894 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.894 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.894 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.894 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041758+0000", 2024-09-14T06:17:37.894 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.894 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.894 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.894 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.895 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041760+0000", 2024-09-14T06:17:37.895 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0860000000000001e-06 2024-09-14T06:17:37.895 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.895 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.895 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.895 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041766+0000", 2024-09-14T06:17:37.895 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-06 2024-09-14T06:17:37.895 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.895 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.895 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.895 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041794+0000", 2024-09-14T06:17:37.895 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7993000000000001e-05 2024-09-14T06:17:37.895 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.895 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.895 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.896 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041813+0000", 2024-09-14T06:17:37.896 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.823e-05 2024-09-14T06:17:37.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.896 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.896 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.896 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055797+0000", 2024-09-14T06:17:37.896 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013984612 2024-09-14T06:17:37.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.896 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.896 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.896 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055900+0000", 2024-09-14T06:17:37.896 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000102464 2024-09-14T06:17:37.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.896 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.896 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.897 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.056002+0000", 2024-09-14T06:17:37.897 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000101869 2024-09-14T06:17:37.897 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.897 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.897 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.897 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.897 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12674 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.897 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.041833+0000", 2024-09-14T06:17:37.897 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.825990398999998, 2024-09-14T06:17:37.897 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022511527, 2024-09-14T06:17:37.897 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.897 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.897 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.898 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.898 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.898 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.898 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12674 2024-09-14T06:17:37.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.898 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.898 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.898 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.898 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041833+0000", 2024-09-14T06:17:37.898 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.898 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.898 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.898 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041833+0000", 2024-09-14T06:17:37.898 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.899 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.899 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.899 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041834+0000", 2024-09-14T06:17:37.899 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7999999999999995e-07 2024-09-14T06:17:37.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.899 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.899 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.899 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041869+0000", 2024-09-14T06:17:37.899 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5766000000000003e-05 2024-09-14T06:17:37.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.899 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.899 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.899 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041922+0000", 2024-09-14T06:17:37.899 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2086000000000002e-05 2024-09-14T06:17:37.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.899 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.900 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.900 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041931+0000", 2024-09-14T06:17:37.900 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6160000000000008e-06 2024-09-14T06:17:37.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.900 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.900 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.900 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.056038+0000", 2024-09-14T06:17:37.900 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01410729 2024-09-14T06:17:37.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.900 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.900 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.900 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.056143+0000", 2024-09-14T06:17:37.900 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000104697 2024-09-14T06:17:37.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.901 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.901 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064345+0000", 2024-09-14T06:17:37.901 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0082013920000000001 2024-09-14T06:17:37.901 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.901 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.901 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.901 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12677 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.901 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.041952+0000", 2024-09-14T06:17:37.901 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.825870981999998, 2024-09-14T06:17:37.901 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022599910000000001, 2024-09-14T06:17:37.901 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.902 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.902 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.902 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.902 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.902 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.902 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12677 2024-09-14T06:17:37.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.902 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.902 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.902 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.902 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041952+0000", 2024-09-14T06:17:37.902 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.902 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.902 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041952+0000", 2024-09-14T06:17:37.903 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.903 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041953+0000", 2024-09-14T06:17:37.903 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1100000000000005e-07 2024-09-14T06:17:37.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.903 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041958+0000", 2024-09-14T06:17:37.903 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3839999999999999e-06 2024-09-14T06:17:37.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.904 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.904 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042800+0000", 2024-09-14T06:17:37.904 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00084272399999999999 2024-09-14T06:17:37.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.904 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.904 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.904 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042817+0000", 2024-09-14T06:17:37.904 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6507e-05 2024-09-14T06:17:37.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.904 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.904 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.904 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064375+0000", 2024-09-14T06:17:37.905 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021558165000000001 2024-09-14T06:17:37.905 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.905 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.905 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.905 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064458+0000", 2024-09-14T06:17:37.905 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3205e-05 2024-09-14T06:17:37.905 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.905 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.905 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.905 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064552+0000", 2024-09-14T06:17:37.905 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.4114e-05 2024-09-14T06:17:37.905 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.905 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.905 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.906 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.906 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.906 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12678 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.906 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.042838+0000", 2024-09-14T06:17:37.906 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.824985353999999, 2024-09-14T06:17:37.906 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021916718000000002, 2024-09-14T06:17:37.906 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.906 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.906 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.906 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.906 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.906 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.906 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12678 2024-09-14T06:17:37.906 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.906 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.907 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.907 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.907 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042838+0000", 2024-09-14T06:17:37.907 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.907 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.907 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.907 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.907 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042838+0000", 2024-09-14T06:17:37.907 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.907 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.907 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.907 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.907 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042839+0000", 2024-09-14T06:17:37.907 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1510000000000001e-06 2024-09-14T06:17:37.907 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.908 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.908 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.908 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042845+0000", 2024-09-14T06:17:37.908 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9190000000000004e-06 2024-09-14T06:17:37.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.908 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.908 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.908 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042859+0000", 2024-09-14T06:17:37.908 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4022000000000001e-05 2024-09-14T06:17:37.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.908 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.908 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.908 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042867+0000", 2024-09-14T06:17:37.908 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9549999999999992e-06 2024-09-14T06:17:37.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.909 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064581+0000", 2024-09-14T06:17:37.909 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021713542999999998 2024-09-14T06:17:37.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.909 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064656+0000", 2024-09-14T06:17:37.909 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5564999999999999e-05 2024-09-14T06:17:37.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.909 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064755+0000", 2024-09-14T06:17:37.909 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.8562999999999998e-05 2024-09-14T06:17:37.909 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.910 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.910 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.910 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12687 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.910 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.042878+0000", 2024-09-14T06:17:37.910 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.824945560000003, 2024-09-14T06:17:37.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022072561000000001, 2024-09-14T06:17:37.910 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.910 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.910 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.910 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.910 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.910 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.910 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12687 2024-09-14T06:17:37.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.911 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.911 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.911 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.911 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042878+0000", 2024-09-14T06:17:37.911 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.911 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.911 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.911 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042878+0000", 2024-09-14T06:17:37.911 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.911 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.911 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.911 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042879+0000", 2024-09-14T06:17:37.911 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9999999999999997e-07 2024-09-14T06:17:37.912 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.912 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.912 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.912 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042882+0000", 2024-09-14T06:17:37.912 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.608e-06 2024-09-14T06:17:37.912 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.912 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.912 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.912 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042892+0000", 2024-09-14T06:17:37.912 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9729999999999992e-06 2024-09-14T06:17:37.912 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.912 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.912 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.913 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042911+0000", 2024-09-14T06:17:37.913 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9369999999999999e-05 2024-09-14T06:17:37.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.913 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.913 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.913 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064785+0000", 2024-09-14T06:17:37.913 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021873182000000001 2024-09-14T06:17:37.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.913 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.913 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.913 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064860+0000", 2024-09-14T06:17:37.913 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5403999999999998e-05 2024-09-14T06:17:37.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064950+0000", 2024-09-14T06:17:37.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0323999999999998e-05 2024-09-14T06:17:37.914 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.914 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.914 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.914 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12688 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.914 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.042922+0000", 2024-09-14T06:17:37.914 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.824901236999999, 2024-09-14T06:17:37.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022202118, 2024-09-14T06:17:37.914 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.914 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.915 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.915 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.915 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.915 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.915 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12688 2024-09-14T06:17:37.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.915 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.915 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.915 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042922+0000", 2024-09-14T06:17:37.915 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.915 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042922+0000", 2024-09-14T06:17:37.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042923+0000", 2024-09-14T06:17:37.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:17:37.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042937+0000", 2024-09-14T06:17:37.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4337e-05 2024-09-14T06:17:37.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042948+0000", 2024-09-14T06:17:37.917 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0569e-05 2024-09-14T06:17:37.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.917 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.917 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.917 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042955+0000", 2024-09-14T06:17:37.917 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3250000000000002e-06 2024-09-14T06:17:37.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.917 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.917 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.917 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064981+0000", 2024-09-14T06:17:37.917 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022026046000000001 2024-09-14T06:17:37.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.917 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.917 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.917 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065055+0000", 2024-09-14T06:17:37.918 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4016000000000004e-05 2024-09-14T06:17:37.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.918 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.918 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.918 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065124+0000", 2024-09-14T06:17:37.918 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9141999999999996e-05 2024-09-14T06:17:37.918 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.918 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.918 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.918 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.918 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12690 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.918 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044605+0000", 2024-09-14T06:17:37.918 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.823218197000003, 2024-09-14T06:17:37.918 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020723715, 2024-09-14T06:17:37.918 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.919 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.919 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.919 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.919 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.919 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.919 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12690 2024-09-14T06:17:37.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.919 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.919 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.919 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.919 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044605+0000", 2024-09-14T06:17:37.919 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.919 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.919 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.919 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044605+0000", 2024-09-14T06:17:37.920 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.920 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.920 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044606+0000", 2024-09-14T06:17:37.920 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2020000000000001e-06 2024-09-14T06:17:37.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.920 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.920 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044612+0000", 2024-09-14T06:17:37.920 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9830000000000001e-06 2024-09-14T06:17:37.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.920 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.920 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044631+0000", 2024-09-14T06:17:37.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8280000000000001e-05 2024-09-14T06:17:37.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.921 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.921 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044642+0000", 2024-09-14T06:17:37.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1018e-05 2024-09-14T06:17:37.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.921 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.921 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065153+0000", 2024-09-14T06:17:37.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020511709999999999 2024-09-14T06:17:37.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.921 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.921 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065230+0000", 2024-09-14T06:17:37.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7003999999999997e-05 2024-09-14T06:17:37.922 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.922 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.922 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.922 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065329+0000", 2024-09-14T06:17:37.922 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.8517999999999994e-05 2024-09-14T06:17:37.922 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.922 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.922 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.922 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.922 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.922 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12692 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.922 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044659+0000", 2024-09-14T06:17:37.922 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.823164368, 2024-09-14T06:17:37.922 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020896346999999999, 2024-09-14T06:17:37.923 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.923 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.923 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.923 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.923 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.923 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.923 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12692 2024-09-14T06:17:37.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.923 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.923 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.923 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.923 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044659+0000", 2024-09-14T06:17:37.923 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.924 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.924 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.924 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044659+0000", 2024-09-14T06:17:37.924 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.924 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.924 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.924 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044660+0000", 2024-09-14T06:17:37.924 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0200000000000001e-07 2024-09-14T06:17:37.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.924 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.924 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.924 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044663+0000", 2024-09-14T06:17:37.924 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:17:37.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044674+0000", 2024-09-14T06:17:37.925 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.096e-05 2024-09-14T06:17:37.925 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044682+0000", 2024-09-14T06:17:37.925 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7149999999999998e-06 2024-09-14T06:17:37.925 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065357+0000", 2024-09-14T06:17:37.925 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020674794999999999 2024-09-14T06:17:37.925 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.926 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.926 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065435+0000", 2024-09-14T06:17:37.926 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8170999999999994e-05 2024-09-14T06:17:37.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.926 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.926 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065555+0000", 2024-09-14T06:17:37.926 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000120609 2024-09-14T06:17:37.926 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.926 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.926 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.927 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12694 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.927 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044692+0000", 2024-09-14T06:17:37.927 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.823131093000001, 2024-09-14T06:17:37.927 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021050059999999999, 2024-09-14T06:17:37.927 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.927 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.927 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.927 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.927 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.927 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.927 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12694 2024-09-14T06:17:37.927 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.927 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.927 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.928 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.928 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044692+0000", 2024-09-14T06:17:37.928 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.928 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.928 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.928 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.928 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044692+0000", 2024-09-14T06:17:37.928 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.928 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.928 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.928 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.928 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044693+0000", 2024-09-14T06:17:37.928 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8800000000000002e-07 2024-09-14T06:17:37.928 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.928 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044705+0000", 2024-09-14T06:17:37.929 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2252000000000001e-05 2024-09-14T06:17:37.929 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044716+0000", 2024-09-14T06:17:37.929 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1206000000000001e-05 2024-09-14T06:17:37.929 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044724+0000", 2024-09-14T06:17:37.929 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7360000000000001e-06 2024-09-14T06:17:37.929 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065584+0000", 2024-09-14T06:17:37.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020859459 2024-09-14T06:17:37.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.930 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065657+0000", 2024-09-14T06:17:37.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3509000000000004e-05 2024-09-14T06:17:37.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.930 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065742+0000", 2024-09-14T06:17:37.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5210000000000006e-05 2024-09-14T06:17:37.930 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.930 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.931 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.931 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.931 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12702 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.931 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044734+0000", 2024-09-14T06:17:37.931 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.823089103000001, 2024-09-14T06:17:37.931 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021236848999999999, 2024-09-14T06:17:37.931 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.931 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.931 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.931 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.931 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.931 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.931 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12702 2024-09-14T06:17:37.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.932 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.932 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.932 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.932 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044734+0000", 2024-09-14T06:17:37.932 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.932 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.932 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.932 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044734+0000", 2024-09-14T06:17:37.932 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.932 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.932 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.932 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044735+0000", 2024-09-14T06:17:37.932 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3600000000000003e-07 2024-09-14T06:17:37.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.933 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.933 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044738+0000", 2024-09-14T06:17:37.933 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9189999999999999e-06 2024-09-14T06:17:37.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.933 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.933 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044748+0000", 2024-09-14T06:17:37.933 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0111e-05 2024-09-14T06:17:37.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.933 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.933 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044766+0000", 2024-09-14T06:17:37.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8199999999999999e-05 2024-09-14T06:17:37.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065771+0000", 2024-09-14T06:17:37.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021004683 2024-09-14T06:17:37.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065849+0000", 2024-09-14T06:17:37.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7684999999999999e-05 2024-09-14T06:17:37.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065971+0000", 2024-09-14T06:17:37.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012251499999999999 2024-09-14T06:17:37.935 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.935 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.935 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.935 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.935 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12703 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 401408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.935 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044777+0000", 2024-09-14T06:17:37.935 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.823046476999998, 2024-09-14T06:17:37.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021382453999999999, 2024-09-14T06:17:37.935 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.935 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.935 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.935 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.935 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.935 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.936 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12703 2024-09-14T06:17:37.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.936 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.936 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.936 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.936 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044777+0000", 2024-09-14T06:17:37.936 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.936 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.936 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.936 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044777+0000", 2024-09-14T06:17:37.936 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.936 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.936 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.937 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044778+0000", 2024-09-14T06:17:37.937 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2999999999999999e-07 2024-09-14T06:17:37.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.937 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.937 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044787+0000", 2024-09-14T06:17:37.937 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2429999999999994e-06 2024-09-14T06:17:37.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.937 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.937 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044798+0000", 2024-09-14T06:17:37.937 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0447e-05 2024-09-14T06:17:37.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044805+0000", 2024-09-14T06:17:37.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.182e-06 2024-09-14T06:17:37.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066001+0000", 2024-09-14T06:17:37.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021196079 2024-09-14T06:17:37.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066082+0000", 2024-09-14T06:17:37.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1011000000000001e-05 2024-09-14T06:17:37.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066159+0000", 2024-09-14T06:17:37.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7562e-05 2024-09-14T06:17:37.939 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.939 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.939 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.939 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12705 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.939 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044815+0000", 2024-09-14T06:17:37.939 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.823008637000001, 2024-09-14T06:17:37.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021573917000000001, 2024-09-14T06:17:37.939 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.940 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.940 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.940 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.940 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.940 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.940 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12705 2024-09-14T06:17:37.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.940 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.940 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.940 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.940 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044815+0000", 2024-09-14T06:17:37.940 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.940 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044815+0000", 2024-09-14T06:17:37.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044815+0000", 2024-09-14T06:17:37.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9999999999999997e-07 2024-09-14T06:17:37.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044818+0000", 2024-09-14T06:17:37.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7779999999999999e-06 2024-09-14T06:17:37.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044828+0000", 2024-09-14T06:17:37.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0095000000000001e-05 2024-09-14T06:17:37.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044835+0000", 2024-09-14T06:17:37.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9419999999999998e-06 2024-09-14T06:17:37.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066188+0000", 2024-09-14T06:17:37.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021352560999999999 2024-09-14T06:17:37.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.943 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.943 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066275+0000", 2024-09-14T06:17:37.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7020000000000004e-05 2024-09-14T06:17:37.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.943 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.943 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066389+0000", 2024-09-14T06:17:37.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000113921 2024-09-14T06:17:37.943 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.943 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.943 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.943 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12707 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:37.943 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044844+0000", 2024-09-14T06:17:37.944 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.822979068000002, 2024-09-14T06:17:37.944 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021756622, 2024-09-14T06:17:37.944 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.944 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.944 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:37.944 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.944 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.944 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.944 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12707 2024-09-14T06:17:37.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.944 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.944 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.944 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044844+0000", 2024-09-14T06:17:37.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044844+0000", 2024-09-14T06:17:37.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044845+0000", 2024-09-14T06:17:37.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3600000000000003e-07 2024-09-14T06:17:37.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044848+0000", 2024-09-14T06:17:37.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4710000000000001e-06 2024-09-14T06:17:37.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044857+0000", 2024-09-14T06:17:37.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5219999999999996e-06 2024-09-14T06:17:37.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044864+0000", 2024-09-14T06:17:37.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:17:37.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066424+0000", 2024-09-14T06:17:37.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021560461999999999 2024-09-14T06:17:37.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066511+0000", 2024-09-14T06:17:37.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6298999999999995e-05 2024-09-14T06:17:37.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066601+0000", 2024-09-14T06:17:37.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0252000000000003e-05 2024-09-14T06:17:37.947 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.947 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.947 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.948 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.948 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:24231 3.c e47/23 3:335dc9f4:::rbd_data.1152db28e106.0000000000000002:head v 47'36, mlcod=46'34)", 2024-09-14T06:17:37.948 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:51.909218+0000", 2024-09-14T06:17:37.948 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 45.958605663999997, 2024-09-14T06:17:37.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.018625953000000001, 2024-09-14T06:17:37.948 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.948 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.948 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:37.948 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.948 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.948 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.948 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.909218+0000", 2024-09-14T06:17:37.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.949 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.949 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.909218+0000", 2024-09-14T06:17:37.949 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.949 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.949 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.909221+0000", 2024-09-14T06:17:37.949 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:17:37.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.949 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.949 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.921926+0000", 2024-09-14T06:17:37.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.012705602 2024-09-14T06:17:37.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.921949+0000", 2024-09-14T06:17:37.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2824e-05 2024-09-14T06:17:37.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.921960+0000", 2024-09-14T06:17:37.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0626e-05 2024-09-14T06:17:37.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.922035+0000", 2024-09-14T06:17:37.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4938999999999995e-05 2024-09-14T06:17:37.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.922058+0000", 2024-09-14T06:17:37.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2847e-05 2024-09-14T06:17:37.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:37.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.927752+0000", 2024-09-14T06:17:37.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056944810000000004 2024-09-14T06:17:37.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.952 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.927844+0000", 2024-09-14T06:17:37.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1524e-05 2024-09-14T06:17:37.952 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.952 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.952 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.952 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:27637 3.b 3:d7b02ca9:::rbd_data.1152ff705991.00000000000000a5:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:17:37.952 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:55.303878+0000", 2024-09-14T06:17:37.952 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 42.563945734000001, 2024-09-14T06:17:37.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.027469031000000001, 2024-09-14T06:17:37.952 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:37.952 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:37.952 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:37.953 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:37.953 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:37.953 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:37.953 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 27637 2024-09-14T06:17:37.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.953 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:37.953 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.953 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:37.953 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303878+0000", 2024-09-14T06:17:37.953 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:37.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303878+0000", 2024-09-14T06:17:37.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:37.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:37.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303881+0000", 2024-09-14T06:17:37.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8430000000000001e-06 2024-09-14T06:17:37.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:37.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303894+0000", 2024-09-14T06:17:37.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3376e-05 2024-09-14T06:17:37.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:37.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303926+0000", 2024-09-14T06:17:37.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1825000000000001e-05 2024-09-14T06:17:37.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:37.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303941+0000", 2024-09-14T06:17:37.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5048999999999999e-05 2024-09-14T06:17:37.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:37.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.304106+0000", 2024-09-14T06:17:37.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00016504700000000001 2024-09-14T06:17:37.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:37.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.304357+0000", 2024-09-14T06:17:37.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025158899999999998 2024-09-14T06:17:37.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 6", 2024-09-14T06:17:37.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.308743+0000", 2024-09-14T06:17:37.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043854419999999998 2024-09-14T06:17:37.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:37.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.331026+0000", 2024-09-14T06:17:37.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022282999000000001 2024-09-14T06:17:37.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.957 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:37.957 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.331072+0000", 2024-09-14T06:17:37.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5694e-05 2024-09-14T06:17:37.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.957 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:37.957 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.331229+0000", 2024-09-14T06:17:37.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015776399999999999 2024-09-14T06:17:37.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:37.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:37.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:37.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.331347+0000", 2024-09-14T06:17:37.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000117403 2024-09-14T06:17:37.958 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.958 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.958 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.958 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:37.958 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:37.958 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:37.960 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:37.957+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:37.960 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:17:38.060 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:38.058+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:38.097 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:38.097 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:38.097 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:38.097 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:38.111 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:17:38.161 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:38.161+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:38.215 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T06:17:38.221 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-09-14T06:17:38.251 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T06:17:38.251 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:17:38.262 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:38.258+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:38.292 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:38.292 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:38.292 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:38.292 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:38.292 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:38.310 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:17:38.320 INFO:teuthology.orchestra.run.smithi039.stdout:leaving image image_client.0-clone30 intact 2024-09-14T06:17:38.363 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:38.361+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:38.418 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-14T06:17:38.452 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:38.453 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:38.453 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:38.453 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:38.453 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.453 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40393 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 868352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.453 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.708703+0000", 2024-09-14T06:17:38.453 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.739883678999998, 2024-09-14T06:17:38.453 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12669228199999999, 2024-09-14T06:17:38.453 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.454 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.454 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.454 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.454 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.454 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.454 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40393 2024-09-14T06:17:38.454 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.454 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.454 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.454 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.455 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708703+0000", 2024-09-14T06:17:38.455 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.455 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.455 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.455 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.455 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708703+0000", 2024-09-14T06:17:38.455 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.455 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.455 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.455 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.456 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708881+0000", 2024-09-14T06:17:38.456 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00017825 2024-09-14T06:17:38.456 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.456 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.456 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.456 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708885+0000", 2024-09-14T06:17:38.456 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.019e-06 2024-09-14T06:17:38.456 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.456 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.456 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.456 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708894+0000", 2024-09-14T06:17:38.456 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5469999999999997e-06 2024-09-14T06:17:38.456 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.457 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.457 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.457 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708900+0000", 2024-09-14T06:17:38.457 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.421e-06 2024-09-14T06:17:38.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.457 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.457 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.457 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835012+0000", 2024-09-14T06:17:38.457 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12611203400000001 2024-09-14T06:17:38.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.457 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.457 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.457 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835095+0000", 2024-09-14T06:17:38.458 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.229e-05 2024-09-14T06:17:38.458 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.458 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.458 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.458 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835395+0000", 2024-09-14T06:17:38.458 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000300721 2024-09-14T06:17:38.458 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.458 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.458 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.458 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.459 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.459 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40395 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 876544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.459 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.708914+0000", 2024-09-14T06:17:38.459 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.739673047, 2024-09-14T06:17:38.459 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12688781199999999, 2024-09-14T06:17:38.459 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.459 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.459 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.459 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.459 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.459 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.459 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40395 2024-09-14T06:17:38.459 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.460 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.460 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.460 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.460 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708914+0000", 2024-09-14T06:17:38.460 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.460 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.460 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.460 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.460 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708914+0000", 2024-09-14T06:17:38.460 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.461 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.461 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.461 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709267+0000", 2024-09-14T06:17:38.461 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00035336999999999999 2024-09-14T06:17:38.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.461 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.461 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.461 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709272+0000", 2024-09-14T06:17:38.461 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4430000000000002e-06 2024-09-14T06:17:38.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.461 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.461 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.461 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709283+0000", 2024-09-14T06:17:38.462 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1034e-05 2024-09-14T06:17:38.462 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.462 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.462 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.462 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709291+0000", 2024-09-14T06:17:38.462 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4739999999999999e-06 2024-09-14T06:17:38.462 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.462 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.462 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.462 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835428+0000", 2024-09-14T06:17:38.462 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126137052 2024-09-14T06:17:38.462 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.462 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.463 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.463 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835506+0000", 2024-09-14T06:17:38.463 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8492000000000007e-05 2024-09-14T06:17:38.463 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.463 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.463 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.463 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835801+0000", 2024-09-14T06:17:38.463 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00029494699999999999 2024-09-14T06:17:38.463 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.463 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.463 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.463 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.463 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.463 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40397 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.464 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.709307+0000", 2024-09-14T06:17:38.464 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.739279944, 2024-09-14T06:17:38.464 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126901031, 2024-09-14T06:17:38.464 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.464 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.464 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.464 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.464 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.464 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.464 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40397 2024-09-14T06:17:38.464 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.464 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.464 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.464 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.464 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709307+0000", 2024-09-14T06:17:38.465 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.465 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.465 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.465 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.465 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709307+0000", 2024-09-14T06:17:38.465 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.465 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.465 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.465 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.465 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709590+0000", 2024-09-14T06:17:38.465 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00028368499999999999 2024-09-14T06:17:38.465 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.465 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.465 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.465 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709596+0000", 2024-09-14T06:17:38.466 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3940000000000004e-06 2024-09-14T06:17:38.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.466 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.466 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.466 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709605+0000", 2024-09-14T06:17:38.466 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8650000000000007e-06 2024-09-14T06:17:38.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.466 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.466 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.466 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709610+0000", 2024-09-14T06:17:38.466 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1139999999999997e-06 2024-09-14T06:17:38.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.466 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.466 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.466 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835834+0000", 2024-09-14T06:17:38.467 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12622423199999999 2024-09-14T06:17:38.467 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.467 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.467 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.467 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835915+0000", 2024-09-14T06:17:38.467 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0859999999999995e-05 2024-09-14T06:17:38.467 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.467 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.467 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.467 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836208+0000", 2024-09-14T06:17:38.467 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00029288099999999999 2024-09-14T06:17:38.467 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.467 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.467 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.467 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.467 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.468 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40399 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.468 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.709624+0000", 2024-09-14T06:17:38.468 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.738962146999999, 2024-09-14T06:17:38.468 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.127024368, 2024-09-14T06:17:38.468 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.468 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.468 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.468 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.468 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.468 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.468 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40399 2024-09-14T06:17:38.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.469 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.469 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.469 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.469 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709624+0000", 2024-09-14T06:17:38.469 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.469 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.469 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.469 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709624+0000", 2024-09-14T06:17:38.469 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.470 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.470 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.470 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710144+0000", 2024-09-14T06:17:38.470 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00051909700000000005 2024-09-14T06:17:38.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.470 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.470 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.470 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710156+0000", 2024-09-14T06:17:38.470 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2846e-05 2024-09-14T06:17:38.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.470 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.470 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.470 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710166+0000", 2024-09-14T06:17:38.470 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:17:38.471 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.471 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.471 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.471 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710174+0000", 2024-09-14T06:17:38.471 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:17:38.471 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.471 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.471 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.471 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836240+0000", 2024-09-14T06:17:38.471 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12606634899999999 2024-09-14T06:17:38.471 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.471 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.471 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.472 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836320+0000", 2024-09-14T06:17:38.472 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0073000000000004e-05 2024-09-14T06:17:38.472 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.472 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.472 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.472 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836649+0000", 2024-09-14T06:17:38.472 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032834999999999999 2024-09-14T06:17:38.472 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.472 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.472 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.472 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.472 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.473 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40401 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.473 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.710189+0000", 2024-09-14T06:17:38.473 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.738397979999998, 2024-09-14T06:17:38.473 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126903455, 2024-09-14T06:17:38.473 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.473 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.473 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.473 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.473 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.473 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.473 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40401 2024-09-14T06:17:38.473 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.473 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.474 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.474 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.474 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710189+0000", 2024-09-14T06:17:38.474 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.474 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.474 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.474 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.474 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710189+0000", 2024-09-14T06:17:38.474 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.474 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.474 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.474 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.474 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712809+0000", 2024-09-14T06:17:38.474 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026201179999999998 2024-09-14T06:17:38.475 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.475 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.475 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.475 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712815+0000", 2024-09-14T06:17:38.475 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7960000000000001e-06 2024-09-14T06:17:38.475 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.475 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.475 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.475 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712825+0000", 2024-09-14T06:17:38.475 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0044000000000001e-05 2024-09-14T06:17:38.475 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.475 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.475 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.475 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712832+0000", 2024-09-14T06:17:38.476 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.328e-06 2024-09-14T06:17:38.476 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.476 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.476 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.476 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836682+0000", 2024-09-14T06:17:38.476 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.123850141 2024-09-14T06:17:38.476 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.476 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.476 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.476 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836762+0000", 2024-09-14T06:17:38.476 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0207e-05 2024-09-14T06:17:38.476 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.476 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.477 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.477 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.837092+0000", 2024-09-14T06:17:38.477 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000329821 2024-09-14T06:17:38.477 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.477 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.477 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.477 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.477 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.477 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40407 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.477 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:38.478 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.735704564999999, 2024-09-14T06:17:38.478 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.127190898, 2024-09-14T06:17:38.478 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.478 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.478 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.478 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.478 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.478 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.478 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40407 2024-09-14T06:17:38.478 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.478 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.478 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.478 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.478 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:38.479 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.479 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.479 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.479 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.479 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:38.479 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.479 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.479 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.479 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.479 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:38.479 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0900000000000002e-07 2024-09-14T06:17:38.479 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.479 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.479 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.479 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712884+0000", 2024-09-14T06:17:38.480 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5579999999999999e-06 2024-09-14T06:17:38.480 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.480 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.480 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.480 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712887+0000", 2024-09-14T06:17:38.480 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3119999999999998e-06 2024-09-14T06:17:38.480 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.480 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.480 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.480 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712890+0000", 2024-09-14T06:17:38.480 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7e-06 2024-09-14T06:17:38.480 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.480 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.481 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.481 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.839114+0000", 2024-09-14T06:17:38.481 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126223954 2024-09-14T06:17:38.481 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.481 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.481 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.481 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.839197+0000", 2024-09-14T06:17:38.481 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3250000000000004e-05 2024-09-14T06:17:38.481 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.481 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.482 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.482 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840073+0000", 2024-09-14T06:17:38.482 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000875715 2024-09-14T06:17:38.482 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.482 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.482 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.482 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.482 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.482 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40409 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.483 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:38.483 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.735692784000001, 2024-09-14T06:17:38.483 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12739108900000001, 2024-09-14T06:17:38.483 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.483 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.483 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.483 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.483 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.483 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.483 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40409 2024-09-14T06:17:38.484 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.484 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.484 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.484 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.484 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:38.484 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.484 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.484 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.484 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.484 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:38.484 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.484 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.485 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.485 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.485 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:38.485 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.65e-07 2024-09-14T06:17:38.485 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.485 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.485 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.485 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712895+0000", 2024-09-14T06:17:38.485 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.325e-06 2024-09-14T06:17:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712899+0000", 2024-09-14T06:17:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.219e-06 2024-09-14T06:17:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712901+0000", 2024-09-14T06:17:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7310000000000002e-06 2024-09-14T06:17:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.839921+0000", 2024-09-14T06:17:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12701995199999999 2024-09-14T06:17:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840006+0000", 2024-09-14T06:17:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5063999999999997e-05 2024-09-14T06:17:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840285+0000", 2024-09-14T06:17:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027853300000000002 2024-09-14T06:17:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40411 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712909+0000", 2024-09-14T06:17:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.735677413000001, 2024-09-14T06:17:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.128251319, 2024-09-14T06:17:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40411 2024-09-14T06:17:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712909+0000", 2024-09-14T06:17:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712909+0000", 2024-09-14T06:17:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712910+0000", 2024-09-14T06:17:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.39e-07 2024-09-14T06:17:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712911+0000", 2024-09-14T06:17:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.37e-06 2024-09-14T06:17:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712914+0000", 2024-09-14T06:17:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4999999999999999e-06 2024-09-14T06:17:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712929+0000", 2024-09-14T06:17:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4406000000000001e-05 2024-09-14T06:17:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840856+0000", 2024-09-14T06:17:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.127927546 2024-09-14T06:17:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840942+0000", 2024-09-14T06:17:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5232e-05 2024-09-14T06:17:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841161+0000", 2024-09-14T06:17:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021892600000000001 2024-09-14T06:17:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40413 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712938+0000", 2024-09-14T06:17:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.735648486999999, 2024-09-14T06:17:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.128612052, 2024-09-14T06:17:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40413 2024-09-14T06:17:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712938+0000", 2024-09-14T06:17:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712938+0000", 2024-09-14T06:17:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712939+0000", 2024-09-14T06:17:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1099999999999996e-07 2024-09-14T06:17:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712942+0000", 2024-09-14T06:17:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0920000000000002e-06 2024-09-14T06:17:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712949+0000", 2024-09-14T06:17:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.075e-06 2024-09-14T06:17:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712954+0000", 2024-09-14T06:17:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2229999999999998e-06 2024-09-14T06:17:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841192+0000", 2024-09-14T06:17:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12823817100000001 2024-09-14T06:17:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841280+0000", 2024-09-14T06:17:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8190999999999996e-05 2024-09-14T06:17:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841550+0000", 2024-09-14T06:17:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00026978899999999999 2024-09-14T06:17:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40415 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712962+0000", 2024-09-14T06:17:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.735624210000001, 2024-09-14T06:17:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12902585499999999, 2024-09-14T06:17:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40415 2024-09-14T06:17:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712962+0000", 2024-09-14T06:17:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712962+0000", 2024-09-14T06:17:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712963+0000", 2024-09-14T06:17:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:17:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712966+0000", 2024-09-14T06:17:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6649999999999999e-06 2024-09-14T06:17:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712972+0000", 2024-09-14T06:17:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4819999999999999e-06 2024-09-14T06:17:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712978+0000", 2024-09-14T06:17:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4299999999999997e-06 2024-09-14T06:17:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841581+0000", 2024-09-14T06:17:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12860382100000001 2024-09-14T06:17:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841665+0000", 2024-09-14T06:17:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3228999999999999e-05 2024-09-14T06:17:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841988+0000", 2024-09-14T06:17:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032370599999999999 2024-09-14T06:17:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40417 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712983+0000", 2024-09-14T06:17:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.735603476000001, 2024-09-14T06:17:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129525837, 2024-09-14T06:17:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40417 2024-09-14T06:17:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712983+0000", 2024-09-14T06:17:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712983+0000", 2024-09-14T06:17:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713182+0000", 2024-09-14T06:17:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000198569 2024-09-14T06:17:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713186+0000", 2024-09-14T06:17:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6160000000000004e-06 2024-09-14T06:17:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713197+0000", 2024-09-14T06:17:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.046e-05 2024-09-14T06:17:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713204+0000", 2024-09-14T06:17:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.515e-06 2024-09-14T06:17:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842021+0000", 2024-09-14T06:17:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12881625199999999 2024-09-14T06:17:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842158+0000", 2024-09-14T06:17:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013778099999999999 2024-09-14T06:17:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842509+0000", 2024-09-14T06:17:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000350644 2024-09-14T06:17:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40419 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.713217+0000", 2024-09-14T06:17:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.735369939000002, 2024-09-14T06:17:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12982854599999999, 2024-09-14T06:17:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40419 2024-09-14T06:17:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713217+0000", 2024-09-14T06:17:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713217+0000", 2024-09-14T06:17:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713831+0000", 2024-09-14T06:17:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061477700000000005 2024-09-14T06:17:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713841+0000", 2024-09-14T06:17:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3789999999999995e-06 2024-09-14T06:17:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713853+0000", 2024-09-14T06:17:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1952e-05 2024-09-14T06:17:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713861+0000", 2024-09-14T06:17:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4500000000000004e-06 2024-09-14T06:17:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842560+0000", 2024-09-14T06:17:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12869836900000001 2024-09-14T06:17:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842655+0000", 2024-09-14T06:17:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5617000000000003e-05 2024-09-14T06:17:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843045+0000", 2024-09-14T06:17:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00039000200000000002 2024-09-14T06:17:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40421 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.713882+0000", 2024-09-14T06:17:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.734704571999998, 2024-09-14T06:17:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12974492600000001, 2024-09-14T06:17:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40421 2024-09-14T06:17:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713882+0000", 2024-09-14T06:17:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713882+0000", 2024-09-14T06:17:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714044+0000", 2024-09-14T06:17:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016224999999999999 2024-09-14T06:17:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714049+0000", 2024-09-14T06:17:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0150000000000003e-06 2024-09-14T06:17:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714068+0000", 2024-09-14T06:17:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8749e-05 2024-09-14T06:17:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714077+0000", 2024-09-14T06:17:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4379999999999995e-06 2024-09-14T06:17:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843079+0000", 2024-09-14T06:17:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129002958 2024-09-14T06:17:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843290+0000", 2024-09-14T06:17:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021005899999999999 2024-09-14T06:17:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843627+0000", 2024-09-14T06:17:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00033745699999999999 2024-09-14T06:17:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40423 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.714090+0000", 2024-09-14T06:17:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.734496200999999, 2024-09-14T06:17:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129886166, 2024-09-14T06:17:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40423 2024-09-14T06:17:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714090+0000", 2024-09-14T06:17:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714090+0000", 2024-09-14T06:17:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714456+0000", 2024-09-14T06:17:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000366058 2024-09-14T06:17:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714461+0000", 2024-09-14T06:17:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.228e-06 2024-09-14T06:17:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714469+0000", 2024-09-14T06:17:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6589999999999996e-06 2024-09-14T06:17:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714476+0000", 2024-09-14T06:17:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:17:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843659+0000", 2024-09-14T06:17:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129183137 2024-09-14T06:17:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843745+0000", 2024-09-14T06:17:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5354999999999998e-05 2024-09-14T06:17:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843977+0000", 2024-09-14T06:17:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000231883 2024-09-14T06:17:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40425 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.714490+0000", 2024-09-14T06:17:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.734097005999999, 2024-09-14T06:17:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13064684600000001, 2024-09-14T06:17:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40425 2024-09-14T06:17:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714490+0000", 2024-09-14T06:17:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714490+0000", 2024-09-14T06:17:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714725+0000", 2024-09-14T06:17:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000235438 2024-09-14T06:17:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714729+0000", 2024-09-14T06:17:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8909999999999997e-06 2024-09-14T06:17:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714737+0000", 2024-09-14T06:17:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5329999999999994e-06 2024-09-14T06:17:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714742+0000", 2024-09-14T06:17:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9559999999999996e-06 2024-09-14T06:17:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.844184+0000", 2024-09-14T06:17:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129441954 2024-09-14T06:17:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.844272+0000", 2024-09-14T06:17:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7151999999999995e-05 2024-09-14T06:17:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845136+0000", 2024-09-14T06:17:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00086492199999999996 2024-09-14T06:17:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40426 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.714753+0000", 2024-09-14T06:17:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.733834042000002, 2024-09-14T06:17:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.130693374, 2024-09-14T06:17:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40426 2024-09-14T06:17:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714753+0000", 2024-09-14T06:17:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714753+0000", 2024-09-14T06:17:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715118+0000", 2024-09-14T06:17:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036512099999999999 2024-09-14T06:17:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715123+0000", 2024-09-14T06:17:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9910000000000002e-06 2024-09-14T06:17:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715141+0000", 2024-09-14T06:17:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8219999999999998e-05 2024-09-14T06:17:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715149+0000", 2024-09-14T06:17:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8979999999999996e-06 2024-09-14T06:17:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.533 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845170+0000", 2024-09-14T06:17:38.533 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13002106799999999 2024-09-14T06:17:38.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.533 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.533 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.533 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845249+0000", 2024-09-14T06:17:38.533 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9163000000000001e-05 2024-09-14T06:17:38.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.533 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.533 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.533 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845446+0000", 2024-09-14T06:17:38.533 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196913 2024-09-14T06:17:38.533 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.534 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.534 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.534 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.534 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.534 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40428 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.534 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.715166+0000", 2024-09-14T06:17:38.534 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.733421091, 2024-09-14T06:17:38.534 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13071775199999999, 2024-09-14T06:17:38.534 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.534 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.534 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:38.534 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.534 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.535 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.535 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40428 2024-09-14T06:17:38.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.535 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.535 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.535 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.535 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715166+0000", 2024-09-14T06:17:38.535 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.535 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.535 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.535 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715166+0000", 2024-09-14T06:17:38.536 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.536 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.536 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.536 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715361+0000", 2024-09-14T06:17:38.536 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00019536400000000001 2024-09-14T06:17:38.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.536 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.536 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.536 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715366+0000", 2024-09-14T06:17:38.536 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4589999999999997e-06 2024-09-14T06:17:38.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.536 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.537 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.537 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715387+0000", 2024-09-14T06:17:38.537 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0829e-05 2024-09-14T06:17:38.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.537 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.537 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.537 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715402+0000", 2024-09-14T06:17:38.537 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4429e-05 2024-09-14T06:17:38.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.538 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.538 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.538 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845476+0000", 2024-09-14T06:17:38.538 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13007447999999999 2024-09-14T06:17:38.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.538 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.538 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:38.538 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845557+0000", 2024-09-14T06:17:38.538 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1100000000000006e-05 2024-09-14T06:17:38.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.538 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.539 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.539 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845883+0000", 2024-09-14T06:17:38.539 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032609099999999998 2024-09-14T06:17:38.539 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.539 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.539 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.539 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.539 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40430 3.15 3.aecb2ff5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.539 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.715416+0000", 2024-09-14T06:17:38.539 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.733170626, 2024-09-14T06:17:38.539 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13113939699999999, 2024-09-14T06:17:38.539 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.539 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.540 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:38.540 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.540 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.540 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.540 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40430 2024-09-14T06:17:38.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.540 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.540 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.540 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.540 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715416+0000", 2024-09-14T06:17:38.540 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.541 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.541 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.541 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715416+0000", 2024-09-14T06:17:38.541 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.541 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.541 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.541 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715995+0000", 2024-09-14T06:17:38.541 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057881100000000004 2024-09-14T06:17:38.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.541 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.541 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.541 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716000+0000", 2024-09-14T06:17:38.542 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.677e-06 2024-09-14T06:17:38.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.542 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.542 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.542 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716013+0000", 2024-09-14T06:17:38.542 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2326e-05 2024-09-14T06:17:38.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.542 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.542 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.542 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716021+0000", 2024-09-14T06:17:38.542 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9750000000000007e-06 2024-09-14T06:17:38.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.542 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.543 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.543 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846538+0000", 2024-09-14T06:17:38.543 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13051717400000001 2024-09-14T06:17:38.543 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.543 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.543 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.543 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846555+0000", 2024-09-14T06:17:38.543 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7433999999999999e-05 2024-09-14T06:17:38.543 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.543 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.543 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.543 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.543 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.543 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40432 3.15 3.aecb2ff5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.544 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:38.544 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.732550987, 2024-09-14T06:17:38.544 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13055824499999999, 2024-09-14T06:17:38.544 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.544 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.544 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:38.544 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.544 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.544 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.544 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40432 2024-09-14T06:17:38.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.544 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.545 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.545 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.545 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:38.545 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.545 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.545 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.545 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.545 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:38.545 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.545 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.546 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.546 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.546 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:38.546 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9599999999999998e-07 2024-09-14T06:17:38.546 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.546 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.546 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.546 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716040+0000", 2024-09-14T06:17:38.546 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.399e-06 2024-09-14T06:17:38.546 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.546 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.546 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.546 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716048+0000", 2024-09-14T06:17:38.547 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2179999999999999e-06 2024-09-14T06:17:38.547 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.547 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.547 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.547 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716054+0000", 2024-09-14T06:17:38.547 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2110000000000003e-06 2024-09-14T06:17:38.547 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.547 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.547 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.547 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846586+0000", 2024-09-14T06:17:38.547 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.130531921 2024-09-14T06:17:38.547 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.547 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.547 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.547 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846594+0000", 2024-09-14T06:17:38.547 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7000000000000008e-06 2024-09-14T06:17:38.548 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.548 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.548 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.548 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.548 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40434 3.15 3.aecb2ff5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:38.548 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.716065+0000", 2024-09-14T06:17:38.548 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.732521160000001, 2024-09-14T06:17:38.548 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13055924899999999, 2024-09-14T06:17:38.548 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:38.548 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:38.548 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:38.548 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:38.548 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:38.548 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:38.549 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40434 2024-09-14T06:17:38.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.549 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:38.549 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.549 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:38.549 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716065+0000", 2024-09-14T06:17:38.549 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.549 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.549 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:38.549 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716065+0000", 2024-09-14T06:17:38.549 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:38.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.549 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.723984+0000", 2024-09-14T06:17:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0079189670000000007 2024-09-14T06:17:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.723991+0000", 2024-09-14T06:17:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1759999999999998e-06 2024-09-14T06:17:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.724003+0000", 2024-09-14T06:17:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1914e-05 2024-09-14T06:17:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.551 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:38.551 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.724013+0000", 2024-09-14T06:17:38.551 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441999999999999e-05 2024-09-14T06:17:38.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.551 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.551 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:38.551 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846617+0000", 2024-09-14T06:17:38.551 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12260452199999999 2024-09-14T06:17:38.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:38.551 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:38.551 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:38.551 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846625+0000", 2024-09-14T06:17:38.551 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2280000000000001e-06 2024-09-14T06:17:38.551 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.551 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.551 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.552 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:38.552 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:38.552 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:38.553 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:17:38.554 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:38.549+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:38.582 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:38.583 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:17:38.612 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-14T06:17:38.406361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1293915938,"num_objects":1540,"num_object_clones":268,"num_object_copies":3080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1540,"num_whiteouts":388,"num_read":95383,"num_read_kb":737187,"num_write":14880,"num_write_kb":399921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1127,"num_bytes_recovered":140497553,"num_keys_recovered":151,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12058,"ondisk_log_size":12058,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":409556,"kb_used_data":76760,"kb_used_omap":12,"kb_used_meta":332595,"kb_avail":754565164,"statfs":{"total":773094113280,"available":772674727936,"internally_reserved":0,"allocated":78602240,"data_stored":691950984,"data_compressed":18840838,"data_compressed_allocated":46493696,"data_compressed_original":667286968,"omap_allocated":12752,"internal_metadata":340577840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1,1,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6459581,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":587,"num_read_kb":13524,"num_write":171,"num_write_kb":1262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":580,"num_bytes_recovered":66691871,"num_keys_recovered":71,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001074"},"pg_stats":[{"pgid":"3.1f","version":"80'368","reported_seq":3121,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.216358+0000","last_change":"2024-09-14T06:17:02.195565+0000","last_active":"2024-09-14T06:17:36.216358+0000","last_peered":"2024-09-14T06:17:36.216358+0000","last_clean":"2024-09-14T06:17:36.216358+0000","last_became_active":"2024-09-14T06:17:02.193313+0000","last_became_peered":"2024-09-14T06:17:02.193313+0000","last_unstale":"2024-09-14T06:17:36.216358+0000","last_undegraded":"2024-09-14T06:17:36.216358+0000","last_fullsized":"2024-09-14T06:17:36.216358+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:51:34.761417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010466430000000001,"stat_sum":{"num_bytes":54336702,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":10,"num_read":3747,"num_read_kb":52933,"num_write":447,"num_write_kb":19145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"79'290","reported_seq":4067,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.216386+0000","last_change":"2024-09-14T06:16:27.310639+0000","last_active":"2024-09-14T06:17:36.216386+0000","last_peered":"2024-09-14T06:17:36.216386+0000","last_clean":"2024-09-14T06:17:36.216386+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:17:36.216386+0000","last_undegraded":"2024-09-14T06:17:36.216386+0000","last_fullsized":"2024-09-14T06:17:36.216386+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:11:42.513574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095849699999999999,"stat_sum":{"num_bytes":62067820,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":15,"num_read":4725,"num_read_kb":42805,"num_write":282,"num_write_kb":15233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"83'912","reported_seq":1980,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-14T06:17:36.822844+0000","last_change":"2024-09-14T06:17:37.647589+0000","last_active":"2024-09-14T06:17:37.647589+0000","last_peered":"2024-09-14T06:17:36.822844+0000","last_clean":"2024-09-14T06:17:33.563657+0000","last_became_active":"2024-09-14T06:17:35.308894+0000","last_became_peered":"2024-09-14T06:17:35.308894+0000","last_unstale":"2024-09-14T06:17:36.822844+0000","last_undegraded":"2024-09-14T06:17:36.822844+0000","last_fullsized":"2024-09-14T06:17:35.213591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":912,"log_dups_size":0,"ondisk_log_size":912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001319401,"stat_sum":{"num_bytes":31291917,"num_objects":48,"num_object_clones":7,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":8,"num_read":4651,"num_read_kb":13930,"num_write":1382,"num_write_kb":8348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10785266,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"80'819","reported_seq":2442,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.224670+0000","last_change":"2024-09-14T06:16:22.201834+0000","last_active":"2024-09-14T06:17:37.224670+0000","last_peered":"2024-09-14T06:17:37.224670+0000","last_clean":"2024-09-14T06:17:37.224670+0000","last_became_active":"2024-09-14T06:16:20.190215+0000","last_became_peered":"2024-09-14T06:16:20.190215+0000","last_unstale":"2024-09-14T06:17:37.224670+0000","last_undegraded":"2024-09-14T06:17:37.224670+0000","last_fullsized":"2024-09-14T06:17:37.224670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:08:24.896344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026065160000000001,"stat_sum":{"num_bytes":25261213,"num_objects":50,"num_object_clones":7,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":12,"num_read":5499,"num_read_kb":9520,"num_write":1216,"num_write_kb":5987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"79'291","reported_seq":3511,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.208602+0000","last_change":"2024-09-14T06:16:22.193116+0000","last_active":"2024-09-14T06:17:36.208602+0000","last_peered":"2024-09-14T06:17:36.208602+0000","last_clean":"2024-09-14T06:17:36.208602+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:17:36.208602+0000","last_undegraded":"2024-09-14T06:17:36.208602+0000","last_fullsized":"2024-09-14T06:17:36.208602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:17:00.634226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014745120000000001,"stat_sum":{"num_bytes":52171639,"num_objects":51,"num_object_clones":9,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":13,"num_read":3322,"num_read_kb":39042,"num_write":328,"num_write_kb":25723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"79'468","reported_seq":2641,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.224338+0000","last_change":"2024-09-14T06:16:22.201461+0000","last_active":"2024-09-14T06:17:37.224338+0000","last_peered":"2024-09-14T06:17:37.224338+0000","last_clean":"2024-09-14T06:17:37.224338+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:17:37.224338+0000","last_undegraded":"2024-09-14T06:17:37.224338+0000","last_fullsized":"2024-09-14T06:17:37.224338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:21:54.740165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002267542,"stat_sum":{"num_bytes":12869368,"num_objects":53,"num_object_clones":8,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":22,"num_read":3546,"num_read_kb":12619,"num_write":607,"num_write_kb":7500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"80'449","reported_seq":1754,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.208298+0000","last_change":"2024-09-14T06:16:22.191773+0000","last_active":"2024-09-14T06:17:36.208298+0000","last_peered":"2024-09-14T06:17:36.208298+0000","last_clean":"2024-09-14T06:17:36.208298+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:17:36.208298+0000","last_undegraded":"2024-09-14T06:17:36.208298+0000","last_fullsized":"2024-09-14T06:17:36.208298+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:45:42.492916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073833899999999997,"stat_sum":{"num_bytes":35400414,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":2661,"num_read_kb":19057,"num_write":647,"num_write_kb":12862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"80'307","reported_seq":2584,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.216288+0000","last_change":"2024-09-14T06:16:22.205296+0000","last_active":"2024-09-14T06:17:36.216288+0000","last_peered":"2024-09-14T06:17:36.216288+0000","last_clean":"2024-09-14T06:17:36.216288+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:17:36.216288+0000","last_undegraded":"2024-09-14T06:17:36.216288+0000","last_fullsized":"2024-09-14T06:17:36.216288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:57:31.710099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012474280000000001,"stat_sum":{"num_bytes":44544444,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":11,"num_read":3410,"num_read_kb":37351,"num_write":297,"num_write_kb":32929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"83'232","reported_seq":1208,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-14T06:17:36.041773+0000","last_change":"2024-09-14T06:17:36.217871+0000","last_active":"2024-09-14T06:17:36.217871+0000","last_peered":"2024-09-14T06:17:36.041773+0000","last_clean":"2024-09-14T06:17:33.574978+0000","last_became_active":"2024-09-14T06:17:35.296774+0000","last_became_peered":"2024-09-14T06:17:35.296774+0000","last_unstale":"2024-09-14T06:17:36.041773+0000","last_undegraded":"2024-09-14T06:17:36.041773+0000","last_fullsized":"2024-09-14T06:17:35.210218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001106627,"stat_sum":{"num_bytes":24170577,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":16,"num_read":930,"num_read_kb":17047,"num_write":246,"num_write_kb":13809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":23467377,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"80'354","reported_seq":1995,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.237257+0000","last_change":"2024-09-14T06:17:37.236314+0000","last_active":"2024-09-14T06:17:37.237257+0000","last_peered":"2024-09-14T06:17:37.237257+0000","last_clean":"2024-09-14T06:17:37.237257+0000","last_became_active":"2024-09-14T06:17:37.235311+0000","last_became_peered":"2024-09-14T06:17:37.235311+0000","last_unstale":"2024-09-14T06:17:37.237257+0000","last_undegraded":"2024-09-14T06:17:37.237257+0000","last_fullsized":"2024-09-14T06:17:37.237257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:09.235644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001557389,"stat_sum":{"num_bytes":57848052,"num_objects":49,"num_object_clones":10,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":11,"num_read":2236,"num_read_kb":16285,"num_write":452,"num_write_kb":10272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":6785399,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"79'348","reported_seq":3395,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.233084+0000","last_change":"2024-09-14T06:16:22.194304+0000","last_active":"2024-09-14T06:17:37.233084+0000","last_peered":"2024-09-14T06:17:37.233084+0000","last_clean":"2024-09-14T06:17:37.233084+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:17:37.233084+0000","last_undegraded":"2024-09-14T06:17:37.233084+0000","last_fullsized":"2024-09-14T06:17:37.233084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:32:31.047615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019733379999999998,"stat_sum":{"num_bytes":71775725,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":7,"num_read":3899,"num_read_kb":64430,"num_write":448,"num_write_kb":22209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"80'246","reported_seq":1643,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.235834+0000","last_change":"2024-09-14T06:17:37.232535+0000","last_active":"2024-09-14T06:17:37.235834+0000","last_peered":"2024-09-14T06:17:37.235834+0000","last_clean":"2024-09-14T06:17:37.235834+0000","last_became_active":"2024-09-14T06:17:37.231244+0000","last_became_peered":"2024-09-14T06:17:37.231244+0000","last_unstale":"2024-09-14T06:17:37.235834+0000","last_undegraded":"2024-09-14T06:17:37.235834+0000","last_fullsized":"2024-09-14T06:17:37.235834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:22:59.070149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001465247,"stat_sum":{"num_bytes":61166345,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":10,"num_read":2620,"num_read_kb":25820,"num_write":230,"num_write_kb":15728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":32506492,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.223820+0000","last_change":"2024-09-14T06:17:34.200709+0000","last_active":"2024-09-14T06:17:37.223820+0000","last_peered":"2024-09-14T06:17:37.223820+0000","last_clean":"2024-09-14T06:17:37.223820+0000","last_became_active":"2024-09-14T06:17:34.200238+0000","last_became_peered":"2024-09-14T06:17:34.200238+0000","last_unstale":"2024-09-14T06:17:37.223820+0000","last_undegraded":"2024-09-14T06:17:37.223820+0000","last_fullsized":"2024-09-14T06:17:37.223820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:34.864695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"79'364","reported_seq":908,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.209111+0000","last_change":"2024-09-14T06:16:22.192972+0000","last_active":"2024-09-14T06:17:36.209111+0000","last_peered":"2024-09-14T06:17:36.209111+0000","last_clean":"2024-09-14T06:17:36.209111+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:17:36.209111+0000","last_undegraded":"2024-09-14T06:17:36.209111+0000","last_fullsized":"2024-09-14T06:17:36.209111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:56:10.514649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086504599999999996,"stat_sum":{"num_bytes":29669358,"num_objects":48,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":10,"num_read":1315,"num_read_kb":9524,"num_write":422,"num_write_kb":9131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.209330+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:17:36.209330+0000","last_peered":"2024-09-14T06:17:36.209330+0000","last_clean":"2024-09-14T06:17:36.209330+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:17:36.209330+0000","last_undegraded":"2024-09-14T06:17:36.209330+0000","last_fullsized":"2024-09-14T06:17:36.209330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"80'257","reported_seq":2796,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.208812+0000","last_change":"2024-09-14T06:16:22.192683+0000","last_active":"2024-09-14T06:17:36.208812+0000","last_peered":"2024-09-14T06:17:36.208812+0000","last_clean":"2024-09-14T06:17:36.208812+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:17:36.208812+0000","last_undegraded":"2024-09-14T06:17:36.208812+0000","last_fullsized":"2024-09-14T06:17:36.208812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:04:04.548750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001328716,"stat_sum":{"num_bytes":31086743,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":11,"num_read":3001,"num_read_kb":27472,"num_write":249,"num_write_kb":12571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.222845+0000","last_change":"2024-09-14T06:17:34.196668+0000","last_active":"2024-09-14T06:17:37.222845+0000","last_peered":"2024-09-14T06:17:37.222845+0000","last_clean":"2024-09-14T06:17:37.222845+0000","last_became_active":"2024-09-14T06:17:34.196072+0000","last_became_peered":"2024-09-14T06:17:34.196072+0000","last_unstale":"2024-09-14T06:17:37.222845+0000","last_undegraded":"2024-09-14T06:17:37.222845+0000","last_fullsized":"2024-09-14T06:17:37.222845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:18:45.616834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"80'258","reported_seq":1546,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.218175+0000","last_change":"2024-09-14T06:16:22.213152+0000","last_active":"2024-09-14T06:17:36.218175+0000","last_peered":"2024-09-14T06:17:36.218175+0000","last_clean":"2024-09-14T06:17:36.218175+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:17:36.218175+0000","last_undegraded":"2024-09-14T06:17:36.218175+0000","last_fullsized":"2024-09-14T06:17:36.218175+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:28:34.843601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057590699999999996,"stat_sum":{"num_bytes":38574596,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":8,"num_read":1529,"num_read_kb":12688,"num_write":240,"num_write_kb":8157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":144,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.216216+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:17:36.216216+0000","last_peered":"2024-09-14T06:17:36.216216+0000","last_clean":"2024-09-14T06:17:36.216216+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:17:36.216216+0000","last_undegraded":"2024-09-14T06:17:36.216216+0000","last_fullsized":"2024-09-14T06:17:36.216216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"79'215","reported_seq":1434,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.218923+0000","last_change":"2024-09-14T06:16:33.548583+0000","last_active":"2024-09-14T06:17:36.218923+0000","last_peered":"2024-09-14T06:17:36.218923+0000","last_clean":"2024-09-14T06:17:36.218923+0000","last_became_active":"2024-09-14T06:16:33.547925+0000","last_became_peered":"2024-09-14T06:16:33.547925+0000","last_unstale":"2024-09-14T06:17:36.218923+0000","last_undegraded":"2024-09-14T06:17:36.218923+0000","last_fullsized":"2024-09-14T06:17:36.218923+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:23:39.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089329700000000004,"stat_sum":{"num_bytes":69139655,"num_objects":49,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":16,"num_read":1310,"num_read_kb":21229,"num_write":226,"num_write_kb":9950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"79'179","reported_seq":636,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.230463+0000","last_change":"2024-09-14T06:16:27.282745+0000","last_active":"2024-09-14T06:17:37.230463+0000","last_peered":"2024-09-14T06:17:37.230463+0000","last_clean":"2024-09-14T06:17:37.230463+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:17:37.230463+0000","last_undegraded":"2024-09-14T06:17:37.230463+0000","last_fullsized":"2024-09-14T06:17:37.230463+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:53:56.212817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002354358,"stat_sum":{"num_bytes":38932892,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":335,"num_read_kb":3987,"num_write":143,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.220953+0000","last_change":"2024-09-14T06:17:34.203977+0000","last_active":"2024-09-14T06:17:37.220953+0000","last_peered":"2024-09-14T06:17:37.220953+0000","last_clean":"2024-09-14T06:17:37.220953+0000","last_became_active":"2024-09-14T06:17:34.199244+0000","last_became_peered":"2024-09-14T06:17:34.199244+0000","last_unstale":"2024-09-14T06:17:37.220953+0000","last_undegraded":"2024-09-14T06:17:37.220953+0000","last_fullsized":"2024-09-14T06:17:37.220953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:21.934094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"80'351","reported_seq":1573,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.216108+0000","last_change":"2024-09-14T06:16:22.205051+0000","last_active":"2024-09-14T06:17:36.216108+0000","last_peered":"2024-09-14T06:17:36.216108+0000","last_clean":"2024-09-14T06:17:36.216108+0000","last_became_active":"2024-09-14T06:16:20.196215+0000","last_became_peered":"2024-09-14T06:16:20.196215+0000","last_unstale":"2024-09-14T06:17:36.216108+0000","last_undegraded":"2024-09-14T06:17:36.216108+0000","last_fullsized":"2024-09-14T06:17:36.216108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:42:24.322885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001116559,"stat_sum":{"num_bytes":33594565,"num_objects":40,"num_object_clones":3,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":9,"num_read":2021,"num_read_kb":18817,"num_write":467,"num_write_kb":11041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":144,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.221860+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:17:36.221860+0000","last_peered":"2024-09-14T06:17:36.221860+0000","last_clean":"2024-09-14T06:17:36.221860+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:17:36.221860+0000","last_undegraded":"2024-09-14T06:17:36.221860+0000","last_fullsized":"2024-09-14T06:17:36.221860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'450","reported_seq":1870,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.218376+0000","last_change":"2024-09-14T06:16:27.316499+0000","last_active":"2024-09-14T06:17:36.218376+0000","last_peered":"2024-09-14T06:17:36.218376+0000","last_clean":"2024-09-14T06:17:36.218376+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:17:36.218376+0000","last_undegraded":"2024-09-14T06:17:36.218376+0000","last_fullsized":"2024-09-14T06:17:36.218376+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:10:23.624742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052773999999999996,"stat_sum":{"num_bytes":13151989,"num_objects":36,"num_object_clones":4,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":9,"num_read":2798,"num_read_kb":8680,"num_write":652,"num_write_kb":5492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":146,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.220489+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:17:37.220489+0000","last_peered":"2024-09-14T06:17:37.220489+0000","last_clean":"2024-09-14T06:17:37.220489+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:17:37.220489+0000","last_undegraded":"2024-09-14T06:17:37.220489+0000","last_fullsized":"2024-09-14T06:17:37.220489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"80'275","reported_seq":1430,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.208345+0000","last_change":"2024-09-14T06:16:27.302045+0000","last_active":"2024-09-14T06:17:36.208345+0000","last_peered":"2024-09-14T06:17:36.208345+0000","last_clean":"2024-09-14T06:17:36.208345+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:17:36.208345+0000","last_undegraded":"2024-09-14T06:17:36.208345+0000","last_fullsized":"2024-09-14T06:17:36.208345+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:33:28.609283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060103200000000004,"stat_sum":{"num_bytes":43182018,"num_objects":50,"num_object_clones":6,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":8,"num_read":1700,"num_read_kb":22421,"num_write":262,"num_write_kb":10461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":167,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.221191+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:17:37.221191+0000","last_peered":"2024-09-14T06:17:37.221191+0000","last_clean":"2024-09-14T06:17:37.221191+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:17:37.221191+0000","last_undegraded":"2024-09-14T06:17:37.221191+0000","last_fullsized":"2024-09-14T06:17:37.221191+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":237,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.223709+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:17:37.223709+0000","last_peered":"2024-09-14T06:17:37.223709+0000","last_clean":"2024-09-14T06:17:37.223709+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:17:37.223709+0000","last_undegraded":"2024-09-14T06:17:37.223709+0000","last_fullsized":"2024-09-14T06:17:37.223709+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"79'379","reported_seq":1315,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.208242+0000","last_change":"2024-09-14T06:16:22.191260+0000","last_active":"2024-09-14T06:17:36.208242+0000","last_peered":"2024-09-14T06:17:36.208242+0000","last_clean":"2024-09-14T06:17:36.208242+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:17:36.208242+0000","last_undegraded":"2024-09-14T06:17:36.208242+0000","last_fullsized":"2024-09-14T06:17:36.208242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:05:10.411581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047498199999999999,"stat_sum":{"num_bytes":12407027,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":14,"num_read":2001,"num_read_kb":5657,"num_write":539,"num_write_kb":6840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"80'343","reported_seq":1571,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-14T06:17:36.660240+0000","last_change":"2024-09-14T06:17:37.647504+0000","last_active":"2024-09-14T06:17:37.647504+0000","last_peered":"2024-09-14T06:17:36.660240+0000","last_clean":"2024-09-14T06:17:33.309444+0000","last_became_active":"2024-09-14T06:17:35.295979+0000","last_became_peered":"2024-09-14T06:17:35.295979+0000","last_unstale":"2024-09-14T06:17:36.660240+0000","last_undegraded":"2024-09-14T06:17:36.660240+0000","last_fullsized":"2024-09-14T06:17:36.660240+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418299,"stat_sum":{"num_bytes":42339757,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":8,"num_read":2182,"num_read_kb":13624,"num_write":509,"num_write_kb":14348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8477378,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"80'263","reported_seq":1963,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.220914+0000","last_change":"2024-09-14T06:16:22.323725+0000","last_active":"2024-09-14T06:17:37.220914+0000","last_peered":"2024-09-14T06:17:37.220914+0000","last_clean":"2024-09-14T06:17:37.220914+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:17:37.220914+0000","last_undegraded":"2024-09-14T06:17:37.220914+0000","last_fullsized":"2024-09-14T06:17:37.220914+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:46:42.122526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024221899999999998,"stat_sum":{"num_bytes":35230985,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":2141,"num_read_kb":16051,"num_write":246,"num_write_kb":13522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"80'206","reported_seq":2483,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.221929+0000","last_change":"2024-09-14T06:16:22.219322+0000","last_active":"2024-09-14T06:17:36.221929+0000","last_peered":"2024-09-14T06:17:36.221929+0000","last_clean":"2024-09-14T06:17:36.221929+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:17:36.221929+0000","last_undegraded":"2024-09-14T06:17:36.221929+0000","last_fullsized":"2024-09-14T06:17:36.221929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:01:35.004877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045017199999999999,"stat_sum":{"num_bytes":50879774,"num_objects":54,"num_object_clones":10,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":15,"num_read":2678,"num_read_kb":16597,"num_write":207,"num_write_kb":6676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"80'138","reported_seq":1383,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.231962+0000","last_change":"2024-09-14T06:16:22.195705+0000","last_active":"2024-09-14T06:17:37.231962+0000","last_peered":"2024-09-14T06:17:37.231962+0000","last_clean":"2024-09-14T06:17:37.231962+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:17:37.231962+0000","last_undegraded":"2024-09-14T06:17:37.231962+0000","last_fullsized":"2024-09-14T06:17:37.231962+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:02:15.957880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025578329999999998,"stat_sum":{"num_bytes":46202051,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":7,"num_read":1255,"num_read_kb":29057,"num_write":210,"num_write_kb":11572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"79'621","reported_seq":2305,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-14T06:17:36.334368+0000","last_change":"2024-09-14T06:17:37.647637+0000","last_active":"2024-09-14T06:17:37.647637+0000","last_peered":"2024-09-14T06:17:36.334368+0000","last_clean":"2024-09-14T06:17:31.562482+0000","last_became_active":"2024-09-14T06:17:35.308839+0000","last_became_peered":"2024-09-14T06:17:35.308839+0000","last_unstale":"2024-09-14T06:17:36.334368+0000","last_undegraded":"2024-09-14T06:17:36.334368+0000","last_fullsized":"2024-09-14T06:17:36.334368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001627671,"stat_sum":{"num_bytes":37374219,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":3733,"num_read_kb":26924,"num_write":955,"num_write_kb":20272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":29390214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"80'604","reported_seq":4564,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.232011+0000","last_change":"2024-09-14T06:16:33.547557+0000","last_active":"2024-09-14T06:17:37.232011+0000","last_peered":"2024-09-14T06:17:37.232011+0000","last_clean":"2024-09-14T06:17:37.232011+0000","last_became_active":"2024-09-14T06:16:33.546796+0000","last_became_peered":"2024-09-14T06:16:33.546796+0000","last_unstale":"2024-09-14T06:17:37.232011+0000","last_undegraded":"2024-09-14T06:17:37.232011+0000","last_fullsized":"2024-09-14T06:17:37.232011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:13:41.806382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026669160000000001,"stat_sum":{"num_bytes":48267984,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":12,"num_read":8998,"num_read_kb":50615,"num_write":581,"num_write_kb":16104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"80'577","reported_seq":2454,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.231967+0000","last_change":"2024-09-14T06:16:22.195247+0000","last_active":"2024-09-14T06:17:37.231967+0000","last_peered":"2024-09-14T06:17:37.231967+0000","last_clean":"2024-09-14T06:17:37.231967+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:17:37.231967+0000","last_undegraded":"2024-09-14T06:17:37.231967+0000","last_fullsized":"2024-09-14T06:17:37.231967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:12:24.464514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022870590000000001,"stat_sum":{"num_bytes":34980582,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":14,"num_read":4026,"num_read_kb":15547,"num_write":707,"num_write_kb":12738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"83'276","reported_seq":2164,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.223402+0000","last_change":"2024-09-14T06:16:22.200094+0000","last_active":"2024-09-14T06:17:37.223402+0000","last_peered":"2024-09-14T06:17:37.223402+0000","last_clean":"2024-09-14T06:17:37.223402+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:17:37.223402+0000","last_undegraded":"2024-09-14T06:17:37.223402+0000","last_fullsized":"2024-09-14T06:17:37.223402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:37:35.508264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011343239999999999,"stat_sum":{"num_bytes":53143462,"num_objects":56,"num_object_clones":8,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":14,"num_read":2228,"num_read_kb":47482,"num_write":277,"num_write_kb":14632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"79'331","reported_seq":1745,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-14T06:17:36.497995+0000","last_change":"2024-09-14T06:17:37.647467+0000","last_active":"2024-09-14T06:17:37.647467+0000","last_peered":"2024-09-14T06:17:36.497995+0000","last_clean":"2024-09-14T06:17:32.583854+0000","last_became_active":"2024-09-14T06:17:35.295763+0000","last_became_peered":"2024-09-14T06:17:35.295763+0000","last_unstale":"2024-09-14T06:17:36.497995+0000","last_undegraded":"2024-09-14T06:17:36.497995+0000","last_fullsized":"2024-09-14T06:17:36.497995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226324,"stat_sum":{"num_bytes":18337929,"num_objects":55,"num_object_clones":12,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":15,"num_read":2350,"num_read_kb":12803,"num_write":423,"num_write_kb":7432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":12239008,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"80'423","reported_seq":2636,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-14T06:17:35.785932+0000","last_change":"2024-09-14T06:17:36.218698+0000","last_active":"2024-09-14T06:17:36.218698+0000","last_peered":"2024-09-14T06:17:35.785932+0000","last_clean":"2024-09-14T06:17:33.356737+0000","last_became_active":"2024-09-14T06:17:35.295838+0000","last_became_peered":"2024-09-14T06:17:35.295838+0000","last_unstale":"2024-09-14T06:17:35.785932+0000","last_undegraded":"2024-09-14T06:17:35.785932+0000","last_fullsized":"2024-09-14T06:17:35.210541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082016299999999995,"stat_sum":{"num_bytes":17887056,"num_objects":47,"num_object_clones":6,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":14,"num_read":3083,"num_read_kb":10830,"num_write":525,"num_write_kb":8437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":10176554,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"79'389","reported_seq":3588,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.220841+0000","last_change":"2024-09-14T06:16:22.322160+0000","last_active":"2024-09-14T06:17:37.220841+0000","last_peered":"2024-09-14T06:17:37.220841+0000","last_clean":"2024-09-14T06:17:37.220841+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:17:37.220841+0000","last_undegraded":"2024-09-14T06:17:37.220841+0000","last_fullsized":"2024-09-14T06:17:37.220841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:32:32.561603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011502319999999999,"stat_sum":{"num_bytes":66038693,"num_objects":56,"num_object_clones":12,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":16,"num_read":5387,"num_read_kb":16288,"num_write":355,"num_write_kb":5570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1293325551,"num_objects":1536,"num_object_clones":268,"num_object_copies":3072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1536,"num_whiteouts":388,"num_read":95317,"num_read_kb":737132,"num_write":14777,"num_write_kb":398291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1124,"num_bytes_recovered":140497534,"num_keys_recovered":151,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":69394432,"data_stored":675316578,"data_compressed":18479156,"data_compressed_allocated":45391872,"data_compressed_original":654446008,"omap_allocated":0,"internal_metadata":0},"log_size":11983,"ondisk_log_size":11983,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41808,"kb_used_data":10624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330032,"statfs":{"total":96636764160,"available":96593952768,"internally_reserved":0,"allocated":10878976,"data_stored":94402241,"data_compressed":3232064,"data_compressed_allocated":7073792,"data_compressed_original":91463680,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38336,"kb_used_data":11304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333504,"statfs":{"total":96636764160,"available":96597508096,"internally_reserved":0,"allocated":11575296,"data_stored":99946330,"data_compressed":2315064,"data_compressed_allocated":6639616,"data_compressed_original":96134209,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80504,"kb_used_data":9060,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94291336,"statfs":{"total":96636764160,"available":96554328064,"internally_reserved":0,"allocated":9277440,"data_stored":78150955,"data_compressed":1942007,"data_compressed_allocated":5300224,"data_compressed_original":75116812,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62128,"kb_used_data":12892,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94309712,"statfs":{"total":96636764160,"available":96573145088,"internally_reserved":0,"allocated":13201408,"data_stored":123444283,"data_compressed":3225683,"data_compressed_allocated":8310784,"data_compressed_original":119406560,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41480,"kb_used_data":6120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330360,"statfs":{"total":96636764160,"available":96594288640,"internally_reserved":0,"allocated":6266880,"data_stored":50015070,"data_compressed":1716395,"data_compressed_allocated":3289088,"data_compressed_original":47828348,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542166,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48664,"kb_used_data":9156,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323176,"statfs":{"total":96636764160,"available":96586932224,"internally_reserved":0,"allocated":9375744,"data_stored":79624763,"data_compressed":2108145,"data_compressed_allocated":5206016,"data_compressed_original":76465051,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48900,"kb_used_data":9388,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322940,"statfs":{"total":96636764160,"available":96586690560,"internally_reserved":0,"allocated":9613312,"data_stored":91514609,"data_compressed":2490644,"data_compressed_allocated":6201344,"data_compressed_original":89055638,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542167,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47736,"kb_used_data":8216,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324104,"statfs":{"total":96636764160,"available":96587882496,"internally_reserved":0,"allocated":8413184,"data_stored":74852733,"data_compressed":1810836,"data_compressed_allocated":4472832,"data_compressed_original":71816670,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10969088,"data_stored":111135700,"data_compressed":2884381,"data_compressed_allocated":7290880,"data_compressed_original":107712480,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8220672,"data_stored":77102512,"data_compressed":1931817,"data_compressed_allocated":5259264,"data_compressed_original":74543372,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10592256,"data_stored":99494229,"data_compressed":2315064,"data_compressed_allocated":6639616,"data_compressed_original":96134209,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9891840,"data_stored":93950121,"data_compressed":3232064,"data_compressed_allocated":7073792,"data_compressed_original":91463680,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7442432,"data_stored":74407040,"data_compressed":1810836,"data_compressed_allocated":4472832,"data_compressed_original":71816670,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8581120,"data_stored":90478548,"data_compressed":2480454,"data_compressed_allocated":6160384,"data_compressed_original":88482198,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":79179051,"data_compressed":2108145,"data_compressed_allocated":5206016,"data_compressed_original":76465051,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5296128,"data_stored":49569377,"data_compressed":1716395,"data_compressed_allocated":3289088,"data_compressed_original":47828348,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:17:38.614 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:17:38.657 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:38.653+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:38.687 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:38.687 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:38.687 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:38.687 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:38.703 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:17:38.795 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:38.794+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:38.816 INFO:teuthology.orchestra.run.smithi039.stdout:5599 trunc from 0x43c0a5b to 0x6b2eee0 2024-09-14T06:17:38.839 INFO:teuthology.orchestra.run.smithi039.stdout:5602 trunc from 0x6b2eee0 to 0xe129dff 2024-09-14T06:17:38.856 INFO:teuthology.orchestra.run.smithi039.stdout:5603 trunc from 0xe129dff to 0x675a342 2024-09-14T06:17:38.862 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:38.857+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:38.896 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:38.896 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:38.896 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:38.896 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:38.896 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:38.922 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:17:38.939 INFO:teuthology.orchestra.run.smithi039.stdout:5604 read 0x6695249 thru 0x66a1eb0 (0xcc68 bytes) 2024-09-14T06:17:38.939 INFO:teuthology.orchestra.run.smithi039.stdout:5605 read 0x6549eeb thru 0x654f211 (0x5327 bytes) 2024-09-14T06:17:38.940 INFO:teuthology.orchestra.run.smithi039.stdout:5606 read 0x244f64b thru 0x2455c66 (0x661c bytes) 2024-09-14T06:17:38.940 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:38.940 INFO:teuthology.orchestra.run.smithi039.stdout:5608 write 0x29d2853 thru 0x29d3b1a (0x12c8 bytes) 2024-09-14T06:17:38.942 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:17:38.946 INFO:teuthology.orchestra.run.smithi039.stdout:5609 trunc from 0x675a342 to 0x953bc6d 2024-09-14T06:17:38.955 INFO:teuthology.orchestra.run.smithi039.stdout:5610 trunc from 0x953bc6d to 0x7ad5060 2024-09-14T06:17:38.963 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:38.958+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:38.968 INFO:teuthology.orchestra.run.smithi039.stdout:5611 write 0x1754342 thru 0x17606ce (0xc38d bytes) 2024-09-14T06:17:38.971 INFO:teuthology.orchestra.run.smithi039.stdout:5612 write 0x728c4f3 thru 0x728d55d (0x106b bytes) 2024-09-14T06:17:38.973 INFO:teuthology.orchestra.run.smithi039.stdout:5614 read 0x3cc8493 thru 0x3ccacef (0x285d bytes) 2024-09-14T06:17:38.981 INFO:teuthology.orchestra.run.smithi039.stdout:5616 writesame 0x839563a thru 0x83a084a data_size 0xf(0xb211 bytes) 2024-09-14T06:17:38.984 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:38.984 INFO:teuthology.orchestra.run.smithi039.stdout:5618 write 0x7657068 thru 0x766247e (0xb417 bytes) 2024-09-14T06:17:38.986 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:38.986 INFO:teuthology.orchestra.run.smithi039.stdout:5620 write 0x579129d thru 0x5797e4b (0x6baf bytes) 2024-09-14T06:17:38.988 INFO:teuthology.orchestra.run.smithi039.stdout:5621 read 0x74ebbb8 thru 0x74f0ebd (0x5306 bytes) 2024-09-14T06:17:38.989 INFO:teuthology.orchestra.run.smithi039.stdout:5622 punch from 0x31df5fa to 0x31ee95b, (0xf361 bytes) 2024-09-14T06:17:38.989 INFO:teuthology.orchestra.run.smithi039.stdout:5623 write 0x1026e1a thru 0x10341dc (0xd3c3 bytes) 2024-09-14T06:17:38.992 INFO:teuthology.orchestra.run.smithi039.stdout:5626 read 0x2d0d4d2 thru 0x2d11417 (0x3f46 bytes) 2024-09-14T06:17:38.992 INFO:teuthology.orchestra.run.smithi039.stdout:5627 writesame 0x348ed11 thru 0x349a065 data_size 0x3(0xb355 bytes) 2024-09-14T06:17:39.019 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:39.019 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:17:39.022 INFO:teuthology.orchestra.run.smithi039.stdout:5628 trunc from 0x83a084b to 0x34d1834 2024-09-14T06:17:39.044 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-14T06:17:38.406361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1293915938,"num_objects":1540,"num_object_clones":268,"num_object_copies":3080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1540,"num_whiteouts":388,"num_read":95383,"num_read_kb":737187,"num_write":14880,"num_write_kb":399921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1127,"num_bytes_recovered":140497553,"num_keys_recovered":151,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12058,"ondisk_log_size":12058,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":409556,"kb_used_data":76760,"kb_used_omap":12,"kb_used_meta":332595,"kb_avail":754565164,"statfs":{"total":773094113280,"available":772674727936,"internally_reserved":0,"allocated":78602240,"data_stored":691950984,"data_compressed":18840838,"data_compressed_allocated":46493696,"data_compressed_original":667286968,"omap_allocated":12752,"internal_metadata":340577840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1,1,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6459581,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":587,"num_read_kb":13524,"num_write":171,"num_write_kb":1262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":580,"num_bytes_recovered":66691871,"num_keys_recovered":71,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001074"},"pg_stats":[{"pgid":"3.1f","version":"80'368","reported_seq":3121,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.216358+0000","last_change":"2024-09-14T06:17:02.195565+0000","last_active":"2024-09-14T06:17:36.216358+0000","last_peered":"2024-09-14T06:17:36.216358+0000","last_clean":"2024-09-14T06:17:36.216358+0000","last_became_active":"2024-09-14T06:17:02.193313+0000","last_became_peered":"2024-09-14T06:17:02.193313+0000","last_unstale":"2024-09-14T06:17:36.216358+0000","last_undegraded":"2024-09-14T06:17:36.216358+0000","last_fullsized":"2024-09-14T06:17:36.216358+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:51:34.761417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010466430000000001,"stat_sum":{"num_bytes":54336702,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":10,"num_read":3747,"num_read_kb":52933,"num_write":447,"num_write_kb":19145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"79'290","reported_seq":4067,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.216386+0000","last_change":"2024-09-14T06:16:27.310639+0000","last_active":"2024-09-14T06:17:36.216386+0000","last_peered":"2024-09-14T06:17:36.216386+0000","last_clean":"2024-09-14T06:17:36.216386+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:17:36.216386+0000","last_undegraded":"2024-09-14T06:17:36.216386+0000","last_fullsized":"2024-09-14T06:17:36.216386+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:11:42.513574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095849699999999999,"stat_sum":{"num_bytes":62067820,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":15,"num_read":4725,"num_read_kb":42805,"num_write":282,"num_write_kb":15233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"83'912","reported_seq":1980,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-14T06:17:36.822844+0000","last_change":"2024-09-14T06:17:37.647589+0000","last_active":"2024-09-14T06:17:37.647589+0000","last_peered":"2024-09-14T06:17:36.822844+0000","last_clean":"2024-09-14T06:17:33.563657+0000","last_became_active":"2024-09-14T06:17:35.308894+0000","last_became_peered":"2024-09-14T06:17:35.308894+0000","last_unstale":"2024-09-14T06:17:36.822844+0000","last_undegraded":"2024-09-14T06:17:36.822844+0000","last_fullsized":"2024-09-14T06:17:35.213591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":912,"log_dups_size":0,"ondisk_log_size":912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001319401,"stat_sum":{"num_bytes":31291917,"num_objects":48,"num_object_clones":7,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":8,"num_read":4651,"num_read_kb":13930,"num_write":1382,"num_write_kb":8348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10785266,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"80'819","reported_seq":2442,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.224670+0000","last_change":"2024-09-14T06:16:22.201834+0000","last_active":"2024-09-14T06:17:37.224670+0000","last_peered":"2024-09-14T06:17:37.224670+0000","last_clean":"2024-09-14T06:17:37.224670+0000","last_became_active":"2024-09-14T06:16:20.190215+0000","last_became_peered":"2024-09-14T06:16:20.190215+0000","last_unstale":"2024-09-14T06:17:37.224670+0000","last_undegraded":"2024-09-14T06:17:37.224670+0000","last_fullsized":"2024-09-14T06:17:37.224670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:08:24.896344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026065160000000001,"stat_sum":{"num_bytes":25261213,"num_objects":50,"num_object_clones":7,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":12,"num_read":5499,"num_read_kb":9520,"num_write":1216,"num_write_kb":5987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"79'291","reported_seq":3511,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.208602+0000","last_change":"2024-09-14T06:16:22.193116+0000","last_active":"2024-09-14T06:17:36.208602+0000","last_peered":"2024-09-14T06:17:36.208602+0000","last_clean":"2024-09-14T06:17:36.208602+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:17:36.208602+0000","last_undegraded":"2024-09-14T06:17:36.208602+0000","last_fullsized":"2024-09-14T06:17:36.208602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:17:00.634226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014745120000000001,"stat_sum":{"num_bytes":52171639,"num_objects":51,"num_object_clones":9,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":13,"num_read":3322,"num_read_kb":39042,"num_write":328,"num_write_kb":25723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"79'468","reported_seq":2641,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.224338+0000","last_change":"2024-09-14T06:16:22.201461+0000","last_active":"2024-09-14T06:17:37.224338+0000","last_peered":"2024-09-14T06:17:37.224338+0000","last_clean":"2024-09-14T06:17:37.224338+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:17:37.224338+0000","last_undegraded":"2024-09-14T06:17:37.224338+0000","last_fullsized":"2024-09-14T06:17:37.224338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:21:54.740165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002267542,"stat_sum":{"num_bytes":12869368,"num_objects":53,"num_object_clones":8,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":22,"num_read":3546,"num_read_kb":12619,"num_write":607,"num_write_kb":7500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"80'449","reported_seq":1754,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.208298+0000","last_change":"2024-09-14T06:16:22.191773+0000","last_active":"2024-09-14T06:17:36.208298+0000","last_peered":"2024-09-14T06:17:36.208298+0000","last_clean":"2024-09-14T06:17:36.208298+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:17:36.208298+0000","last_undegraded":"2024-09-14T06:17:36.208298+0000","last_fullsized":"2024-09-14T06:17:36.208298+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:45:42.492916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073833899999999997,"stat_sum":{"num_bytes":35400414,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":2661,"num_read_kb":19057,"num_write":647,"num_write_kb":12862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"80'307","reported_seq":2584,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.216288+0000","last_change":"2024-09-14T06:16:22.205296+0000","last_active":"2024-09-14T06:17:36.216288+0000","last_peered":"2024-09-14T06:17:36.216288+0000","last_clean":"2024-09-14T06:17:36.216288+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:17:36.216288+0000","last_undegraded":"2024-09-14T06:17:36.216288+0000","last_fullsized":"2024-09-14T06:17:36.216288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:57:31.710099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012474280000000001,"stat_sum":{"num_bytes":44544444,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":11,"num_read":3410,"num_read_kb":37351,"num_write":297,"num_write_kb":32929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"83'232","reported_seq":1208,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-14T06:17:36.041773+0000","last_change":"2024-09-14T06:17:36.217871+0000","last_active":"2024-09-14T06:17:36.217871+0000","last_peered":"2024-09-14T06:17:36.041773+0000","last_clean":"2024-09-14T06:17:33.574978+0000","last_became_active":"2024-09-14T06:17:35.296774+0000","last_became_peered":"2024-09-14T06:17:35.296774+0000","last_unstale":"2024-09-14T06:17:36.041773+0000","last_undegraded":"2024-09-14T06:17:36.041773+0000","last_fullsized":"2024-09-14T06:17:35.210218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001106627,"stat_sum":{"num_bytes":24170577,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":16,"num_read":930,"num_read_kb":17047,"num_write":246,"num_write_kb":13809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":23467377,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"80'354","reported_seq":1995,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.237257+0000","last_change":"2024-09-14T06:17:37.236314+0000","last_active":"2024-09-14T06:17:37.237257+0000","last_peered":"2024-09-14T06:17:37.237257+0000","last_clean":"2024-09-14T06:17:37.237257+0000","last_became_active":"2024-09-14T06:17:37.235311+0000","last_became_peered":"2024-09-14T06:17:37.235311+0000","last_unstale":"2024-09-14T06:17:37.237257+0000","last_undegraded":"2024-09-14T06:17:37.237257+0000","last_fullsized":"2024-09-14T06:17:37.237257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:09.235644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001557389,"stat_sum":{"num_bytes":57848052,"num_objects":49,"num_object_clones":10,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":11,"num_read":2236,"num_read_kb":16285,"num_write":452,"num_write_kb":10272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":6785399,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"79'348","reported_seq":3395,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.233084+0000","last_change":"2024-09-14T06:16:22.194304+0000","last_active":"2024-09-14T06:17:37.233084+0000","last_peered":"2024-09-14T06:17:37.233084+0000","last_clean":"2024-09-14T06:17:37.233084+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:17:37.233084+0000","last_undegraded":"2024-09-14T06:17:37.233084+0000","last_fullsized":"2024-09-14T06:17:37.233084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:32:31.047615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019733379999999998,"stat_sum":{"num_bytes":71775725,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":7,"num_read":3899,"num_read_kb":64430,"num_write":448,"num_write_kb":22209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"80'246","reported_seq":1643,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.235834+0000","last_change":"2024-09-14T06:17:37.232535+0000","last_active":"2024-09-14T06:17:37.235834+0000","last_peered":"2024-09-14T06:17:37.235834+0000","last_clean":"2024-09-14T06:17:37.235834+0000","last_became_active":"2024-09-14T06:17:37.231244+0000","last_became_peered":"2024-09-14T06:17:37.231244+0000","last_unstale":"2024-09-14T06:17:37.235834+0000","last_undegraded":"2024-09-14T06:17:37.235834+0000","last_fullsized":"2024-09-14T06:17:37.235834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:22:59.070149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001465247,"stat_sum":{"num_bytes":61166345,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":10,"num_read":2620,"num_read_kb":25820,"num_write":230,"num_write_kb":15728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":32506492,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.223820+0000","last_change":"2024-09-14T06:17:34.200709+0000","last_active":"2024-09-14T06:17:37.223820+0000","last_peered":"2024-09-14T06:17:37.223820+0000","last_clean":"2024-09-14T06:17:37.223820+0000","last_became_active":"2024-09-14T06:17:34.200238+0000","last_became_peered":"2024-09-14T06:17:34.200238+0000","last_unstale":"2024-09-14T06:17:37.223820+0000","last_undegraded":"2024-09-14T06:17:37.223820+0000","last_fullsized":"2024-09-14T06:17:37.223820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:34.864695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"79'364","reported_seq":908,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.209111+0000","last_change":"2024-09-14T06:16:22.192972+0000","last_active":"2024-09-14T06:17:36.209111+0000","last_peered":"2024-09-14T06:17:36.209111+0000","last_clean":"2024-09-14T06:17:36.209111+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:17:36.209111+0000","last_undegraded":"2024-09-14T06:17:36.209111+0000","last_fullsized":"2024-09-14T06:17:36.209111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:56:10.514649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086504599999999996,"stat_sum":{"num_bytes":29669358,"num_objects":48,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":10,"num_read":1315,"num_read_kb":9524,"num_write":422,"num_write_kb":9131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.209330+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:17:36.209330+0000","last_peered":"2024-09-14T06:17:36.209330+0000","last_clean":"2024-09-14T06:17:36.209330+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:17:36.209330+0000","last_undegraded":"2024-09-14T06:17:36.209330+0000","last_fullsized":"2024-09-14T06:17:36.209330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"80'257","reported_seq":2796,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.208812+0000","last_change":"2024-09-14T06:16:22.192683+0000","last_active":"2024-09-14T06:17:36.208812+0000","last_peered":"2024-09-14T06:17:36.208812+0000","last_clean":"2024-09-14T06:17:36.208812+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:17:36.208812+0000","last_undegraded":"2024-09-14T06:17:36.208812+0000","last_fullsized":"2024-09-14T06:17:36.208812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:04:04.548750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001328716,"stat_sum":{"num_bytes":31086743,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":11,"num_read":3001,"num_read_kb":27472,"num_write":249,"num_write_kb":12571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.222845+0000","last_change":"2024-09-14T06:17:34.196668+0000","last_active":"2024-09-14T06:17:37.222845+0000","last_peered":"2024-09-14T06:17:37.222845+0000","last_clean":"2024-09-14T06:17:37.222845+0000","last_became_active":"2024-09-14T06:17:34.196072+0000","last_became_peered":"2024-09-14T06:17:34.196072+0000","last_unstale":"2024-09-14T06:17:37.222845+0000","last_undegraded":"2024-09-14T06:17:37.222845+0000","last_fullsized":"2024-09-14T06:17:37.222845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:18:45.616834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"80'258","reported_seq":1546,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.218175+0000","last_change":"2024-09-14T06:16:22.213152+0000","last_active":"2024-09-14T06:17:36.218175+0000","last_peered":"2024-09-14T06:17:36.218175+0000","last_clean":"2024-09-14T06:17:36.218175+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:17:36.218175+0000","last_undegraded":"2024-09-14T06:17:36.218175+0000","last_fullsized":"2024-09-14T06:17:36.218175+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:28:34.843601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057590699999999996,"stat_sum":{"num_bytes":38574596,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":8,"num_read":1529,"num_read_kb":12688,"num_write":240,"num_write_kb":8157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":144,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.216216+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:17:36.216216+0000","last_peered":"2024-09-14T06:17:36.216216+0000","last_clean":"2024-09-14T06:17:36.216216+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:17:36.216216+0000","last_undegraded":"2024-09-14T06:17:36.216216+0000","last_fullsized":"2024-09-14T06:17:36.216216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"79'215","reported_seq":1434,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.218923+0000","last_change":"2024-09-14T06:16:33.548583+0000","last_active":"2024-09-14T06:17:36.218923+0000","last_peered":"2024-09-14T06:17:36.218923+0000","last_clean":"2024-09-14T06:17:36.218923+0000","last_became_active":"2024-09-14T06:16:33.547925+0000","last_became_peered":"2024-09-14T06:16:33.547925+0000","last_unstale":"2024-09-14T06:17:36.218923+0000","last_undegraded":"2024-09-14T06:17:36.218923+0000","last_fullsized":"2024-09-14T06:17:36.218923+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:23:39.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089329700000000004,"stat_sum":{"num_bytes":69139655,"num_objects":49,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":16,"num_read":1310,"num_read_kb":21229,"num_write":226,"num_write_kb":9950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"79'179","reported_seq":636,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.230463+0000","last_change":"2024-09-14T06:16:27.282745+0000","last_active":"2024-09-14T06:17:37.230463+0000","last_peered":"2024-09-14T06:17:37.230463+0000","last_clean":"2024-09-14T06:17:37.230463+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:17:37.230463+0000","last_undegraded":"2024-09-14T06:17:37.230463+0000","last_fullsized":"2024-09-14T06:17:37.230463+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:53:56.212817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002354358,"stat_sum":{"num_bytes":38932892,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":335,"num_read_kb":3987,"num_write":143,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.220953+0000","last_change":"2024-09-14T06:17:34.203977+0000","last_active":"2024-09-14T06:17:37.220953+0000","last_peered":"2024-09-14T06:17:37.220953+0000","last_clean":"2024-09-14T06:17:37.220953+0000","last_became_active":"2024-09-14T06:17:34.199244+0000","last_became_peered":"2024-09-14T06:17:34.199244+0000","last_unstale":"2024-09-14T06:17:37.220953+0000","last_undegraded":"2024-09-14T06:17:37.220953+0000","last_fullsized":"2024-09-14T06:17:37.220953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:21.934094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"80'351","reported_seq":1573,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.216108+0000","last_change":"2024-09-14T06:16:22.205051+0000","last_active":"2024-09-14T06:17:36.216108+0000","last_peered":"2024-09-14T06:17:36.216108+0000","last_clean":"2024-09-14T06:17:36.216108+0000","last_became_active":"2024-09-14T06:16:20.196215+0000","last_became_peered":"2024-09-14T06:16:20.196215+0000","last_unstale":"2024-09-14T06:17:36.216108+0000","last_undegraded":"2024-09-14T06:17:36.216108+0000","last_fullsized":"2024-09-14T06:17:36.216108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:42:24.322885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001116559,"stat_sum":{"num_bytes":33594565,"num_objects":40,"num_object_clones":3,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":9,"num_read":2021,"num_read_kb":18817,"num_write":467,"num_write_kb":11041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":144,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.221860+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:17:36.221860+0000","last_peered":"2024-09-14T06:17:36.221860+0000","last_clean":"2024-09-14T06:17:36.221860+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:17:36.221860+0000","last_undegraded":"2024-09-14T06:17:36.221860+0000","last_fullsized":"2024-09-14T06:17:36.221860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'450","reported_seq":1870,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.218376+0000","last_change":"2024-09-14T06:16:27.316499+0000","last_active":"2024-09-14T06:17:36.218376+0000","last_peered":"2024-09-14T06:17:36.218376+0000","last_clean":"2024-09-14T06:17:36.218376+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:17:36.218376+0000","last_undegraded":"2024-09-14T06:17:36.218376+0000","last_fullsized":"2024-09-14T06:17:36.218376+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:10:23.624742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052773999999999996,"stat_sum":{"num_bytes":13151989,"num_objects":36,"num_object_clones":4,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":9,"num_read":2798,"num_read_kb":8680,"num_write":652,"num_write_kb":5492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":146,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.220489+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:17:37.220489+0000","last_peered":"2024-09-14T06:17:37.220489+0000","last_clean":"2024-09-14T06:17:37.220489+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:17:37.220489+0000","last_undegraded":"2024-09-14T06:17:37.220489+0000","last_fullsized":"2024-09-14T06:17:37.220489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"80'275","reported_seq":1430,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.208345+0000","last_change":"2024-09-14T06:16:27.302045+0000","last_active":"2024-09-14T06:17:36.208345+0000","last_peered":"2024-09-14T06:17:36.208345+0000","last_clean":"2024-09-14T06:17:36.208345+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:17:36.208345+0000","last_undegraded":"2024-09-14T06:17:36.208345+0000","last_fullsized":"2024-09-14T06:17:36.208345+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:33:28.609283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060103200000000004,"stat_sum":{"num_bytes":43182018,"num_objects":50,"num_object_clones":6,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":8,"num_read":1700,"num_read_kb":22421,"num_write":262,"num_write_kb":10461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":167,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.221191+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:17:37.221191+0000","last_peered":"2024-09-14T06:17:37.221191+0000","last_clean":"2024-09-14T06:17:37.221191+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:17:37.221191+0000","last_undegraded":"2024-09-14T06:17:37.221191+0000","last_fullsized":"2024-09-14T06:17:37.221191+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":237,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.223709+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:17:37.223709+0000","last_peered":"2024-09-14T06:17:37.223709+0000","last_clean":"2024-09-14T06:17:37.223709+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:17:37.223709+0000","last_undegraded":"2024-09-14T06:17:37.223709+0000","last_fullsized":"2024-09-14T06:17:37.223709+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"79'379","reported_seq":1315,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.208242+0000","last_change":"2024-09-14T06:16:22.191260+0000","last_active":"2024-09-14T06:17:36.208242+0000","last_peered":"2024-09-14T06:17:36.208242+0000","last_clean":"2024-09-14T06:17:36.208242+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:17:36.208242+0000","last_undegraded":"2024-09-14T06:17:36.208242+0000","last_fullsized":"2024-09-14T06:17:36.208242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:05:10.411581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047498199999999999,"stat_sum":{"num_bytes":12407027,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":14,"num_read":2001,"num_read_kb":5657,"num_write":539,"num_write_kb":6840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"80'343","reported_seq":1571,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-14T06:17:36.660240+0000","last_change":"2024-09-14T06:17:37.647504+0000","last_active":"2024-09-14T06:17:37.647504+0000","last_peered":"2024-09-14T06:17:36.660240+0000","last_clean":"2024-09-14T06:17:33.309444+0000","last_became_active":"2024-09-14T06:17:35.295979+0000","last_became_peered":"2024-09-14T06:17:35.295979+0000","last_unstale":"2024-09-14T06:17:36.660240+0000","last_undegraded":"2024-09-14T06:17:36.660240+0000","last_fullsized":"2024-09-14T06:17:36.660240+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418299,"stat_sum":{"num_bytes":42339757,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":8,"num_read":2182,"num_read_kb":13624,"num_write":509,"num_write_kb":14348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8477378,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"80'263","reported_seq":1963,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.220914+0000","last_change":"2024-09-14T06:16:22.323725+0000","last_active":"2024-09-14T06:17:37.220914+0000","last_peered":"2024-09-14T06:17:37.220914+0000","last_clean":"2024-09-14T06:17:37.220914+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:17:37.220914+0000","last_undegraded":"2024-09-14T06:17:37.220914+0000","last_fullsized":"2024-09-14T06:17:37.220914+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:46:42.122526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024221899999999998,"stat_sum":{"num_bytes":35230985,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":2141,"num_read_kb":16051,"num_write":246,"num_write_kb":13522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"80'206","reported_seq":2483,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.221929+0000","last_change":"2024-09-14T06:16:22.219322+0000","last_active":"2024-09-14T06:17:36.221929+0000","last_peered":"2024-09-14T06:17:36.221929+0000","last_clean":"2024-09-14T06:17:36.221929+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:17:36.221929+0000","last_undegraded":"2024-09-14T06:17:36.221929+0000","last_fullsized":"2024-09-14T06:17:36.221929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:01:35.004877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045017199999999999,"stat_sum":{"num_bytes":50879774,"num_objects":54,"num_object_clones":10,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":15,"num_read":2678,"num_read_kb":16597,"num_write":207,"num_write_kb":6676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"80'138","reported_seq":1383,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.231962+0000","last_change":"2024-09-14T06:16:22.195705+0000","last_active":"2024-09-14T06:17:37.231962+0000","last_peered":"2024-09-14T06:17:37.231962+0000","last_clean":"2024-09-14T06:17:37.231962+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:17:37.231962+0000","last_undegraded":"2024-09-14T06:17:37.231962+0000","last_fullsized":"2024-09-14T06:17:37.231962+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:02:15.957880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025578329999999998,"stat_sum":{"num_bytes":46202051,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":7,"num_read":1255,"num_read_kb":29057,"num_write":210,"num_write_kb":11572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"79'621","reported_seq":2305,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-14T06:17:36.334368+0000","last_change":"2024-09-14T06:17:37.647637+0000","last_active":"2024-09-14T06:17:37.647637+0000","last_peered":"2024-09-14T06:17:36.334368+0000","last_clean":"2024-09-14T06:17:31.562482+0000","last_became_active":"2024-09-14T06:17:35.308839+0000","last_became_peered":"2024-09-14T06:17:35.308839+0000","last_unstale":"2024-09-14T06:17:36.334368+0000","last_undegraded":"2024-09-14T06:17:36.334368+0000","last_fullsized":"2024-09-14T06:17:36.334368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001627671,"stat_sum":{"num_bytes":37374219,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":3733,"num_read_kb":26924,"num_write":955,"num_write_kb":20272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":29390214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"80'604","reported_seq":4564,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.232011+0000","last_change":"2024-09-14T06:16:33.547557+0000","last_active":"2024-09-14T06:17:37.232011+0000","last_peered":"2024-09-14T06:17:37.232011+0000","last_clean":"2024-09-14T06:17:37.232011+0000","last_became_active":"2024-09-14T06:16:33.546796+0000","last_became_peered":"2024-09-14T06:16:33.546796+0000","last_unstale":"2024-09-14T06:17:37.232011+0000","last_undegraded":"2024-09-14T06:17:37.232011+0000","last_fullsized":"2024-09-14T06:17:37.232011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:13:41.806382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026669160000000001,"stat_sum":{"num_bytes":48267984,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":12,"num_read":8998,"num_read_kb":50615,"num_write":581,"num_write_kb":16104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"80'577","reported_seq":2454,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.231967+0000","last_change":"2024-09-14T06:16:22.195247+0000","last_active":"2024-09-14T06:17:37.231967+0000","last_peered":"2024-09-14T06:17:37.231967+0000","last_clean":"2024-09-14T06:17:37.231967+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:17:37.231967+0000","last_undegraded":"2024-09-14T06:17:37.231967+0000","last_fullsized":"2024-09-14T06:17:37.231967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:12:24.464514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022870590000000001,"stat_sum":{"num_bytes":34980582,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":14,"num_read":4026,"num_read_kb":15547,"num_write":707,"num_write_kb":12738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"83'276","reported_seq":2164,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.223402+0000","last_change":"2024-09-14T06:16:22.200094+0000","last_active":"2024-09-14T06:17:37.223402+0000","last_peered":"2024-09-14T06:17:37.223402+0000","last_clean":"2024-09-14T06:17:37.223402+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:17:37.223402+0000","last_undegraded":"2024-09-14T06:17:37.223402+0000","last_fullsized":"2024-09-14T06:17:37.223402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:37:35.508264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011343239999999999,"stat_sum":{"num_bytes":53143462,"num_objects":56,"num_object_clones":8,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":14,"num_read":2228,"num_read_kb":47482,"num_write":277,"num_write_kb":14632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"79'331","reported_seq":1745,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-14T06:17:36.497995+0000","last_change":"2024-09-14T06:17:37.647467+0000","last_active":"2024-09-14T06:17:37.647467+0000","last_peered":"2024-09-14T06:17:36.497995+0000","last_clean":"2024-09-14T06:17:32.583854+0000","last_became_active":"2024-09-14T06:17:35.295763+0000","last_became_peered":"2024-09-14T06:17:35.295763+0000","last_unstale":"2024-09-14T06:17:36.497995+0000","last_undegraded":"2024-09-14T06:17:36.497995+0000","last_fullsized":"2024-09-14T06:17:36.497995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226324,"stat_sum":{"num_bytes":18337929,"num_objects":55,"num_object_clones":12,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":15,"num_read":2350,"num_read_kb":12803,"num_write":423,"num_write_kb":7432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":12239008,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"80'423","reported_seq":2636,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-14T06:17:35.785932+0000","last_change":"2024-09-14T06:17:36.218698+0000","last_active":"2024-09-14T06:17:36.218698+0000","last_peered":"2024-09-14T06:17:35.785932+0000","last_clean":"2024-09-14T06:17:33.356737+0000","last_became_active":"2024-09-14T06:17:35.295838+0000","last_became_peered":"2024-09-14T06:17:35.295838+0000","last_unstale":"2024-09-14T06:17:35.785932+0000","last_undegraded":"2024-09-14T06:17:35.785932+0000","last_fullsized":"2024-09-14T06:17:35.210541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082016299999999995,"stat_sum":{"num_bytes":17887056,"num_objects":47,"num_object_clones":6,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":14,"num_read":3083,"num_read_kb":10830,"num_write":525,"num_write_kb":8437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":10176554,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"79'389","reported_seq":3588,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.220841+0000","last_change":"2024-09-14T06:16:22.322160+0000","last_active":"2024-09-14T06:17:37.220841+0000","last_peered":"2024-09-14T06:17:37.220841+0000","last_clean":"2024-09-14T06:17:37.220841+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:17:37.220841+0000","last_undegraded":"2024-09-14T06:17:37.220841+0000","last_fullsized":"2024-09-14T06:17:37.220841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:32:32.561603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011502319999999999,"stat_sum":{"num_bytes":66038693,"num_objects":56,"num_object_clones":12,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":16,"num_read":5387,"num_read_kb":16288,"num_write":355,"num_write_kb":5570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1293325551,"num_objects":1536,"num_object_clones":268,"num_object_copies":3072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1536,"num_whiteouts":388,"num_read":95317,"num_read_kb":737132,"num_write":14777,"num_write_kb":398291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1124,"num_bytes_recovered":140497534,"num_keys_recovered":151,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":69394432,"data_stored":675316578,"data_compressed":18479156,"data_compressed_allocated":45391872,"data_compressed_original":654446008,"omap_allocated":0,"internal_metadata":0},"log_size":11983,"ondisk_log_size":11983,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41808,"kb_used_data":10624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330032,"statfs":{"total":96636764160,"available":96593952768,"internally_reserved":0,"allocated":10878976,"data_stored":94402241,"data_compressed":3232064,"data_compressed_allocated":7073792,"data_compressed_original":91463680,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38336,"kb_used_data":11304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333504,"statfs":{"total":96636764160,"available":96597508096,"internally_reserved":0,"allocated":11575296,"data_stored":99946330,"data_compressed":2315064,"data_compressed_allocated":6639616,"data_compressed_original":96134209,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80504,"kb_used_data":9060,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94291336,"statfs":{"total":96636764160,"available":96554328064,"internally_reserved":0,"allocated":9277440,"data_stored":78150955,"data_compressed":1942007,"data_compressed_allocated":5300224,"data_compressed_original":75116812,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62128,"kb_used_data":12892,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94309712,"statfs":{"total":96636764160,"available":96573145088,"internally_reserved":0,"allocated":13201408,"data_stored":123444283,"data_compressed":3225683,"data_compressed_allocated":8310784,"data_compressed_original":119406560,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41480,"kb_used_data":6120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330360,"statfs":{"total":96636764160,"available":96594288640,"internally_reserved":0,"allocated":6266880,"data_stored":50015070,"data_compressed":1716395,"data_compressed_allocated":3289088,"data_compressed_original":47828348,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542166,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48664,"kb_used_data":9156,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323176,"statfs":{"total":96636764160,"available":96586932224,"internally_reserved":0,"allocated":9375744,"data_stored":79624763,"data_compressed":2108145,"data_compressed_allocated":5206016,"data_compressed_original":76465051,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48900,"kb_used_data":9388,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322940,"statfs":{"total":96636764160,"available":96586690560,"internally_reserved":0,"allocated":9613312,"data_stored":91514609,"data_compressed":2490644,"data_compressed_allocated":6201344,"data_compressed_original":89055638,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542167,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47736,"kb_used_data":8216,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324104,"statfs":{"total":96636764160,"available":96587882496,"internally_reserved":0,"allocated":8413184,"data_stored":74852733,"data_compressed":1810836,"data_compressed_allocated":4472832,"data_compressed_original":71816670,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10969088,"data_stored":111135700,"data_compressed":2884381,"data_compressed_allocated":7290880,"data_compressed_original":107712480,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8220672,"data_stored":77102512,"data_compressed":1931817,"data_compressed_allocated":5259264,"data_compressed_original":74543372,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10592256,"data_stored":99494229,"data_compressed":2315064,"data_compressed_allocated":6639616,"data_compressed_original":96134209,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9891840,"data_stored":93950121,"data_compressed":3232064,"data_compressed_allocated":7073792,"data_compressed_original":91463680,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7442432,"data_stored":74407040,"data_compressed":1810836,"data_compressed_allocated":4472832,"data_compressed_original":71816670,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8581120,"data_stored":90478548,"data_compressed":2480454,"data_compressed_allocated":6160384,"data_compressed_original":88482198,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":79179051,"data_compressed":2108145,"data_compressed_allocated":5206016,"data_compressed_original":76465051,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5296128,"data_stored":49569377,"data_compressed":1716395,"data_compressed_allocated":3289088,"data_compressed_original":47828348,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:17:39.047 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-14T06:17:39.049 INFO:teuthology.orchestra.run.smithi039.stdout:5629 writesame 0x7487e68 thru 0x7489e95 data_size 0x3(0x202e bytes) 2024-09-14T06:17:39.051 INFO:teuthology.orchestra.run.smithi039.stdout:5631 read 0xe06843 thru 0xe142b7 (0xda75 bytes) 2024-09-14T06:17:39.052 INFO:teuthology.orchestra.run.smithi039.stdout:5632 write 0x5821a thru 0x5aba0 (0x2987 bytes) 2024-09-14T06:17:39.055 INFO:teuthology.orchestra.run.smithi039.stdout:5634 read 0x3898901 thru 0x38a1bcd (0x92cd bytes) 2024-09-14T06:17:39.065 INFO:teuthology.orchestra.run.smithi039.stdout:5635 write 0x7bc9168 thru 0x7bcb656 (0x24ef bytes) 2024-09-14T06:17:39.066 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:39.061+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:39.068 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:39.068 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:39.068 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:39.068 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:39.068 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.068 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:17:39.068 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:17:39.068 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 94.989115995999995, 2024-09-14T06:17:39.068 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.56009052699999995, 2024-09-14T06:17:39.069 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.069 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.069 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:39.069 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.069 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:17:39.069 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:17:39.069 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 1 2024-09-14T06:17:39.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.069 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.069 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.069 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.069 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:17:39.069 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.070 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.070 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.070 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:17:39.070 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.070 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.070 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.070 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077065+0000", 2024-09-14T06:17:39.070 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6429999999999999e-06 2024-09-14T06:17:39.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.071 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.071 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.071 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077079+0000", 2024-09-14T06:17:39.071 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4447000000000001e-05 2024-09-14T06:17:39.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.071 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.071 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.071 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077103+0000", 2024-09-14T06:17:39.071 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3856e-05 2024-09-14T06:17:39.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.072 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.072 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.072 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077120+0000", 2024-09-14T06:17:39.072 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7337e-05 2024-09-14T06:17:39.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.072 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.072 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.072 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077255+0000", 2024-09-14T06:17:39.072 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000134652 2024-09-14T06:17:39.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.072 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.072 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:39.072 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077274+0000", 2024-09-14T06:17:39.072 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9003000000000001e-05 2024-09-14T06:17:39.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.073 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.073 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.073 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634272+0000", 2024-09-14T06:17:39.073 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.55699813099999995 2024-09-14T06:17:39.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.073 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.073 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.073 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634591+0000", 2024-09-14T06:17:39.073 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031910800000000002 2024-09-14T06:17:39.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.073 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.073 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:17:39.073 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634892+0000", 2024-09-14T06:17:39.074 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00030066400000000002 2024-09-14T06:17:39.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.074 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.074 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:39.074 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636345+0000", 2024-09-14T06:17:39.074 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001452896 2024-09-14T06:17:39.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.074 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.074 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:39.074 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636974+0000", 2024-09-14T06:17:39.074 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00062927800000000004 2024-09-14T06:17:39.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.074 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.074 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:39.075 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637071+0000", 2024-09-14T06:17:39.075 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7178999999999999e-05 2024-09-14T06:17:39.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.075 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.075 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.075 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637153+0000", 2024-09-14T06:17:39.075 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1333000000000002e-05 2024-09-14T06:17:39.075 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.075 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.075 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.075 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.075 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38947 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 159744~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.075 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:17:39.076 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.386057708999999, 2024-09-14T06:17:39.076 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085496275999999996, 2024-09-14T06:17:39.076 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.076 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.076 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.076 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.076 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.076 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.076 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38947 2024-09-14T06:17:39.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.076 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.076 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.076 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.077 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:17:39.077 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.077 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.077 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:17:39.077 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.077 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.077 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680121+0000", 2024-09-14T06:17:39.077 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.82e-07 2024-09-14T06:17:39.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.078 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.078 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680123+0000", 2024-09-14T06:17:39.078 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5629999999999999e-06 2024-09-14T06:17:39.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.078 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.078 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.078 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680133+0000", 2024-09-14T06:17:39.078 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0890000000000006e-06 2024-09-14T06:17:39.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.078 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.078 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.078 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680138+0000", 2024-09-14T06:17:39.078 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.728e-06 2024-09-14T06:17:39.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.079 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.079 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.079 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765422+0000", 2024-09-14T06:17:39.079 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085283701000000003 2024-09-14T06:17:39.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.079 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.079 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.079 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765496+0000", 2024-09-14T06:17:39.079 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4276000000000005e-05 2024-09-14T06:17:39.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.079 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.079 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.079 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765617+0000", 2024-09-14T06:17:39.079 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000120437 2024-09-14T06:17:39.080 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.080 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.080 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.080 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.080 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38990 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 163840~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.080 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:17:39.080 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.386031725999999, 2024-09-14T06:17:39.080 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085695970999999996, 2024-09-14T06:17:39.080 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.080 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.080 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.080 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.080 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.080 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.081 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38990 2024-09-14T06:17:39.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.081 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.081 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.081 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.081 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:17:39.081 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.081 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.081 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.081 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:17:39.081 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.082 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.082 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.082 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680147+0000", 2024-09-14T06:17:39.082 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7599999999999997e-07 2024-09-14T06:17:39.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.082 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.082 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.082 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680149+0000", 2024-09-14T06:17:39.082 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4700000000000001e-06 2024-09-14T06:17:39.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.082 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.082 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.082 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680156+0000", 2024-09-14T06:17:39.082 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.058e-06 2024-09-14T06:17:39.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.083 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.083 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.083 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680162+0000", 2024-09-14T06:17:39.083 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7080000000000002e-06 2024-09-14T06:17:39.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.083 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.083 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.083 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765657+0000", 2024-09-14T06:17:39.083 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085494566999999994 2024-09-14T06:17:39.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.083 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.083 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.084 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765741+0000", 2024-09-14T06:17:39.084 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4782000000000002e-05 2024-09-14T06:17:39.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.084 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.084 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.084 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765842+0000", 2024-09-14T06:17:39.084 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010090999999999999 2024-09-14T06:17:39.084 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.084 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.084 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.084 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.084 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38991 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 172032~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.084 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:17:39.085 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.386007768999999, 2024-09-14T06:17:39.085 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085875878000000003, 2024-09-14T06:17:39.085 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.085 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.085 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.085 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.085 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.085 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.085 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38991 2024-09-14T06:17:39.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.085 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.085 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.085 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:17:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:17:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680171+0000", 2024-09-14T06:17:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4299999999999998e-07 2024-09-14T06:17:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.087 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680173+0000", 2024-09-14T06:17:39.087 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1639999999999999e-06 2024-09-14T06:17:39.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.087 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.087 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.087 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680181+0000", 2024-09-14T06:17:39.087 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0539999999999995e-06 2024-09-14T06:17:39.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.087 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.087 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.087 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680187+0000", 2024-09-14T06:17:39.087 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7529999999999998e-06 2024-09-14T06:17:39.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.087 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.088 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.088 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765875+0000", 2024-09-14T06:17:39.088 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085688722999999994 2024-09-14T06:17:39.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.088 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.088 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.088 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765948+0000", 2024-09-14T06:17:39.088 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2126999999999996e-05 2024-09-14T06:17:39.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.088 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.088 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.088 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766046+0000", 2024-09-14T06:17:39.088 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8614000000000001e-05 2024-09-14T06:17:39.088 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.089 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.089 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.089 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.089 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38992 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 180224~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.089 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:39.089 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.385982493, 2024-09-14T06:17:39.089 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086050225999999994, 2024-09-14T06:17:39.089 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.089 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.089 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.089 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.089 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.090 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.090 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38992 2024-09-14T06:17:39.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.090 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.090 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.090 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.090 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:39.090 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.090 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.090 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.090 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:39.090 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.091 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.091 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.091 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:39.091 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1199999999999998e-07 2024-09-14T06:17:39.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.091 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.091 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.091 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680199+0000", 2024-09-14T06:17:39.091 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:17:39.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.091 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.091 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.091 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680227+0000", 2024-09-14T06:17:39.091 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8238999999999998e-05 2024-09-14T06:17:39.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.092 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.092 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.092 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680234+0000", 2024-09-14T06:17:39.092 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0550000000000002e-06 2024-09-14T06:17:39.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.092 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.092 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.092 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766077+0000", 2024-09-14T06:17:39.092 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085843002000000002 2024-09-14T06:17:39.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.092 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.092 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.092 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766155+0000", 2024-09-14T06:17:39.092 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8112e-05 2024-09-14T06:17:39.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.093 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.093 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766246+0000", 2024-09-14T06:17:39.093 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0785999999999994e-05 2024-09-14T06:17:39.093 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.093 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.093 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.093 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38993 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 188416~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.093 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:17:39.093 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.385891141999998, 2024-09-14T06:17:39.094 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086166131000000007, 2024-09-14T06:17:39.094 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.094 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.094 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.094 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.094 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.094 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.094 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38993 2024-09-14T06:17:39.094 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.094 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.094 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.094 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.094 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:17:39.094 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:17:39.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680288+0000", 2024-09-14T06:17:39.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7199999999999998e-07 2024-09-14T06:17:39.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680291+0000", 2024-09-14T06:17:39.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7280000000000001e-06 2024-09-14T06:17:39.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.096 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.096 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.096 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680301+0000", 2024-09-14T06:17:39.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3360000000000001e-06 2024-09-14T06:17:39.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.096 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.096 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.096 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680308+0000", 2024-09-14T06:17:39.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1389999999999998e-06 2024-09-14T06:17:39.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.096 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.097 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.097 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766280+0000", 2024-09-14T06:17:39.097 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085972111000000004 2024-09-14T06:17:39.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.097 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.097 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.097 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766357+0000", 2024-09-14T06:17:39.097 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7477000000000004e-05 2024-09-14T06:17:39.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.097 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.098 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.098 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766453+0000", 2024-09-14T06:17:39.098 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5668000000000006e-05 2024-09-14T06:17:39.098 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.098 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.098 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.098 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.098 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38994 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 196608~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.098 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:39.098 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.385801335, 2024-09-14T06:17:39.098 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086290477000000004, 2024-09-14T06:17:39.098 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.098 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.099 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.099 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.099 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.099 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.099 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38994 2024-09-14T06:17:39.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.099 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.099 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:39.099 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:39.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:39.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3e-07 2024-09-14T06:17:39.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680381+0000", 2024-09-14T06:17:39.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.439e-06 2024-09-14T06:17:39.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680438+0000", 2024-09-14T06:17:39.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6693000000000001e-05 2024-09-14T06:17:39.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680446+0000", 2024-09-14T06:17:39.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9160000000000001e-06 2024-09-14T06:17:39.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766490+0000", 2024-09-14T06:17:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086044617000000004 2024-09-14T06:17:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766563+0000", 2024-09-14T06:17:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2430999999999998e-05 2024-09-14T06:17:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766667+0000", 2024-09-14T06:17:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000104651 2024-09-14T06:17:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38995 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 204800~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:17:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.385718309000001, 2024-09-14T06:17:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086415360999999996, 2024-09-14T06:17:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38995 2024-09-14T06:17:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:17:39.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:17:39.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680461+0000", 2024-09-14T06:17:39.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:17:39.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.105 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680464+0000", 2024-09-14T06:17:39.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4929999999999998e-06 2024-09-14T06:17:39.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.105 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680479+0000", 2024-09-14T06:17:39.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5322999999999999e-05 2024-09-14T06:17:39.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.105 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680496+0000", 2024-09-14T06:17:39.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6801000000000001e-05 2024-09-14T06:17:39.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.106 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.106 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766699+0000", 2024-09-14T06:17:39.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086203265000000001 2024-09-14T06:17:39.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.106 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.106 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766777+0000", 2024-09-14T06:17:39.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7284000000000001e-05 2024-09-14T06:17:39.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.106 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.106 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766875+0000", 2024-09-14T06:17:39.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8461000000000005e-05 2024-09-14T06:17:39.106 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.107 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.107 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.107 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38996 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 212992~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.107 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:17:39.107 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.385670869999998, 2024-09-14T06:17:39.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086576474, 2024-09-14T06:17:39.107 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.107 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.107 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.107 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.107 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.107 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.107 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38996 2024-09-14T06:17:39.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.108 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:17:39.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:17:39.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680508+0000", 2024-09-14T06:17:39.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1799999999999995e-07 2024-09-14T06:17:39.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680511+0000", 2024-09-14T06:17:39.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9739999999999998e-06 2024-09-14T06:17:39.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680520+0000", 2024-09-14T06:17:39.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.329e-06 2024-09-14T06:17:39.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.110 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.110 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680528+0000", 2024-09-14T06:17:39.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4660000000000002e-06 2024-09-14T06:17:39.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.110 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.110 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766907+0000", 2024-09-14T06:17:39.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086379529999999996 2024-09-14T06:17:39.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.110 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.110 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766987+0000", 2024-09-14T06:17:39.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0280999999999999e-05 2024-09-14T06:17:39.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.111 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767084+0000", 2024-09-14T06:17:39.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6275999999999998e-05 2024-09-14T06:17:39.111 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.111 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.111 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.111 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38997 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 221184~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.111 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:39.111 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.385641529000001, 2024-09-14T06:17:39.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086757327999999995, 2024-09-14T06:17:39.111 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.112 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.112 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.112 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.112 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.112 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.112 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38997 2024-09-14T06:17:39.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.112 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.112 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.112 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:39.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:39.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:39.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:17:39.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680540+0000", 2024-09-14T06:17:39.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6910000000000002e-06 2024-09-14T06:17:39.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680548+0000", 2024-09-14T06:17:39.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7339999999999997e-06 2024-09-14T06:17:39.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680555+0000", 2024-09-14T06:17:39.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0900000000000001e-06 2024-09-14T06:17:39.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767116+0000", 2024-09-14T06:17:39.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086561551 2024-09-14T06:17:39.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767192+0000", 2024-09-14T06:17:39.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5504000000000001e-05 2024-09-14T06:17:39.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767294+0000", 2024-09-14T06:17:39.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000102308 2024-09-14T06:17:39.116 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.116 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.117 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.117 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38998 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 229376~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.117 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:17:39.117 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.385614918000002, 2024-09-14T06:17:39.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087010193999999999, 2024-09-14T06:17:39.117 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.117 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.117 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.117 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.117 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.117 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.118 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38998 2024-09-14T06:17:39.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.118 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.118 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.118 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:17:39.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.118 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.118 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:17:39.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680564+0000", 2024-09-14T06:17:39.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1800000000000001e-07 2024-09-14T06:17:39.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680566+0000", 2024-09-14T06:17:39.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.503e-06 2024-09-14T06:17:39.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680574+0000", 2024-09-14T06:17:39.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3979999999999999e-06 2024-09-14T06:17:39.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680580+0000", 2024-09-14T06:17:39.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4859999999999997e-06 2024-09-14T06:17:39.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767359+0000", 2024-09-14T06:17:39.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086779447999999995 2024-09-14T06:17:39.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.121 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.121 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767445+0000", 2024-09-14T06:17:39.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6007000000000005e-05 2024-09-14T06:17:39.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.121 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.121 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767573+0000", 2024-09-14T06:17:39.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012793400000000001 2024-09-14T06:17:39.121 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.121 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.121 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.121 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38999 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 237568~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.121 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:17:39.122 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.385586054000001, 2024-09-14T06:17:39.122 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087211699000000004, 2024-09-14T06:17:39.122 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.122 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.122 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.122 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.122 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.122 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.122 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38999 2024-09-14T06:17:39.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.122 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.122 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.122 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:17:39.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:17:39.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680593+0000", 2024-09-14T06:17:39.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7700000000000005e-07 2024-09-14T06:17:39.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680600+0000", 2024-09-14T06:17:39.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7130000000000005e-06 2024-09-14T06:17:39.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680610+0000", 2024-09-14T06:17:39.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.55e-06 2024-09-14T06:17:39.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680616+0000", 2024-09-14T06:17:39.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3069999999999999e-06 2024-09-14T06:17:39.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767607+0000", 2024-09-14T06:17:39.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086990703000000003 2024-09-14T06:17:39.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767682+0000", 2024-09-14T06:17:39.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4944999999999995e-05 2024-09-14T06:17:39.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767804+0000", 2024-09-14T06:17:39.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012200400000000001 2024-09-14T06:17:39.125 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.126 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.126 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.126 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.126 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39000 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 245760~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.126 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:39.126 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.385552432000001, 2024-09-14T06:17:39.126 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087408649000000005, 2024-09-14T06:17:39.126 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.126 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.126 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.126 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.126 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.126 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.127 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39000 2024-09-14T06:17:39.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.127 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.127 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:39.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.127 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:39.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.128 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.128 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:39.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1699999999999998e-07 2024-09-14T06:17:39.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.128 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.128 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680629+0000", 2024-09-14T06:17:39.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5469999999999999e-06 2024-09-14T06:17:39.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.128 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.128 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680636+0000", 2024-09-14T06:17:39.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.973e-06 2024-09-14T06:17:39.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680642+0000", 2024-09-14T06:17:39.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0349999999999998e-06 2024-09-14T06:17:39.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767839+0000", 2024-09-14T06:17:39.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087197067000000003 2024-09-14T06:17:39.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.130 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.130 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767913+0000", 2024-09-14T06:17:39.130 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3832999999999996e-05 2024-09-14T06:17:39.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.130 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.130 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768034+0000", 2024-09-14T06:17:39.130 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012167699999999999 2024-09-14T06:17:39.130 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.130 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.130 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.130 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39001 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 253952~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.131 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:17:39.131 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.385527989, 2024-09-14T06:17:39.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087623312999999994, 2024-09-14T06:17:39.131 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.131 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.131 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.131 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.131 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.131 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.131 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39001 2024-09-14T06:17:39.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.131 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.131 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.131 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.132 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:17:39.132 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.132 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.132 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:17:39.132 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.132 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.132 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680651+0000", 2024-09-14T06:17:39.132 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7700000000000004e-07 2024-09-14T06:17:39.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.133 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.133 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680653+0000", 2024-09-14T06:17:39.133 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5739999999999998e-06 2024-09-14T06:17:39.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.133 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.133 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680702+0000", 2024-09-14T06:17:39.133 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8476000000000001e-05 2024-09-14T06:17:39.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.133 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.133 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680709+0000", 2024-09-14T06:17:39.133 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8170000000000001e-06 2024-09-14T06:17:39.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768066+0000", 2024-09-14T06:17:39.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087357852999999999 2024-09-14T06:17:39.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768163+0000", 2024-09-14T06:17:39.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6869999999999999e-05 2024-09-14T06:17:39.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768273+0000", 2024-09-14T06:17:39.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000110146 2024-09-14T06:17:39.135 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.135 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.135 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.135 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39002 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 262144~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.135 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:17:39.135 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.385447883000001, 2024-09-14T06:17:39.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088150478000000004, 2024-09-14T06:17:39.135 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.135 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.135 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.135 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.135 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.136 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.136 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39002 2024-09-14T06:17:39.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.136 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.136 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:17:39.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.136 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:17:39.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.137 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.137 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680739+0000", 2024-09-14T06:17:39.137 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3979999999999999e-06 2024-09-14T06:17:39.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.137 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.137 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680742+0000", 2024-09-14T06:17:39.137 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8349999999999997e-06 2024-09-14T06:17:39.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.137 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.137 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680751+0000", 2024-09-14T06:17:39.137 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8599999999999999e-06 2024-09-14T06:17:39.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.138 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.138 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680758+0000", 2024-09-14T06:17:39.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4790000000000001e-06 2024-09-14T06:17:39.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.138 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.138 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768307+0000", 2024-09-14T06:17:39.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087549342000000002 2024-09-14T06:17:39.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.138 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.138 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768392+0000", 2024-09-14T06:17:39.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.441e-05 2024-09-14T06:17:39.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768881+0000", 2024-09-14T06:17:39.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000489154 2024-09-14T06:17:39.139 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.139 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.139 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.139 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39003 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 270336~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.139 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:39.139 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.385395592999998, 2024-09-14T06:17:39.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088375555999999994, 2024-09-14T06:17:39.139 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.140 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.140 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.140 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.140 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.140 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.140 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39003 2024-09-14T06:17:39.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.140 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:39.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.141 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.141 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:39.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.141 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.141 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:39.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7000000000000004e-07 2024-09-14T06:17:39.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.141 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.141 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680786+0000", 2024-09-14T06:17:39.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:17:39.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.142 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680794+0000", 2024-09-14T06:17:39.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9840000000000001e-06 2024-09-14T06:17:39.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.142 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680800+0000", 2024-09-14T06:17:39.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1550000000000004e-06 2024-09-14T06:17:39.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.142 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768965+0000", 2024-09-14T06:17:39.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088165091000000001 2024-09-14T06:17:39.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.143 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769059+0000", 2024-09-14T06:17:39.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3923000000000002e-05 2024-09-14T06:17:39.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.143 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769158+0000", 2024-09-14T06:17:39.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8820000000000006e-05 2024-09-14T06:17:39.143 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.143 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.143 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.144 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39004 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 278528~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.144 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:17:39.144 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.385367904999999, 2024-09-14T06:17:39.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088499624999999998, 2024-09-14T06:17:39.144 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.144 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.144 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.144 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.144 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.144 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.144 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39004 2024-09-14T06:17:39.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.145 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:17:39.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:17:39.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680811+0000", 2024-09-14T06:17:39.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:17:39.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680814+0000", 2024-09-14T06:17:39.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:17:39.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680821+0000", 2024-09-14T06:17:39.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9009999999999997e-06 2024-09-14T06:17:39.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680826+0000", 2024-09-14T06:17:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5350000000000004e-06 2024-09-14T06:17:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769189+0000", 2024-09-14T06:17:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088363024999999998 2024-09-14T06:17:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769246+0000", 2024-09-14T06:17:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7017e-05 2024-09-14T06:17:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769310+0000", 2024-09-14T06:17:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3541999999999996e-05 2024-09-14T06:17:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39005 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 286720~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:17:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.385343820999999, 2024-09-14T06:17:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088681092000000003, 2024-09-14T06:17:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.149 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.149 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.149 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.149 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.149 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39005 2024-09-14T06:17:39.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.149 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.149 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.149 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:17:39.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.149 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:17:39.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.150 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680835+0000", 2024-09-14T06:17:39.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2699999999999999e-07 2024-09-14T06:17:39.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.150 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680837+0000", 2024-09-14T06:17:39.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.413e-06 2024-09-14T06:17:39.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.151 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680844+0000", 2024-09-14T06:17:39.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8820000000000003e-06 2024-09-14T06:17:39.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.151 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680850+0000", 2024-09-14T06:17:39.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5169999999999999e-06 2024-09-14T06:17:39.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.151 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769337+0000", 2024-09-14T06:17:39.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088487471999999998 2024-09-14T06:17:39.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769404+0000", 2024-09-14T06:17:39.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7111000000000002e-05 2024-09-14T06:17:39.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769515+0000", 2024-09-14T06:17:39.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011116999999999999 2024-09-14T06:17:39.152 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.152 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.152 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.152 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39006 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 294912~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.153 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:17:39.153 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.385320658000001, 2024-09-14T06:17:39.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088886572999999997, 2024-09-14T06:17:39.153 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.153 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.153 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.153 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.153 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.153 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.153 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39006 2024-09-14T06:17:39.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.153 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:17:39.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:17:39.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680858+0000", 2024-09-14T06:17:39.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7e-07 2024-09-14T06:17:39.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680860+0000", 2024-09-14T06:17:39.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:17:39.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680867+0000", 2024-09-14T06:17:39.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0430000000000002e-06 2024-09-14T06:17:39.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680873+0000", 2024-09-14T06:17:39.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7389999999999996e-06 2024-09-14T06:17:39.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.156 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.156 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769547+0000", 2024-09-14T06:17:39.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088673830999999995 2024-09-14T06:17:39.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.156 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.156 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769631+0000", 2024-09-14T06:17:39.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3660999999999997e-05 2024-09-14T06:17:39.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.156 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769744+0000", 2024-09-14T06:17:39.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000113482 2024-09-14T06:17:39.157 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.157 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.157 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.157 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39007 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 303104~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:39.157 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:17:39.157 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 29.385297246, 2024-09-14T06:17:39.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.089077397000000003, 2024-09-14T06:17:39.157 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:39.157 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:39.157 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:39.158 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:39.158 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:39.158 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:39.158 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39007 2024-09-14T06:17:39.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.158 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:39.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:39.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:17:39.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:39.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:17:39.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:39.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:39.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687953+0000", 2024-09-14T06:17:39.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00707203 2024-09-14T06:17:39.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:39.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687960+0000", 2024-09-14T06:17:39.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9890000000000003e-06 2024-09-14T06:17:39.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:39.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687974+0000", 2024-09-14T06:17:39.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4226999999999999e-05 2024-09-14T06:17:39.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:39.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687983+0000", 2024-09-14T06:17:39.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9800000000000004e-06 2024-09-14T06:17:39.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:39.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769775+0000", 2024-09-14T06:17:39.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.081792089999999998 2024-09-14T06:17:39.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:39.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769849+0000", 2024-09-14T06:17:39.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3925999999999997e-05 2024-09-14T06:17:39.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:39.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:39.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:39.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769958+0000", 2024-09-14T06:17:39.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010915499999999999 2024-09-14T06:17:39.161 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.161 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.161 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.161 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:39.161 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:39.161 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:39.162 INFO:teuthology.orchestra.run.smithi039.stdout:5636 punch from 0x72ba442 to 0x72bc8b7, (0x2475 bytes) 2024-09-14T06:17:39.162 INFO:teuthology.orchestra.run.smithi039.stdout:5640 punch from 0x606d10f to 0x6077e03, (0xacf4 bytes) 2024-09-14T06:17:39.162 INFO:teuthology.orchestra.run.smithi039.stdout:5642 write 0x5c4e114 thru 0x5c563c1 (0x82ae bytes) 2024-09-14T06:17:39.163 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:17:39.164 INFO:teuthology.orchestra.run.smithi039.stdout:5643 write 0xd675a88 thru 0xd682c0f (0xd188 bytes) 2024-09-14T06:17:39.164 INFO:teuthology.orchestra.run.smithi039.stdout:5644 punch from 0x534422d to 0x534839a, (0x416d bytes) 2024-09-14T06:17:39.164 INFO:teuthology.orchestra.run.smithi039.stdout:5647 read 0x496f298 thru 0x4972544 (0x32ad bytes) 2024-09-14T06:17:39.164 INFO:teuthology.orchestra.run.smithi039.stdout:5648 read 0x4bc5398 thru 0x4bd5228 (0xfe91 bytes) 2024-09-14T06:17:39.164 INFO:teuthology.orchestra.run.smithi039.stdout:5649 trunc from 0xd682c10 to 0xc5ec284 2024-09-14T06:17:39.165 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:39.165 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:39.166 INFO:teuthology.orchestra.run.smithi039.stdout:5650 write 0x612d22c thru 0x612eff2 (0x1dc7 bytes) 2024-09-14T06:17:39.166 INFO:teuthology.orchestra.run.smithi039.stdout:5652 writesame 0xcc5bf92 thru 0xcc678b2 data_size 0x1(0xb921 bytes) 2024-09-14T06:17:39.166 INFO:teuthology.orchestra.run.smithi039.stdout:5653 read 0x9e56aa0 thru 0x9e5be48 (0x53a9 bytes) 2024-09-14T06:17:39.166 INFO:teuthology.orchestra.run.smithi039.stdout:5654 punch from 0xda1b74 to 0xdaca2d, (0xaeb9 bytes) 2024-09-14T06:17:39.166 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:39.166 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:39.166 INFO:teuthology.orchestra.run.smithi039.stdout:5660 trunc from 0xcc678b3 to 0x9182271 2024-09-14T06:17:39.166 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:39.167 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:39.167 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:39.167 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:39.168 INFO:teuthology.orchestra.run.smithi039.stdout:5661 write 0xc605be3 thru 0xc605f38 (0x356 bytes) 2024-09-14T06:17:39.168 INFO:teuthology.orchestra.run.smithi039.stdout:5662 writesame 0x1f51017 thru 0x1f57a74 data_size 0x23(0x6a5e bytes) 2024-09-14T06:17:39.168 INFO:teuthology.orchestra.run.smithi039.stdout:5663 punch from 0x71ee0d5 to 0x71f1848, (0x3773 bytes) 2024-09-14T06:17:39.168 INFO:teuthology.orchestra.run.smithi039.stdout:5664 read 0xa6dc1b5 thru 0xa6e946d (0xd2b9 bytes) 2024-09-14T06:17:39.168 INFO:teuthology.orchestra.run.smithi039.stdout:5665 write 0x42d9829 thru 0x42e16dd (0x7eb5 bytes) 2024-09-14T06:17:39.169 INFO:teuthology.orchestra.run.smithi039.stdout:5666 trunc from 0xc605f39 to 0x888d4b9 2024-09-14T06:17:39.169 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:39.169 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:39.170 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:39.165+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:39.178 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:39.187 INFO:teuthology.orchestra.run.smithi039.stdout:5668 trunc from 0x888d4b9 to 0xb2dd7c 2024-09-14T06:17:39.197 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:39.197 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:39.219 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:39.219 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:39.234 INFO:teuthology.orchestra.run.smithi039.stdout:5669 write 0xe512a06 thru 0xe513a06 (0x1001 bytes) 2024-09-14T06:17:39.237 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:39.237 INFO:teuthology.orchestra.run.smithi039.stdout:5671 punch from 0xd779d71 to 0xd782537, (0x87c6 bytes) 2024-09-14T06:17:39.237 INFO:teuthology.orchestra.run.smithi039.stdout:5672 read 0xad829e6 thru 0xad84e75 (0x2490 bytes) 2024-09-14T06:17:39.238 INFO:teuthology.orchestra.run.smithi039.stdout:5673 read 0xca8ba5 thru 0xcb3da6 (0xb202 bytes) 2024-09-14T06:17:39.238 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:39.243 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:39.243 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:39.244 INFO:teuthology.orchestra.run.smithi039.stdout:5676 trunc from 0xe513a07 to 0x8fe62c4 2024-09-14T06:17:39.258 INFO:teuthology.orchestra.run.smithi039.stdout:5677 punch from 0x6b12412 to 0x6b1e108, (0xbcf6 bytes) 2024-09-14T06:17:39.258 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:39.265 INFO:teuthology.orchestra.run.smithi039.stdout:5679 trunc from 0x8fe62c4 to 0x37e6735 2024-09-14T06:17:39.271 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:39.269+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:39.281 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:39.281 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:39.288 INFO:teuthology.orchestra.run.smithi039.stdout:5680 write 0x6571370 thru 0x657bbf6 (0xa887 bytes) 2024-09-14T06:17:39.291 INFO:teuthology.orchestra.run.smithi039.stdout:5681 read 0x3f21bed thru 0x3f30eb7 (0xf2cb bytes) 2024-09-14T06:17:39.299 INFO:teuthology.orchestra.run.smithi039.stdout:5682 trunc from 0x657bbf7 to 0xbe1defe 2024-09-14T06:17:39.306 INFO:teuthology.orchestra.run.smithi039.stdout:5683 read 0x5dca85c thru 0x5dcce3b (0x25e0 bytes) 2024-09-14T06:17:39.315 INFO:teuthology.orchestra.run.smithi039.stdout:5684 write 0xdfa6997 thru 0xdfb3694 (0xccfe bytes) 2024-09-14T06:17:39.317 INFO:teuthology.orchestra.run.smithi039.stdout:5685 write 0x7c6dd65 thru 0x7c7a308 (0xc5a4 bytes) 2024-09-14T06:17:39.320 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:39.320 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:39.321 INFO:teuthology.orchestra.run.smithi039.stdout:5686 write 0x2b0b5da thru 0x2b1b519 (0xff40 bytes) 2024-09-14T06:17:39.324 INFO:teuthology.orchestra.run.smithi039.stdout:5688 read 0x494d832 thru 0x4958116 (0xa8e5 bytes) 2024-09-14T06:17:39.325 INFO:teuthology.orchestra.run.smithi039.stdout:5689 write 0x8ab0391 thru 0x8abac79 (0xa8e9 bytes) 2024-09-14T06:17:39.328 INFO:teuthology.orchestra.run.smithi039.stdout:5690 read 0xc828da0 thru 0xc829983 (0xbe4 bytes) 2024-09-14T06:17:39.328 INFO:teuthology.orchestra.run.smithi039.stdout:5691 write 0x9a37e57 thru 0x9a3db42 (0x5cec bytes) 2024-09-14T06:17:39.330 INFO:teuthology.orchestra.run.smithi039.stdout:5693 write 0x40ce8b4 thru 0x40d1f3b (0x3688 bytes) 2024-09-14T06:17:39.334 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:17:39.351 INFO:teuthology.orchestra.run.smithi039.stdout:5696 trunc from 0xdfb3695 to 0x15a598 2024-09-14T06:17:39.395 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:39.394+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:39.430 INFO:teuthology.orchestra.run.smithi039.stdout:5698 write 0xeb8d935 thru 0xeb9cb67 (0xf233 bytes) 2024-09-14T06:17:39.435 INFO:teuthology.orchestra.run.smithi039.stdout:5701 trunc from 0xeb9cb68 to 0xd0de0ba 2024-09-14T06:17:39.452 INFO:teuthology.orchestra.run.smithi039.stdout:5703 punch from 0xab9614e to 0xab9a466, (0x4318 bytes) 2024-09-14T06:17:39.452 INFO:teuthology.orchestra.run.smithi039.stdout:5704 read 0x3f19925 thru 0x3f1ada8 (0x1484 bytes) 2024-09-14T06:17:39.453 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:39.453 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"038563b1-268f-461f-b6a6-4b4b41ad0236","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"nodeep-scrub flag(s) set","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":109,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":86,"num_osds":8,"num_up_osds":8,"osd_up_since":1726294562,"num_in_osds":8,"osd_in_since":1726294586,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":35},{"state_name":"peering","count":5},{"state_name":"creating+peering","count":1}],"num_pgs":41,"num_pools":3,"num_objects":1540,"data_bytes":1293915938,"bytes_used":419385344,"bytes_avail":772674727936,"bytes_total":773094113280,"inactive_pgs_ratio":0.14634145796298981,"recovering_objects_per_sec":144,"recovering_bytes_per_sec":16668490,"recovering_keys_per_sec":17,"num_objects_recovered":580,"num_bytes_recovered":66691871,"num_keys_recovered":71,"read_bytes_sec":3461214,"write_bytes_sec":322985,"read_op_per_sec":146,"write_op_per_sec":42},"fsmap":{"epoch":1,"btime":"2024-09-14T06:15:49:429058+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-14T06:17:04.398314+0000","services":{}},"progress_events":{}} 2024-09-14T06:17:39.460 INFO:teuthology.orchestra.run.smithi039.stdout:5705 write 0xd0dd84f thru 0xd0e0c14 (0x33c6 bytes) 2024-09-14T06:17:39.463 INFO:teuthology.orchestra.run.smithi039.stdout:5706 writesame 0x22cba49 thru 0x22d9644 data_size 0x9c(0xdbfc bytes) 2024-09-14T06:17:39.473 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:39.470+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:39.478 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-14T06:17:39.478 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:17:39.478 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:39.479 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:39.479 INFO:teuthology.orchestra.run.smithi039.stdout:5707 trunc from 0xd0e0c15 to 0x7a75c3f 2024-09-14T06:17:39.487 INFO:teuthology.orchestra.run.smithi039.stdout:5708 read 0x43f0328 thru 0x43f9d18 (0x99f1 bytes) 2024-09-14T06:17:39.487 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:39.487 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:39.487 INFO:teuthology.orchestra.run.smithi039.stdout:5712 writesame 0x36f356e thru 0x36fd67d data_size 0x18(0xa110 bytes) 2024-09-14T06:17:39.491 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:17:39.493 INFO:teuthology.orchestra.run.smithi039.stdout:5713 punch from 0x79cdd65 to 0x79d9dd7, (0xc072 bytes) 2024-09-14T06:17:39.493 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:39.493 INFO:teuthology.orchestra.run.smithi039.stdout:5715 read 0x3dbc509 thru 0x3dc68bb (0xa3b3 bytes) 2024-09-14T06:17:39.499 INFO:teuthology.orchestra.run.smithi039.stdout:5716 trunc from 0x7a75c3f to 0x20bb482 2024-09-14T06:17:39.511 INFO:teuthology.orchestra.run.smithi039.stdout:5717 read 0x81badc thru 0x827b3c (0xc061 bytes) 2024-09-14T06:17:39.524 INFO:teuthology.orchestra.run.smithi039.stdout:5718 trunc from 0x20bb482 to 0xcf06ffe 2024-09-14T06:17:39.531 INFO:teuthology.orchestra.run.smithi039.stdout:5719 punch from 0xca74df4 to 0xca7d36d, (0x8579 bytes) 2024-09-14T06:17:39.534 INFO:teuthology.orchestra.run.smithi039.stdout:5720 trunc from 0xcf06ffe to 0xacdfb01 2024-09-14T06:17:39.555 INFO:teuthology.orchestra.run.smithi039.stdout:5722 write 0xdc17ea5 thru 0xdc19073 (0x11cf bytes) 2024-09-14T06:17:39.560 INFO:teuthology.orchestra.run.smithi039.stdout:5725 write 0x95741fb thru 0x957e4b1 (0xa2b7 bytes) 2024-09-14T06:17:39.573 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:39.570+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:39.577 INFO:teuthology.orchestra.run.smithi039.stdout:5726 trunc from 0xdc19074 to 0x375fdad 2024-09-14T06:17:39.587 INFO:teuthology.orchestra.run.smithi039.stdout:5727 read 0x2aa6662 thru 0x2ab5658 (0xeff7 bytes) 2024-09-14T06:17:39.598 INFO:teuthology.orchestra.run.smithi039.stdout:5732 write 0x434fd66 thru 0x435bd35 (0xbfd0 bytes) 2024-09-14T06:17:39.599 INFO:teuthology.orchestra.run.smithi039.stdout:5733 write 0xce3430 thru 0xce4ad2 (0x16a3 bytes) 2024-09-14T06:17:39.609 INFO:teuthology.orchestra.run.smithi039.stdout:5734 trunc from 0x435bd36 to 0xb65828e 2024-09-14T06:17:39.617 INFO:teuthology.orchestra.run.smithi039.stdout:5735 read 0x46da0f9 thru 0x46dbb11 (0x1a19 bytes) 2024-09-14T06:17:39.617 INFO:teuthology.orchestra.run.smithi039.stdout:5736 writesame 0x268d1f thru 0x273901 data_size 0x1(0xabe3 bytes) 2024-09-14T06:17:39.625 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:39.625 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:39.640 INFO:teuthology.orchestra.run.smithi039.stdout:5737 read 0x726a325 thru 0x7270f9e (0x6c7a bytes) 2024-09-14T06:17:39.640 INFO:teuthology.orchestra.run.smithi039.stdout:5739 read 0x58e37f7 thru 0x58e8de5 (0x55ef bytes) 2024-09-14T06:17:39.640 INFO:teuthology.orchestra.run.smithi039.stdout:5740 writesame 0x557917e thru 0x558489b data_size 0x27(0xb71e bytes) 2024-09-14T06:17:39.641 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:17:39.647 INFO:teuthology.orchestra.run.smithi039.stdout:5742 writesame 0x8635173 thru 0x8643c48 data_size 0x1(0xead6 bytes) 2024-09-14T06:17:39.695 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:39.694+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:39.734 INFO:teuthology.orchestra.run.smithi039.stdout:5743 writesame 0xd9091e9 thru 0xd918b36 data_size 0x21(0xf94e bytes) 2024-09-14T06:17:39.740 INFO:teuthology.orchestra.run.smithi039.stdout:5744 trunc from 0xd918b37 to 0x7dc9bdc 2024-09-14T06:17:39.752 INFO:teuthology.orchestra.run.smithi039.stdout:5746 writesame 0x3ffeb57 thru 0x400bbeb data_size 0x33(0xd095 bytes) 2024-09-14T06:17:39.756 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:39.761 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:39.762 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:39.772 INFO:teuthology.orchestra.run.smithi039.stdout:5750 write 0xe4c15b7 thru 0xe4cd434 (0xbe7e bytes) 2024-09-14T06:17:39.775 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:39.775 INFO:teuthology.orchestra.run.smithi039.stdout:5752 read 0x5a7a767 thru 0x5a803a8 (0x5c42 bytes) 2024-09-14T06:17:39.776 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:39.773+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:39.776 INFO:teuthology.orchestra.run.smithi039.stdout:5754 writesame 0xd2ccfb4 thru 0xd2db4c4 data_size 0xb(0xe511 bytes) 2024-09-14T06:17:39.778 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:17:39.780 INFO:teuthology.orchestra.run.smithi039.stdout:5756 write 0xd7ad2f8 thru 0xd7af4ef (0x21f8 bytes) 2024-09-14T06:17:39.783 INFO:teuthology.orchestra.run.smithi039.stdout:5757 read 0x8e797ff thru 0x8e8176f (0x7f71 bytes) 2024-09-14T06:17:39.783 INFO:teuthology.orchestra.run.smithi039.stdout:5758 writesame 0x5b8f4cc thru 0x5b93147 data_size 0x4f(0x3c7c bytes) 2024-09-14T06:17:39.786 INFO:teuthology.orchestra.run.smithi039.stdout:5759 writesame 0xcc89daf thru 0xcc93014 data_size 0x7(0x9266 bytes) 2024-09-14T06:17:39.793 INFO:teuthology.orchestra.run.smithi039.stdout:5760 write 0x59c084 thru 0x5a9fce (0xdf4b bytes) 2024-09-14T06:17:39.797 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:39.798 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:17:39.799 INFO:teuthology.orchestra.run.smithi039.stdout:5761 read 0xbcd5fc4 thru 0xbcdddb8 (0x7df5 bytes) 2024-09-14T06:17:39.800 INFO:teuthology.orchestra.run.smithi039.stdout:5762 punch from 0x95e6d4f to 0x95e78eb, (0xb9c bytes) 2024-09-14T06:17:39.818 INFO:teuthology.orchestra.run.smithi039.stdout:5763 trunc from 0xe4cd435 to 0x101040e 2024-09-14T06:17:39.826 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-14T06:17:38.406361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1293915938,"num_objects":1540,"num_object_clones":268,"num_object_copies":3080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1540,"num_whiteouts":388,"num_read":95383,"num_read_kb":737187,"num_write":14880,"num_write_kb":399921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1127,"num_bytes_recovered":140497553,"num_keys_recovered":151,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12058,"ondisk_log_size":12058,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":409556,"kb_used_data":76760,"kb_used_omap":12,"kb_used_meta":332595,"kb_avail":754565164,"statfs":{"total":773094113280,"available":772674727936,"internally_reserved":0,"allocated":78602240,"data_stored":691950984,"data_compressed":18840838,"data_compressed_allocated":46493696,"data_compressed_original":667286968,"omap_allocated":12752,"internal_metadata":340577840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1,1,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6459581,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":587,"num_read_kb":13524,"num_write":171,"num_write_kb":1262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":580,"num_bytes_recovered":66691871,"num_keys_recovered":71,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001074"},"pg_stats":[{"pgid":"3.1f","version":"80'368","reported_seq":3121,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.216358+0000","last_change":"2024-09-14T06:17:02.195565+0000","last_active":"2024-09-14T06:17:36.216358+0000","last_peered":"2024-09-14T06:17:36.216358+0000","last_clean":"2024-09-14T06:17:36.216358+0000","last_became_active":"2024-09-14T06:17:02.193313+0000","last_became_peered":"2024-09-14T06:17:02.193313+0000","last_unstale":"2024-09-14T06:17:36.216358+0000","last_undegraded":"2024-09-14T06:17:36.216358+0000","last_fullsized":"2024-09-14T06:17:36.216358+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:51:34.761417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010466430000000001,"stat_sum":{"num_bytes":54336702,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":10,"num_read":3747,"num_read_kb":52933,"num_write":447,"num_write_kb":19145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"79'290","reported_seq":4067,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.216386+0000","last_change":"2024-09-14T06:16:27.310639+0000","last_active":"2024-09-14T06:17:36.216386+0000","last_peered":"2024-09-14T06:17:36.216386+0000","last_clean":"2024-09-14T06:17:36.216386+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:17:36.216386+0000","last_undegraded":"2024-09-14T06:17:36.216386+0000","last_fullsized":"2024-09-14T06:17:36.216386+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:11:42.513574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095849699999999999,"stat_sum":{"num_bytes":62067820,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":15,"num_read":4725,"num_read_kb":42805,"num_write":282,"num_write_kb":15233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"83'912","reported_seq":1980,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-14T06:17:36.822844+0000","last_change":"2024-09-14T06:17:37.647589+0000","last_active":"2024-09-14T06:17:37.647589+0000","last_peered":"2024-09-14T06:17:36.822844+0000","last_clean":"2024-09-14T06:17:33.563657+0000","last_became_active":"2024-09-14T06:17:35.308894+0000","last_became_peered":"2024-09-14T06:17:35.308894+0000","last_unstale":"2024-09-14T06:17:36.822844+0000","last_undegraded":"2024-09-14T06:17:36.822844+0000","last_fullsized":"2024-09-14T06:17:35.213591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":912,"log_dups_size":0,"ondisk_log_size":912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001319401,"stat_sum":{"num_bytes":31291917,"num_objects":48,"num_object_clones":7,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":8,"num_read":4651,"num_read_kb":13930,"num_write":1382,"num_write_kb":8348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10785266,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"80'819","reported_seq":2442,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.224670+0000","last_change":"2024-09-14T06:16:22.201834+0000","last_active":"2024-09-14T06:17:37.224670+0000","last_peered":"2024-09-14T06:17:37.224670+0000","last_clean":"2024-09-14T06:17:37.224670+0000","last_became_active":"2024-09-14T06:16:20.190215+0000","last_became_peered":"2024-09-14T06:16:20.190215+0000","last_unstale":"2024-09-14T06:17:37.224670+0000","last_undegraded":"2024-09-14T06:17:37.224670+0000","last_fullsized":"2024-09-14T06:17:37.224670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:08:24.896344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026065160000000001,"stat_sum":{"num_bytes":25261213,"num_objects":50,"num_object_clones":7,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":12,"num_read":5499,"num_read_kb":9520,"num_write":1216,"num_write_kb":5987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"79'291","reported_seq":3511,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.208602+0000","last_change":"2024-09-14T06:16:22.193116+0000","last_active":"2024-09-14T06:17:36.208602+0000","last_peered":"2024-09-14T06:17:36.208602+0000","last_clean":"2024-09-14T06:17:36.208602+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:17:36.208602+0000","last_undegraded":"2024-09-14T06:17:36.208602+0000","last_fullsized":"2024-09-14T06:17:36.208602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:17:00.634226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014745120000000001,"stat_sum":{"num_bytes":52171639,"num_objects":51,"num_object_clones":9,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":13,"num_read":3322,"num_read_kb":39042,"num_write":328,"num_write_kb":25723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"79'468","reported_seq":2641,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.224338+0000","last_change":"2024-09-14T06:16:22.201461+0000","last_active":"2024-09-14T06:17:37.224338+0000","last_peered":"2024-09-14T06:17:37.224338+0000","last_clean":"2024-09-14T06:17:37.224338+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:17:37.224338+0000","last_undegraded":"2024-09-14T06:17:37.224338+0000","last_fullsized":"2024-09-14T06:17:37.224338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:21:54.740165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002267542,"stat_sum":{"num_bytes":12869368,"num_objects":53,"num_object_clones":8,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":22,"num_read":3546,"num_read_kb":12619,"num_write":607,"num_write_kb":7500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"80'449","reported_seq":1754,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.208298+0000","last_change":"2024-09-14T06:16:22.191773+0000","last_active":"2024-09-14T06:17:36.208298+0000","last_peered":"2024-09-14T06:17:36.208298+0000","last_clean":"2024-09-14T06:17:36.208298+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:17:36.208298+0000","last_undegraded":"2024-09-14T06:17:36.208298+0000","last_fullsized":"2024-09-14T06:17:36.208298+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:45:42.492916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073833899999999997,"stat_sum":{"num_bytes":35400414,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":2661,"num_read_kb":19057,"num_write":647,"num_write_kb":12862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"80'307","reported_seq":2584,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.216288+0000","last_change":"2024-09-14T06:16:22.205296+0000","last_active":"2024-09-14T06:17:36.216288+0000","last_peered":"2024-09-14T06:17:36.216288+0000","last_clean":"2024-09-14T06:17:36.216288+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:17:36.216288+0000","last_undegraded":"2024-09-14T06:17:36.216288+0000","last_fullsized":"2024-09-14T06:17:36.216288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:57:31.710099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012474280000000001,"stat_sum":{"num_bytes":44544444,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":11,"num_read":3410,"num_read_kb":37351,"num_write":297,"num_write_kb":32929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"83'232","reported_seq":1208,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-14T06:17:36.041773+0000","last_change":"2024-09-14T06:17:36.217871+0000","last_active":"2024-09-14T06:17:36.217871+0000","last_peered":"2024-09-14T06:17:36.041773+0000","last_clean":"2024-09-14T06:17:33.574978+0000","last_became_active":"2024-09-14T06:17:35.296774+0000","last_became_peered":"2024-09-14T06:17:35.296774+0000","last_unstale":"2024-09-14T06:17:36.041773+0000","last_undegraded":"2024-09-14T06:17:36.041773+0000","last_fullsized":"2024-09-14T06:17:35.210218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001106627,"stat_sum":{"num_bytes":24170577,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":16,"num_read":930,"num_read_kb":17047,"num_write":246,"num_write_kb":13809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":23467377,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"80'354","reported_seq":1995,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.237257+0000","last_change":"2024-09-14T06:17:37.236314+0000","last_active":"2024-09-14T06:17:37.237257+0000","last_peered":"2024-09-14T06:17:37.237257+0000","last_clean":"2024-09-14T06:17:37.237257+0000","last_became_active":"2024-09-14T06:17:37.235311+0000","last_became_peered":"2024-09-14T06:17:37.235311+0000","last_unstale":"2024-09-14T06:17:37.237257+0000","last_undegraded":"2024-09-14T06:17:37.237257+0000","last_fullsized":"2024-09-14T06:17:37.237257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:09.235644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001557389,"stat_sum":{"num_bytes":57848052,"num_objects":49,"num_object_clones":10,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":11,"num_read":2236,"num_read_kb":16285,"num_write":452,"num_write_kb":10272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":6785399,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"79'348","reported_seq":3395,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.233084+0000","last_change":"2024-09-14T06:16:22.194304+0000","last_active":"2024-09-14T06:17:37.233084+0000","last_peered":"2024-09-14T06:17:37.233084+0000","last_clean":"2024-09-14T06:17:37.233084+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:17:37.233084+0000","last_undegraded":"2024-09-14T06:17:37.233084+0000","last_fullsized":"2024-09-14T06:17:37.233084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:32:31.047615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019733379999999998,"stat_sum":{"num_bytes":71775725,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":7,"num_read":3899,"num_read_kb":64430,"num_write":448,"num_write_kb":22209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"80'246","reported_seq":1643,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.235834+0000","last_change":"2024-09-14T06:17:37.232535+0000","last_active":"2024-09-14T06:17:37.235834+0000","last_peered":"2024-09-14T06:17:37.235834+0000","last_clean":"2024-09-14T06:17:37.235834+0000","last_became_active":"2024-09-14T06:17:37.231244+0000","last_became_peered":"2024-09-14T06:17:37.231244+0000","last_unstale":"2024-09-14T06:17:37.235834+0000","last_undegraded":"2024-09-14T06:17:37.235834+0000","last_fullsized":"2024-09-14T06:17:37.235834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:22:59.070149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001465247,"stat_sum":{"num_bytes":61166345,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":10,"num_read":2620,"num_read_kb":25820,"num_write":230,"num_write_kb":15728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":32506492,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.223820+0000","last_change":"2024-09-14T06:17:34.200709+0000","last_active":"2024-09-14T06:17:37.223820+0000","last_peered":"2024-09-14T06:17:37.223820+0000","last_clean":"2024-09-14T06:17:37.223820+0000","last_became_active":"2024-09-14T06:17:34.200238+0000","last_became_peered":"2024-09-14T06:17:34.200238+0000","last_unstale":"2024-09-14T06:17:37.223820+0000","last_undegraded":"2024-09-14T06:17:37.223820+0000","last_fullsized":"2024-09-14T06:17:37.223820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:34.864695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"79'364","reported_seq":908,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.209111+0000","last_change":"2024-09-14T06:16:22.192972+0000","last_active":"2024-09-14T06:17:36.209111+0000","last_peered":"2024-09-14T06:17:36.209111+0000","last_clean":"2024-09-14T06:17:36.209111+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:17:36.209111+0000","last_undegraded":"2024-09-14T06:17:36.209111+0000","last_fullsized":"2024-09-14T06:17:36.209111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:56:10.514649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086504599999999996,"stat_sum":{"num_bytes":29669358,"num_objects":48,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":10,"num_read":1315,"num_read_kb":9524,"num_write":422,"num_write_kb":9131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.209330+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:17:36.209330+0000","last_peered":"2024-09-14T06:17:36.209330+0000","last_clean":"2024-09-14T06:17:36.209330+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:17:36.209330+0000","last_undegraded":"2024-09-14T06:17:36.209330+0000","last_fullsized":"2024-09-14T06:17:36.209330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"80'257","reported_seq":2796,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.208812+0000","last_change":"2024-09-14T06:16:22.192683+0000","last_active":"2024-09-14T06:17:36.208812+0000","last_peered":"2024-09-14T06:17:36.208812+0000","last_clean":"2024-09-14T06:17:36.208812+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:17:36.208812+0000","last_undegraded":"2024-09-14T06:17:36.208812+0000","last_fullsized":"2024-09-14T06:17:36.208812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:04:04.548750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001328716,"stat_sum":{"num_bytes":31086743,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":11,"num_read":3001,"num_read_kb":27472,"num_write":249,"num_write_kb":12571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.222845+0000","last_change":"2024-09-14T06:17:34.196668+0000","last_active":"2024-09-14T06:17:37.222845+0000","last_peered":"2024-09-14T06:17:37.222845+0000","last_clean":"2024-09-14T06:17:37.222845+0000","last_became_active":"2024-09-14T06:17:34.196072+0000","last_became_peered":"2024-09-14T06:17:34.196072+0000","last_unstale":"2024-09-14T06:17:37.222845+0000","last_undegraded":"2024-09-14T06:17:37.222845+0000","last_fullsized":"2024-09-14T06:17:37.222845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:18:45.616834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"80'258","reported_seq":1546,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.218175+0000","last_change":"2024-09-14T06:16:22.213152+0000","last_active":"2024-09-14T06:17:36.218175+0000","last_peered":"2024-09-14T06:17:36.218175+0000","last_clean":"2024-09-14T06:17:36.218175+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:17:36.218175+0000","last_undegraded":"2024-09-14T06:17:36.218175+0000","last_fullsized":"2024-09-14T06:17:36.218175+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:28:34.843601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057590699999999996,"stat_sum":{"num_bytes":38574596,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":8,"num_read":1529,"num_read_kb":12688,"num_write":240,"num_write_kb":8157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":144,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.216216+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:17:36.216216+0000","last_peered":"2024-09-14T06:17:36.216216+0000","last_clean":"2024-09-14T06:17:36.216216+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:17:36.216216+0000","last_undegraded":"2024-09-14T06:17:36.216216+0000","last_fullsized":"2024-09-14T06:17:36.216216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"79'215","reported_seq":1434,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.218923+0000","last_change":"2024-09-14T06:16:33.548583+0000","last_active":"2024-09-14T06:17:36.218923+0000","last_peered":"2024-09-14T06:17:36.218923+0000","last_clean":"2024-09-14T06:17:36.218923+0000","last_became_active":"2024-09-14T06:16:33.547925+0000","last_became_peered":"2024-09-14T06:16:33.547925+0000","last_unstale":"2024-09-14T06:17:36.218923+0000","last_undegraded":"2024-09-14T06:17:36.218923+0000","last_fullsized":"2024-09-14T06:17:36.218923+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:23:39.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089329700000000004,"stat_sum":{"num_bytes":69139655,"num_objects":49,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":16,"num_read":1310,"num_read_kb":21229,"num_write":226,"num_write_kb":9950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"79'179","reported_seq":636,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.230463+0000","last_change":"2024-09-14T06:16:27.282745+0000","last_active":"2024-09-14T06:17:37.230463+0000","last_peered":"2024-09-14T06:17:37.230463+0000","last_clean":"2024-09-14T06:17:37.230463+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:17:37.230463+0000","last_undegraded":"2024-09-14T06:17:37.230463+0000","last_fullsized":"2024-09-14T06:17:37.230463+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:53:56.212817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002354358,"stat_sum":{"num_bytes":38932892,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":335,"num_read_kb":3987,"num_write":143,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.220953+0000","last_change":"2024-09-14T06:17:34.203977+0000","last_active":"2024-09-14T06:17:37.220953+0000","last_peered":"2024-09-14T06:17:37.220953+0000","last_clean":"2024-09-14T06:17:37.220953+0000","last_became_active":"2024-09-14T06:17:34.199244+0000","last_became_peered":"2024-09-14T06:17:34.199244+0000","last_unstale":"2024-09-14T06:17:37.220953+0000","last_undegraded":"2024-09-14T06:17:37.220953+0000","last_fullsized":"2024-09-14T06:17:37.220953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:21.934094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"80'351","reported_seq":1573,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.216108+0000","last_change":"2024-09-14T06:16:22.205051+0000","last_active":"2024-09-14T06:17:36.216108+0000","last_peered":"2024-09-14T06:17:36.216108+0000","last_clean":"2024-09-14T06:17:36.216108+0000","last_became_active":"2024-09-14T06:16:20.196215+0000","last_became_peered":"2024-09-14T06:16:20.196215+0000","last_unstale":"2024-09-14T06:17:36.216108+0000","last_undegraded":"2024-09-14T06:17:36.216108+0000","last_fullsized":"2024-09-14T06:17:36.216108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:42:24.322885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001116559,"stat_sum":{"num_bytes":33594565,"num_objects":40,"num_object_clones":3,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":9,"num_read":2021,"num_read_kb":18817,"num_write":467,"num_write_kb":11041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":144,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.221860+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:17:36.221860+0000","last_peered":"2024-09-14T06:17:36.221860+0000","last_clean":"2024-09-14T06:17:36.221860+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:17:36.221860+0000","last_undegraded":"2024-09-14T06:17:36.221860+0000","last_fullsized":"2024-09-14T06:17:36.221860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'450","reported_seq":1870,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.218376+0000","last_change":"2024-09-14T06:16:27.316499+0000","last_active":"2024-09-14T06:17:36.218376+0000","last_peered":"2024-09-14T06:17:36.218376+0000","last_clean":"2024-09-14T06:17:36.218376+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:17:36.218376+0000","last_undegraded":"2024-09-14T06:17:36.218376+0000","last_fullsized":"2024-09-14T06:17:36.218376+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:10:23.624742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052773999999999996,"stat_sum":{"num_bytes":13151989,"num_objects":36,"num_object_clones":4,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":9,"num_read":2798,"num_read_kb":8680,"num_write":652,"num_write_kb":5492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":146,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.220489+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:17:37.220489+0000","last_peered":"2024-09-14T06:17:37.220489+0000","last_clean":"2024-09-14T06:17:37.220489+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:17:37.220489+0000","last_undegraded":"2024-09-14T06:17:37.220489+0000","last_fullsized":"2024-09-14T06:17:37.220489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"80'275","reported_seq":1430,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.208345+0000","last_change":"2024-09-14T06:16:27.302045+0000","last_active":"2024-09-14T06:17:36.208345+0000","last_peered":"2024-09-14T06:17:36.208345+0000","last_clean":"2024-09-14T06:17:36.208345+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:17:36.208345+0000","last_undegraded":"2024-09-14T06:17:36.208345+0000","last_fullsized":"2024-09-14T06:17:36.208345+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:33:28.609283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060103200000000004,"stat_sum":{"num_bytes":43182018,"num_objects":50,"num_object_clones":6,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":8,"num_read":1700,"num_read_kb":22421,"num_write":262,"num_write_kb":10461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":167,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.221191+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:17:37.221191+0000","last_peered":"2024-09-14T06:17:37.221191+0000","last_clean":"2024-09-14T06:17:37.221191+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:17:37.221191+0000","last_undegraded":"2024-09-14T06:17:37.221191+0000","last_fullsized":"2024-09-14T06:17:37.221191+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":237,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.223709+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:17:37.223709+0000","last_peered":"2024-09-14T06:17:37.223709+0000","last_clean":"2024-09-14T06:17:37.223709+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:17:37.223709+0000","last_undegraded":"2024-09-14T06:17:37.223709+0000","last_fullsized":"2024-09-14T06:17:37.223709+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"79'379","reported_seq":1315,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.208242+0000","last_change":"2024-09-14T06:16:22.191260+0000","last_active":"2024-09-14T06:17:36.208242+0000","last_peered":"2024-09-14T06:17:36.208242+0000","last_clean":"2024-09-14T06:17:36.208242+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:17:36.208242+0000","last_undegraded":"2024-09-14T06:17:36.208242+0000","last_fullsized":"2024-09-14T06:17:36.208242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:05:10.411581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047498199999999999,"stat_sum":{"num_bytes":12407027,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":14,"num_read":2001,"num_read_kb":5657,"num_write":539,"num_write_kb":6840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"80'343","reported_seq":1571,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-14T06:17:36.660240+0000","last_change":"2024-09-14T06:17:37.647504+0000","last_active":"2024-09-14T06:17:37.647504+0000","last_peered":"2024-09-14T06:17:36.660240+0000","last_clean":"2024-09-14T06:17:33.309444+0000","last_became_active":"2024-09-14T06:17:35.295979+0000","last_became_peered":"2024-09-14T06:17:35.295979+0000","last_unstale":"2024-09-14T06:17:36.660240+0000","last_undegraded":"2024-09-14T06:17:36.660240+0000","last_fullsized":"2024-09-14T06:17:36.660240+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418299,"stat_sum":{"num_bytes":42339757,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":8,"num_read":2182,"num_read_kb":13624,"num_write":509,"num_write_kb":14348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8477378,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"80'263","reported_seq":1963,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.220914+0000","last_change":"2024-09-14T06:16:22.323725+0000","last_active":"2024-09-14T06:17:37.220914+0000","last_peered":"2024-09-14T06:17:37.220914+0000","last_clean":"2024-09-14T06:17:37.220914+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:17:37.220914+0000","last_undegraded":"2024-09-14T06:17:37.220914+0000","last_fullsized":"2024-09-14T06:17:37.220914+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:46:42.122526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024221899999999998,"stat_sum":{"num_bytes":35230985,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":2141,"num_read_kb":16051,"num_write":246,"num_write_kb":13522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"80'206","reported_seq":2483,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-14T06:17:36.221929+0000","last_change":"2024-09-14T06:16:22.219322+0000","last_active":"2024-09-14T06:17:36.221929+0000","last_peered":"2024-09-14T06:17:36.221929+0000","last_clean":"2024-09-14T06:17:36.221929+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:17:36.221929+0000","last_undegraded":"2024-09-14T06:17:36.221929+0000","last_fullsized":"2024-09-14T06:17:36.221929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:01:35.004877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045017199999999999,"stat_sum":{"num_bytes":50879774,"num_objects":54,"num_object_clones":10,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":15,"num_read":2678,"num_read_kb":16597,"num_write":207,"num_write_kb":6676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"80'138","reported_seq":1383,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.231962+0000","last_change":"2024-09-14T06:16:22.195705+0000","last_active":"2024-09-14T06:17:37.231962+0000","last_peered":"2024-09-14T06:17:37.231962+0000","last_clean":"2024-09-14T06:17:37.231962+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:17:37.231962+0000","last_undegraded":"2024-09-14T06:17:37.231962+0000","last_fullsized":"2024-09-14T06:17:37.231962+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:02:15.957880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025578329999999998,"stat_sum":{"num_bytes":46202051,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":7,"num_read":1255,"num_read_kb":29057,"num_write":210,"num_write_kb":11572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"79'621","reported_seq":2305,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-14T06:17:36.334368+0000","last_change":"2024-09-14T06:17:37.647637+0000","last_active":"2024-09-14T06:17:37.647637+0000","last_peered":"2024-09-14T06:17:36.334368+0000","last_clean":"2024-09-14T06:17:31.562482+0000","last_became_active":"2024-09-14T06:17:35.308839+0000","last_became_peered":"2024-09-14T06:17:35.308839+0000","last_unstale":"2024-09-14T06:17:36.334368+0000","last_undegraded":"2024-09-14T06:17:36.334368+0000","last_fullsized":"2024-09-14T06:17:36.334368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001627671,"stat_sum":{"num_bytes":37374219,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":3733,"num_read_kb":26924,"num_write":955,"num_write_kb":20272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":29390214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"80'604","reported_seq":4564,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.232011+0000","last_change":"2024-09-14T06:16:33.547557+0000","last_active":"2024-09-14T06:17:37.232011+0000","last_peered":"2024-09-14T06:17:37.232011+0000","last_clean":"2024-09-14T06:17:37.232011+0000","last_became_active":"2024-09-14T06:16:33.546796+0000","last_became_peered":"2024-09-14T06:16:33.546796+0000","last_unstale":"2024-09-14T06:17:37.232011+0000","last_undegraded":"2024-09-14T06:17:37.232011+0000","last_fullsized":"2024-09-14T06:17:37.232011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:13:41.806382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026669160000000001,"stat_sum":{"num_bytes":48267984,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":12,"num_read":8998,"num_read_kb":50615,"num_write":581,"num_write_kb":16104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"80'577","reported_seq":2454,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.231967+0000","last_change":"2024-09-14T06:16:22.195247+0000","last_active":"2024-09-14T06:17:37.231967+0000","last_peered":"2024-09-14T06:17:37.231967+0000","last_clean":"2024-09-14T06:17:37.231967+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:17:37.231967+0000","last_undegraded":"2024-09-14T06:17:37.231967+0000","last_fullsized":"2024-09-14T06:17:37.231967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:12:24.464514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022870590000000001,"stat_sum":{"num_bytes":34980582,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":14,"num_read":4026,"num_read_kb":15547,"num_write":707,"num_write_kb":12738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"83'276","reported_seq":2164,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.223402+0000","last_change":"2024-09-14T06:16:22.200094+0000","last_active":"2024-09-14T06:17:37.223402+0000","last_peered":"2024-09-14T06:17:37.223402+0000","last_clean":"2024-09-14T06:17:37.223402+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:17:37.223402+0000","last_undegraded":"2024-09-14T06:17:37.223402+0000","last_fullsized":"2024-09-14T06:17:37.223402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:37:35.508264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011343239999999999,"stat_sum":{"num_bytes":53143462,"num_objects":56,"num_object_clones":8,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":14,"num_read":2228,"num_read_kb":47482,"num_write":277,"num_write_kb":14632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"79'331","reported_seq":1745,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-14T06:17:36.497995+0000","last_change":"2024-09-14T06:17:37.647467+0000","last_active":"2024-09-14T06:17:37.647467+0000","last_peered":"2024-09-14T06:17:36.497995+0000","last_clean":"2024-09-14T06:17:32.583854+0000","last_became_active":"2024-09-14T06:17:35.295763+0000","last_became_peered":"2024-09-14T06:17:35.295763+0000","last_unstale":"2024-09-14T06:17:36.497995+0000","last_undegraded":"2024-09-14T06:17:36.497995+0000","last_fullsized":"2024-09-14T06:17:36.497995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226324,"stat_sum":{"num_bytes":18337929,"num_objects":55,"num_object_clones":12,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":15,"num_read":2350,"num_read_kb":12803,"num_write":423,"num_write_kb":7432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":12239008,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"80'423","reported_seq":2636,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-14T06:17:35.785932+0000","last_change":"2024-09-14T06:17:36.218698+0000","last_active":"2024-09-14T06:17:36.218698+0000","last_peered":"2024-09-14T06:17:35.785932+0000","last_clean":"2024-09-14T06:17:33.356737+0000","last_became_active":"2024-09-14T06:17:35.295838+0000","last_became_peered":"2024-09-14T06:17:35.295838+0000","last_unstale":"2024-09-14T06:17:35.785932+0000","last_undegraded":"2024-09-14T06:17:35.785932+0000","last_fullsized":"2024-09-14T06:17:35.210541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082016299999999995,"stat_sum":{"num_bytes":17887056,"num_objects":47,"num_object_clones":6,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":14,"num_read":3083,"num_read_kb":10830,"num_write":525,"num_write_kb":8437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":10176554,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"79'389","reported_seq":3588,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T06:17:37.220841+0000","last_change":"2024-09-14T06:16:22.322160+0000","last_active":"2024-09-14T06:17:37.220841+0000","last_peered":"2024-09-14T06:17:37.220841+0000","last_clean":"2024-09-14T06:17:37.220841+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:17:37.220841+0000","last_undegraded":"2024-09-14T06:17:37.220841+0000","last_fullsized":"2024-09-14T06:17:37.220841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:32:32.561603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011502319999999999,"stat_sum":{"num_bytes":66038693,"num_objects":56,"num_object_clones":12,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":16,"num_read":5387,"num_read_kb":16288,"num_write":355,"num_write_kb":5570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1293325551,"num_objects":1536,"num_object_clones":268,"num_object_copies":3072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1536,"num_whiteouts":388,"num_read":95317,"num_read_kb":737132,"num_write":14777,"num_write_kb":398291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1124,"num_bytes_recovered":140497534,"num_keys_recovered":151,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":69394432,"data_stored":675316578,"data_compressed":18479156,"data_compressed_allocated":45391872,"data_compressed_original":654446008,"omap_allocated":0,"internal_metadata":0},"log_size":11983,"ondisk_log_size":11983,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41808,"kb_used_data":10624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330032,"statfs":{"total":96636764160,"available":96593952768,"internally_reserved":0,"allocated":10878976,"data_stored":94402241,"data_compressed":3232064,"data_compressed_allocated":7073792,"data_compressed_original":91463680,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38336,"kb_used_data":11304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333504,"statfs":{"total":96636764160,"available":96597508096,"internally_reserved":0,"allocated":11575296,"data_stored":99946330,"data_compressed":2315064,"data_compressed_allocated":6639616,"data_compressed_original":96134209,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80504,"kb_used_data":9060,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94291336,"statfs":{"total":96636764160,"available":96554328064,"internally_reserved":0,"allocated":9277440,"data_stored":78150955,"data_compressed":1942007,"data_compressed_allocated":5300224,"data_compressed_original":75116812,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62128,"kb_used_data":12892,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94309712,"statfs":{"total":96636764160,"available":96573145088,"internally_reserved":0,"allocated":13201408,"data_stored":123444283,"data_compressed":3225683,"data_compressed_allocated":8310784,"data_compressed_original":119406560,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41480,"kb_used_data":6120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330360,"statfs":{"total":96636764160,"available":96594288640,"internally_reserved":0,"allocated":6266880,"data_stored":50015070,"data_compressed":1716395,"data_compressed_allocated":3289088,"data_compressed_original":47828348,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542166,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48664,"kb_used_data":9156,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323176,"statfs":{"total":96636764160,"available":96586932224,"internally_reserved":0,"allocated":9375744,"data_stored":79624763,"data_compressed":2108145,"data_compressed_allocated":5206016,"data_compressed_original":76465051,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48900,"kb_used_data":9388,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322940,"statfs":{"total":96636764160,"available":96586690560,"internally_reserved":0,"allocated":9613312,"data_stored":91514609,"data_compressed":2490644,"data_compressed_allocated":6201344,"data_compressed_original":89055638,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542167,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47736,"kb_used_data":8216,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324104,"statfs":{"total":96636764160,"available":96587882496,"internally_reserved":0,"allocated":8413184,"data_stored":74852733,"data_compressed":1810836,"data_compressed_allocated":4472832,"data_compressed_original":71816670,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10969088,"data_stored":111135700,"data_compressed":2884381,"data_compressed_allocated":7290880,"data_compressed_original":107712480,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8220672,"data_stored":77102512,"data_compressed":1931817,"data_compressed_allocated":5259264,"data_compressed_original":74543372,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10592256,"data_stored":99494229,"data_compressed":2315064,"data_compressed_allocated":6639616,"data_compressed_original":96134209,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9891840,"data_stored":93950121,"data_compressed":3232064,"data_compressed_allocated":7073792,"data_compressed_original":91463680,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7442432,"data_stored":74407040,"data_compressed":1810836,"data_compressed_allocated":4472832,"data_compressed_original":71816670,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8581120,"data_stored":90478548,"data_compressed":2480454,"data_compressed_allocated":6160384,"data_compressed_original":88482198,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":79179051,"data_compressed":2108145,"data_compressed_allocated":5206016,"data_compressed_original":76465051,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5296128,"data_stored":49569377,"data_compressed":1716395,"data_compressed_allocated":3289088,"data_compressed_original":47828348,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:17:39.831 INFO:teuthology.orchestra.run.smithi039.stdout:5765 read 0xf48951 thru 0xf544c1 (0xbb71 bytes) 2024-09-14T06:17:39.832 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:39.832 INFO:teuthology.orchestra.run.smithi039.stdout:5767 punch from 0x543d17 to 0x550c83, (0xcf6c bytes) 2024-09-14T06:17:39.832 INFO:teuthology.orchestra.run.smithi039.stdout:5768 write 0xe240a9 thru 0xe2e228 (0xa180 bytes) 2024-09-14T06:17:39.858 INFO:teuthology.orchestra.run.smithi039.stdout:5769 writesame 0xd08bf6b thru 0xd08dc96 data_size 0x2(0x1d2c bytes) 2024-09-14T06:17:39.861 INFO:teuthology.orchestra.run.smithi039.stdout:5770 read 0x5f71d8b thru 0x5f7a044 (0x82ba bytes) 2024-09-14T06:17:39.862 INFO:teuthology.orchestra.run.smithi039.stdout:5772 read 0x56b3fa thru 0x576897 (0xb49e bytes) 2024-09-14T06:17:39.863 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:39.863 INFO:teuthology.orchestra.run.smithi039.stdout:5774 read 0x7faf0f3 thru 0x7fba9d5 (0xb8e3 bytes) 2024-09-14T06:17:39.876 INFO:teuthology.orchestra.run.smithi039.stdout:5775 trunc from 0xd08dc97 to 0x360c230 2024-09-14T06:17:39.876 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:39.874+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:39.887 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:39.903 INFO:teuthology.orchestra.run.smithi039.stdout:5777 writesame 0x9b783e0 thru 0x9b7d894 data_size 0x1(0x54b5 bytes) 2024-09-14T06:17:39.907 INFO:teuthology.orchestra.run.smithi039.stdout:5778 trunc from 0x9b7d895 to 0x88400c1 2024-09-14T06:17:39.933 INFO:teuthology.orchestra.run.smithi039.stdout:5779 write 0xb460e3b thru 0xb470345 (0xf50b bytes) 2024-09-14T06:17:39.936 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:39.936 INFO:teuthology.orchestra.run.smithi039.stdout:5784 read 0x4359f29 thru 0x435dbd3 (0x3cab bytes) 2024-09-14T06:17:39.937 INFO:teuthology.orchestra.run.smithi039.stdout:5785 writesame 0x9da6dff thru 0x9db010f data_size 0x1(0x9311 bytes) 2024-09-14T06:17:39.938 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:39.938 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:39.952 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:17:39.977 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:39.973+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:39.989 INFO:teuthology.orchestra.run.smithi039.stdout:5786 read 0x5dcd679 thru 0x5dd3c82 (0x660a bytes) 2024-09-14T06:17:39.989 INFO:teuthology.orchestra.run.smithi039.stdout:5787 write 0x43eea48 thru 0x43f772d (0x8ce6 bytes) 2024-09-14T06:17:40.005 INFO:teuthology.orchestra.run.smithi039.stdout:5789 write 0xccaa340 thru 0xccaa867 (0x528 bytes) 2024-09-14T06:17:40.007 INFO:teuthology.orchestra.run.smithi039.stdout:5790 write 0x895242d thru 0x8959cba (0x788e bytes) 2024-09-14T06:17:40.011 INFO:teuthology.orchestra.run.smithi039.stdout:5791 read 0xb103354 thru 0xb1038ee (0x59b bytes) 2024-09-14T06:17:40.027 INFO:teuthology.orchestra.run.smithi039.stdout:5792 trunc from 0xccaa868 to 0x56c7b9f 2024-09-14T06:17:40.057 INFO:teuthology.orchestra.run.smithi039.stdout:5794 trunc from 0x56c7b9f to 0xe0c8c5d 2024-09-14T06:17:40.066 INFO:teuthology.orchestra.run.smithi039.stdout:5799 punch from 0xdd102db to 0xdd1176f, (0x1494 bytes) 2024-09-14T06:17:40.078 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:40.073+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:40.079 INFO:teuthology.orchestra.run.smithi039.stdout:5800 trunc from 0xe0c8c5d to 0x7637c88 2024-09-14T06:17:40.114 INFO:teuthology.orchestra.run.smithi039.stdout:5801 write 0xe20b722 thru 0xe21ab7c (0xf45b bytes) 2024-09-14T06:17:40.118 INFO:teuthology.orchestra.run.smithi039.stdout:5802 read 0x8d36885 thru 0x8d45947 (0xf0c3 bytes) 2024-09-14T06:17:40.123 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:40.123 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:40.134 INFO:teuthology.orchestra.run.smithi039.stdout:5803 trunc from 0xe21ab7d to 0x5fa44d5 2024-09-14T06:17:40.144 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:17:40.161 INFO:teuthology.orchestra.run.smithi039.stdout:5804 writesame 0x7019f86 thru 0x7020b28 data_size 0x21(0x6ba3 bytes) 2024-09-14T06:17:40.186 INFO:teuthology.orchestra.run.smithi039.stdout:5806 write 0xe002307 thru 0xe00aae1 (0x87db bytes) 2024-09-14T06:17:40.191 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:40.189+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:40.192 INFO:teuthology.orchestra.run.smithi039.stdout:5810 read 0xdb61b7b thru 0xdb6b243 (0x96c9 bytes) 2024-09-14T06:17:40.193 INFO:teuthology.orchestra.run.smithi039.stdout:5812 read 0x41a3658 thru 0x41a7088 (0x3a31 bytes) 2024-09-14T06:17:40.193 INFO:teuthology.orchestra.run.smithi039.stdout:5813 write 0xa58e787 thru 0xa592594 (0x3e0e bytes) 2024-09-14T06:17:40.213 INFO:teuthology.orchestra.run.smithi039.stdout:5814 trunc from 0xe00aae2 to 0x5defb92 2024-09-14T06:17:40.227 INFO:teuthology.orchestra.run.smithi039.stdout:5815 read 0xcaf838 thru 0xcb34dc (0x3ca5 bytes) 2024-09-14T06:17:40.228 INFO:teuthology.orchestra.run.smithi039.stdout:5816 write 0x51e8b0b thru 0x51f0c3a (0x8130 bytes) 2024-09-14T06:17:40.250 INFO:teuthology.orchestra.run.smithi039.stdout:5817 write 0xd056edc thru 0xd05dcc2 (0x6de7 bytes) 2024-09-14T06:17:40.262 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:17:40.268 INFO:teuthology.orchestra.run.smithi039.stdout:5818 write 0xeb206fe thru 0xeb2c8ea (0xc1ed bytes) 2024-09-14T06:17:40.271 INFO:teuthology.orchestra.run.smithi039.stdout:5819 write 0xe836a05 thru 0xe83c33d (0x5939 bytes) 2024-09-14T06:17:40.273 INFO:teuthology.orchestra.run.smithi039.stdout:5820 writesame 0xd479f65 thru 0xd47e4c9 data_size 0x55(0x4565 bytes) 2024-09-14T06:17:40.276 INFO:teuthology.orchestra.run.smithi039.stdout:5821 writesame 0xc83cf95 thru 0xc84af8e data_size 0x1(0xdffa bytes) 2024-09-14T06:17:40.279 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:40.277+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:40.290 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:40.290 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:40.311 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:17:40.353 INFO:teuthology.orchestra.run.smithi039.stdout:5822 trunc from 0xeb2c8eb to 0x25724ff 2024-09-14T06:17:40.375 INFO:teuthology.orchestra.run.smithi039.stdout:5823 read 0xd8fad2 thru 0xd988d1 (0x8e00 bytes) 2024-09-14T06:17:40.380 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:40.378+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:40.393 INFO:teuthology.orchestra.run.smithi039.stdout:5825 writesame 0x58b07d3 thru 0x58b6fd4 data_size 0x1(0x6802 bytes) 2024-09-14T06:17:40.405 INFO:teuthology.orchestra.run.smithi039.stdout:5826 write 0x76ddffb thru 0x76e8eed (0xaef3 bytes) 2024-09-14T06:17:40.407 INFO:teuthology.orchestra.run.smithi039.stdout:5827 read 0x385e649 thru 0x38619f2 (0x33aa bytes) 2024-09-14T06:17:40.408 INFO:teuthology.orchestra.run.smithi039.stdout:5828 punch from 0x4e6fae6 to 0x4e72508, (0x2a22 bytes) 2024-09-14T06:17:40.408 INFO:teuthology.orchestra.run.smithi039.stdout:5829 writesame 0x5641d6d thru 0x564b4fb data_size 0x1b(0x978f bytes) 2024-09-14T06:17:40.410 INFO:teuthology.orchestra.run.smithi039.stdout:5830 writesame 0x13572b3 thru 0x13655c0 data_size 0x1(0xe30e bytes) 2024-09-14T06:17:40.456 INFO:teuthology.orchestra.run.smithi039.stdout:5831 write 0x1da8c5c thru 0x1daec57 (0x5ffc bytes) 2024-09-14T06:17:40.480 INFO:teuthology.orchestra.run.smithi039.stdout:5833 write 0xbc4f16f thru 0xbc5681a (0x76ac bytes) 2024-09-14T06:17:40.481 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:40.478+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:40.483 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:40.483 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:40.498 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:17:40.500 INFO:teuthology.orchestra.run.smithi039.stdout:5834 writesame 0xda6b79d thru 0xda70d0e data_size 0x1(0x5572 bytes) 2024-09-14T06:17:40.500 INFO:teuthology.orchestra.run.smithi039.stdout:5835 write 0xff0054 thru 0xff504f (0x4ffc bytes) 2024-09-14T06:17:40.504 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:40.504 INFO:teuthology.orchestra.run.smithi039.stdout:5837 write 0x7a3d511 thru 0x7a4bf14 (0xea04 bytes) 2024-09-14T06:17:40.506 INFO:teuthology.orchestra.run.smithi039.stdout:5838 read 0x2904958 thru 0x2907e1b (0x34c4 bytes) 2024-09-14T06:17:40.506 INFO:teuthology.orchestra.run.smithi039.stdout:5841 writesame 0x808dfaa thru 0x808f73c data_size 0x11(0x1793 bytes) 2024-09-14T06:17:40.510 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:40.510 INFO:teuthology.orchestra.run.smithi039.stdout:5843 write 0x71bcb74 thru 0x71c9926 (0xcdb3 bytes) 2024-09-14T06:17:40.513 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:40.513 INFO:teuthology.orchestra.run.smithi039.stdout:5847 punch from 0xc179eaf to 0xc17cc1b, (0x2d6c bytes) 2024-09-14T06:17:40.514 INFO:teuthology.orchestra.run.smithi039.stdout:5849 punch from 0xdbc1c0 to 0xdbf25b, (0x309b bytes) 2024-09-14T06:17:40.514 INFO:teuthology.orchestra.run.smithi039.stdout:5850 read 0xc33ce59 thru 0xc3464a1 (0x9649 bytes) 2024-09-14T06:17:40.515 INFO:teuthology.orchestra.run.smithi039.stdout:5851 punch from 0x991d961 to 0x992a7bf, (0xce5e bytes) 2024-09-14T06:17:40.540 INFO:teuthology.orchestra.run.smithi039.stdout:5852 trunc from 0xda70d0f to 0x1f14013 2024-09-14T06:17:40.581 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:40.577+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:40.583 INFO:teuthology.orchestra.run.smithi039.stdout:5858 writesame 0x8e92807 thru 0x8ea0499 data_size 0x1(0xdc93 bytes) 2024-09-14T06:17:40.597 INFO:teuthology.orchestra.run.smithi039.stdout:5859 trunc from 0x8ea049a to 0x3eb4d47 2024-09-14T06:17:40.615 INFO:teuthology.orchestra.run.smithi039.stdout:5860 read 0x1ada74c thru 0x1ae3c0b (0x94c0 bytes) 2024-09-14T06:17:40.616 INFO:teuthology.orchestra.run.smithi039.stdout:5861 read 0x43b8e8 thru 0x44918c (0xd8a5 bytes) 2024-09-14T06:17:40.617 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:40.618 INFO:teuthology.orchestra.run.smithi039.stdout:5865 read 0x361b5a2 thru 0x361bc15 (0x674 bytes) 2024-09-14T06:17:40.626 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:40.626 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:40.629 INFO:teuthology.orchestra.run.smithi039.stdout:5866 writesame 0x5174458 thru 0x5183451 data_size 0x9(0xeffa bytes) 2024-09-14T06:17:40.634 INFO:teuthology.orchestra.run.smithi039.stdout:5867 trunc from 0x5183452 to 0x425f737 2024-09-14T06:17:40.642 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:17:40.652 INFO:teuthology.orchestra.run.smithi039.stdout:5868 trunc from 0x425f737 to 0x70a404e 2024-09-14T06:17:40.661 INFO:teuthology.orchestra.run.smithi039.stdout:5869 read 0x49e7665 thru 0x49f7299 (0xfc35 bytes) 2024-09-14T06:17:40.662 INFO:teuthology.orchestra.run.smithi039.stdout:5870 write 0x2f49823 thru 0x2f548ea (0xb0c8 bytes) 2024-09-14T06:17:40.677 INFO:teuthology.orchestra.run.smithi039.stdout:5871 trunc from 0x70a404e to 0xcd238bb 2024-09-14T06:17:40.682 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:40.678+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:40.686 INFO:teuthology.orchestra.run.smithi039.stdout:5873 writesame 0x4a0aba thru 0x4ae31a data_size 0x1(0xd861 bytes) 2024-09-14T06:17:40.731 INFO:teuthology.orchestra.run.smithi039.stdout:5874 write 0xe50738 thru 0xe520c3 (0x198c bytes) 2024-09-14T06:17:40.735 INFO:teuthology.orchestra.run.smithi039.stdout:5875 punch from 0x6670bf5 to 0x66786b2, (0x7abd bytes) 2024-09-14T06:17:40.735 INFO:teuthology.orchestra.run.smithi039.stdout:5878 read 0x8f8b65d thru 0x8f95223 (0x9bc7 bytes) 2024-09-14T06:17:40.735 INFO:teuthology.orchestra.run.smithi039.stdout:5880 read 0xc83e0e8 thru 0xc848b12 (0xaa2b bytes) 2024-09-14T06:17:40.736 INFO:teuthology.orchestra.run.smithi039.stdout:5881 write 0xb3b4e66 thru 0xb3bd33b (0x84d6 bytes) 2024-09-14T06:17:40.744 INFO:teuthology.orchestra.run.smithi039.stdout:5882 punch from 0x49c488b to 0x49cbd22, (0x7497 bytes) 2024-09-14T06:17:40.744 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:40.744 INFO:teuthology.orchestra.run.smithi039.stdout:5886 writesame 0xa5eb104 thru 0xa5ebeb2 data_size 0x1(0xdaf bytes) 2024-09-14T06:17:40.765 INFO:teuthology.orchestra.run.smithi039.stdout:5888 writesame 0xd1ad888 thru 0xd1b24c9 data_size 0x1(0x4c42 bytes) 2024-09-14T06:17:40.768 INFO:teuthology.orchestra.run.smithi039.stdout:5890 write 0x74bb5c thru 0x75688e (0xad33 bytes) 2024-09-14T06:17:40.771 INFO:teuthology.orchestra.run.smithi039.stdout:5892 write 0x7aca191 thru 0x7aca2e6 (0x156 bytes) 2024-09-14T06:17:40.775 INFO:teuthology.orchestra.run.smithi039.stdout:5894 trunc from 0xd1b24ca to 0xcccad4f 2024-09-14T06:17:40.783 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:40.778+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:40.811 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:40.811 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:40.825 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:17:40.828 INFO:teuthology.orchestra.run.smithi039.stdout:5895 trunc from 0xcccad4f to 0xc1f62d 2024-09-14T06:17:40.868 INFO:teuthology.orchestra.run.smithi039.stdout:5896 write 0x36dab26 thru 0x36ea01c (0xf4f7 bytes) 2024-09-14T06:17:40.884 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:40.881+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:40.895 INFO:teuthology.orchestra.run.smithi039.stdout:5897 writesame 0x739e6f6 thru 0x73a1f9a data_size 0x1(0x38a5 bytes) 2024-09-14T06:17:40.916 INFO:teuthology.orchestra.run.smithi039.stdout:5898 trunc from 0x73a1f9b to 0x1323b6b 2024-09-14T06:17:40.934 INFO:teuthology.orchestra.run.smithi039.stdout:5899 read 0xfde9f thru 0x1050ed (0x724f bytes) 2024-09-14T06:17:40.949 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:40.949 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:40.958 INFO:teuthology.orchestra.run.smithi039.stdout:5900 write 0x3c4c2f0 thru 0x3c5b58b (0xf29c bytes) 2024-09-14T06:17:40.964 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:17:41.001 INFO:teuthology.orchestra.run.smithi039.stdout:5903 write 0xc61b16b thru 0xc6238c7 (0x875d bytes) 2024-09-14T06:17:41.007 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:41.005+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:41.023 INFO:teuthology.orchestra.run.smithi039.stdout:5907 trunc from 0xc6238c8 to 0x6186cba 2024-09-14T06:17:41.042 INFO:teuthology.orchestra.run.smithi039.stdout:5909 read 0x5c131e9 thru 0x5c1b5fc (0x8414 bytes) 2024-09-14T06:17:41.043 INFO:teuthology.orchestra.run.smithi039.stdout:5911 read 0x1e3b10d thru 0x1e499b2 (0xe8a6 bytes) 2024-09-14T06:17:41.060 INFO:teuthology.orchestra.run.smithi039.stdout:5912 write 0x6b16444 thru 0x6b1dceb (0x78a8 bytes) 2024-09-14T06:17:41.080 INFO:teuthology.orchestra.run.smithi039.stdout:5913 write 0x7dc12e8 thru 0x7dc572d (0x4446 bytes) 2024-09-14T06:17:41.084 INFO:teuthology.orchestra.run.smithi039.stdout:5914 read 0x7b6c2df thru 0x7b70b91 (0x48b3 bytes) 2024-09-14T06:17:41.085 INFO:teuthology.orchestra.run.smithi039.stdout:5915 read 0x4d792f4 thru 0x4d87ac3 (0xe7d0 bytes) 2024-09-14T06:17:41.086 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:41.081+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:41.086 INFO:teuthology.orchestra.run.smithi039.stdout:5917 read 0x2278282 thru 0x22837fd (0xb57c bytes) 2024-09-14T06:17:41.094 INFO:teuthology.orchestra.run.smithi039.stdout:5918 trunc from 0x7dc572e to 0x50b93d0 2024-09-14T06:17:41.114 INFO:teuthology.orchestra.run.smithi039.stdout:5919 read 0xf1287b thru 0xf1998a (0x7110 bytes) 2024-09-14T06:17:41.115 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:41.126 INFO:teuthology.orchestra.run.smithi039.stdout:5921 trunc from 0x50b93d0 to 0x8b82b04 2024-09-14T06:17:41.134 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:41.134 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:41.139 INFO:teuthology.orchestra.run.smithi039.stdout:5922 read 0x3ccda10 thru 0x3cd60b3 (0x86a4 bytes) 2024-09-14T06:17:41.140 INFO:teuthology.orchestra.run.smithi039.stdout:5923 write 0x489e3ac thru 0x48ab90a (0xd55f bytes) 2024-09-14T06:17:41.149 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:17:41.168 INFO:teuthology.orchestra.run.smithi039.stdout:5924 write 0xc21c21b thru 0xc229b62 (0xd948 bytes) 2024-09-14T06:17:41.172 INFO:teuthology.orchestra.run.smithi039.stdout:5925 writesame 0xa53cd98 thru 0xa53db76 data_size 0x1(0xddf bytes) 2024-09-14T06:17:41.177 INFO:teuthology.orchestra.run.smithi039.stdout:5926 punch from 0x4c272b9 to 0x4c30091, (0x8dd8 bytes) 2024-09-14T06:17:41.178 INFO:teuthology.orchestra.run.smithi039.stdout:5928 trunc from 0xc229b63 to 0xbcc4580 2024-09-14T06:17:41.191 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:41.189+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:41.197 INFO:teuthology.orchestra.run.smithi039.stdout:5929 write 0x133bbcb thru 0x134662f (0xaa65 bytes) 2024-09-14T06:17:41.222 INFO:teuthology.orchestra.run.smithi039.stdout:5930 write 0xe725786 thru 0xe72af9b (0x5816 bytes) 2024-09-14T06:17:41.225 INFO:teuthology.orchestra.run.smithi039.stdout:5931 punch from 0x5a89efa to 0x5a8dcc1, (0x3dc7 bytes) 2024-09-14T06:17:41.248 INFO:teuthology.orchestra.run.smithi039.stdout:5933 trunc from 0xe72af9c to 0x60d3959 2024-09-14T06:17:41.279 INFO:teuthology.orchestra.run.smithi039.stdout:5934 trunc from 0x60d3959 to 0x180003d 2024-09-14T06:17:41.288 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:41.286+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:41.302 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:17:41.321 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:41.321 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:41.321 INFO:teuthology.orchestra.run.smithi039.stdout:5935 writesame 0x7367f7a thru 0x7368d3c data_size 0x1(0xdc3 bytes) 2024-09-14T06:17:41.335 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:17:41.343 INFO:teuthology.orchestra.run.smithi039.stdout:5936 writesame 0xd6d7419 thru 0xd6d99f1 data_size 0x1(0x25d9 bytes) 2024-09-14T06:17:41.345 INFO:teuthology.orchestra.run.smithi039.stdout:5937 punch from 0x8f76689 to 0x8f7d68a, (0x7001 bytes) 2024-09-14T06:17:41.345 INFO:teuthology.orchestra.run.smithi039.stdout:5938 read 0x2f34c59 thru 0x2f3f792 (0xab3a bytes) 2024-09-14T06:17:41.345 INFO:teuthology.orchestra.run.smithi039.stdout:5939 write 0x2f8e9c2 thru 0x2f946c4 (0x5d03 bytes) 2024-09-14T06:17:41.348 INFO:teuthology.orchestra.run.smithi039.stdout:5940 write 0xa0b631e thru 0xa0b97fc (0x34df bytes) 2024-09-14T06:17:41.350 INFO:teuthology.orchestra.run.smithi039.stdout:5942 read 0xcadd775 thru 0xcade980 (0x120c bytes) 2024-09-14T06:17:41.350 INFO:teuthology.orchestra.run.smithi039.stdout:5943 punch from 0xcd0d54f to 0xcd1c16c, (0xec1d bytes) 2024-09-14T06:17:41.351 INFO:teuthology.orchestra.run.smithi039.stdout:5944 write 0x80b906b thru 0x80c5cd3 (0xcc69 bytes) 2024-09-14T06:17:41.355 INFO:teuthology.orchestra.run.smithi039.stdout:5945 trunc from 0xd6d99f2 to 0xca971d7 2024-09-14T06:17:41.367 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:41.367 INFO:teuthology.orchestra.run.smithi039.stdout:5947 read 0x16e3659 thru 0x16e96f0 (0x6098 bytes) 2024-09-14T06:17:41.384 INFO:teuthology.orchestra.run.smithi039.stdout:5951 trunc from 0xca971d7 to 0x1f94ad2 2024-09-14T06:17:41.388 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:41.386+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:41.397 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:41.429 INFO:teuthology.orchestra.run.smithi039.stdout:5953 write 0xee2d6d6 thru 0xee3a227 (0xcb52 bytes) 2024-09-14T06:17:41.432 INFO:teuthology.orchestra.run.smithi039.stdout:5955 write 0x5ecde86 thru 0x5ecdeb1 (0x2c bytes) 2024-09-14T06:17:41.434 INFO:teuthology.orchestra.run.smithi039.stdout:5957 write 0x53c2db8 thru 0x53ce4da (0xb723 bytes) 2024-09-14T06:17:41.442 INFO:teuthology.orchestra.run.smithi039.stdout:5959 trunc from 0xee3a228 to 0xc881cd5 2024-09-14T06:17:41.453 INFO:teuthology.orchestra.run.smithi039.stdout:5960 write 0x3fa5e8a thru 0x3faecc9 (0x8e40 bytes) 2024-09-14T06:17:41.455 INFO:teuthology.orchestra.run.smithi039.stdout:5962 read 0x9f18dd9 thru 0x9f28430 (0xf658 bytes) 2024-09-14T06:17:41.455 INFO:teuthology.orchestra.run.smithi039.stdout:5963 write 0x805318f thru 0x8057aa1 (0x4913 bytes) 2024-09-14T06:17:41.458 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:41.458 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:41.465 INFO:teuthology.orchestra.run.smithi039.stdout:5964 write 0xd941f5f thru 0xd94e063 (0xc105 bytes) 2024-09-14T06:17:41.466 INFO:teuthology.orchestra.run.smithi039.stdout:5965 read 0x56e7ed6 thru 0x56ec924 (0x4a4f bytes) 2024-09-14T06:17:41.467 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:41.467 INFO:teuthology.orchestra.run.smithi039.stdout:5967 punch from 0xd163529 to 0xd169cc6, (0x679d bytes) 2024-09-14T06:17:41.467 INFO:teuthology.orchestra.run.smithi039.stdout:5968 trunc from 0xd94e064 to 0xd5114e2 2024-09-14T06:17:41.474 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:17:41.475 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:41.475 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:41.477 INFO:teuthology.orchestra.run.smithi039.stdout:5969 read 0x37e91fd thru 0x37ef3ab (0x61af bytes) 2024-09-14T06:17:41.478 INFO:teuthology.orchestra.run.smithi039.stdout:5970 punch from 0x685bc08 to 0x685c448, (0x840 bytes) 2024-09-14T06:17:41.478 INFO:teuthology.orchestra.run.smithi039.stdout:5971 punch from 0x79c7110 to 0x79cdd20, (0x6c10 bytes) 2024-09-14T06:17:41.478 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:41.478 INFO:teuthology.orchestra.run.smithi039.stdout:5973 read 0x2ff6de5 thru 0x2ff8ddb (0x1ff7 bytes) 2024-09-14T06:17:41.478 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:41.478 INFO:teuthology.orchestra.run.smithi039.stdout:5977 punch from 0x28da50a to 0x28e3dad, (0x98a3 bytes) 2024-09-14T06:17:41.478 INFO:teuthology.orchestra.run.smithi039.stdout:5978 read 0x9a30dcc thru 0x9a36cf3 (0x5f28 bytes) 2024-09-14T06:17:41.478 INFO:teuthology.orchestra.run.smithi039.stdout:5980 read 0x61d6d5f thru 0x61dd2ff (0x65a1 bytes) 2024-09-14T06:17:41.493 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:41.493 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:41.493 INFO:teuthology.orchestra.run.smithi039.stdout:5982 trunc from 0xd5114e2 to 0x2fb83a5 2024-09-14T06:17:41.503 INFO:teuthology.orchestra.run.smithi039.stdout:5984 read 0x1b72a9d thru 0x1b72f6d (0x4d1 bytes) 2024-09-14T06:17:41.503 INFO:teuthology.orchestra.run.smithi039.stdout:5985 read 0x212ebd9 thru 0x2135e75 (0x729d bytes) 2024-09-14T06:17:41.504 INFO:teuthology.orchestra.run.smithi039.stdout:5986 read 0x1c030df thru 0x1c0623a (0x315c bytes) 2024-09-14T06:17:41.513 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:41.513 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:41.514 INFO:teuthology.orchestra.run.smithi039.stdout:5988 write 0x599ce5e thru 0x59a9652 (0xc7f5 bytes) 2024-09-14T06:17:41.519 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:41.517+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:41.527 INFO:teuthology.orchestra.run.smithi039.stdout:5990 write 0x8e6c08b thru 0x8e78b05 (0xca7b bytes) 2024-09-14T06:17:41.534 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:41.545 INFO:teuthology.orchestra.run.smithi039.stdout:5991 write 0xee3d174 thru 0xee43e1d (0x6caa bytes) 2024-09-14T06:17:41.548 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:41.548 INFO:teuthology.orchestra.run.smithi039.stdout:5993 punch from 0x8092640 to 0x8098b6a, (0x652a bytes) 2024-09-14T06:17:41.548 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:41.548 INFO:teuthology.orchestra.run.smithi039.stdout:5995 punch from 0x57a63f0 to 0x57b20d0, (0xbce0 bytes) 2024-09-14T06:17:41.548 INFO:teuthology.orchestra.run.smithi039.stdout:5996 read 0xcf6bc09 thru 0xcf717c6 (0x5bbe bytes) 2024-09-14T06:17:41.548 INFO:teuthology.orchestra.run.smithi039.stdout:5997 write 0x6842898 thru 0x684446a (0x1bd3 bytes) 2024-09-14T06:17:41.550 INFO:teuthology.orchestra.run.smithi039.stdout:5998 write 0xd08969e thru 0xd0976d6 (0xe039 bytes) 2024-09-14T06:17:41.552 INFO:teuthology.orchestra.run.smithi039.stdout:compare and write disabled 2024-09-14T06:17:41.553 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:41.553 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:41.573 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:41.574 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:41.578 INFO:teuthology.orchestra.run.smithi039.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-14T06:17:41.590 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:41.586+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:41.595 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:41.596 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:41.598 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:41.598 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:41.599 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:41.599 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:41.599 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.599 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:3595 3.1 3:811c1c24:::rbd_data.1152bd92e04f.000000000000006e:head [set-alloc-hint object_size 2097152 write_size 2097152,write 440183~2185] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T06:17:41.599 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:41.599 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 75.374088287000006, 2024-09-14T06:17:41.599 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0365806689999999, 2024-09-14T06:17:41.599 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.599 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.599 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.599 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.600 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.600 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.600 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 3595 2024-09-14T06:17:41.600 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.600 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.600 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.600 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.600 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:41.600 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.600 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.600 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.600 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.601 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:17:41.601 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.601 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.601 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222823+0000", 2024-09-14T06:17:41.601 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8219999999999998e-06 2024-09-14T06:17:41.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.601 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.601 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222835+0000", 2024-09-14T06:17:41.601 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2052000000000001e-05 2024-09-14T06:17:41.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.602 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.602 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222859+0000", 2024-09-14T06:17:41.602 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3694e-05 2024-09-14T06:17:41.602 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.602 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.602 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.602 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222872+0000", 2024-09-14T06:17:41.602 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3203999999999999e-05 2024-09-14T06:17:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223068+0000", 2024-09-14T06:17:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000195995 2024-09-14T06:17:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223312+0000", 2024-09-14T06:17:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024424199999999998 2024-09-14T06:17:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.603 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:41.604 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223786+0000", 2024-09-14T06:17:41.604 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00047356600000000003 2024-09-14T06:17:41.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.604 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.604 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:41.604 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.225374+0000", 2024-09-14T06:17:41.604 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015875150000000001 2024-09-14T06:17:41.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.604 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.604 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.604 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.233122+0000", 2024-09-14T06:17:41.604 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0077480700000000001 2024-09-14T06:17:41.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.605 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:17:41.605 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.233157+0000", 2024-09-14T06:17:41.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5113000000000001e-05 2024-09-14T06:17:41.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.605 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.605 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.252998+0000", 2024-09-14T06:17:41.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.019841142 2024-09-14T06:17:41.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.605 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for degraded object", 2024-09-14T06:17:41.605 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.254121+0000", 2024-09-14T06:17:41.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0011227979999999999 2024-09-14T06:17:41.606 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.606 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.606 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.606 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.259263+0000", 2024-09-14T06:17:41.606 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0051426700000000002 2024-09-14T06:17:41.606 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.606 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.606 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.606 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.259401+0000", 2024-09-14T06:17:41.606 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013778600000000001 2024-09-14T06:17:41.606 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.606 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.606 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.607 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.607 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11421 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.607 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:41.607 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.611193100999998, 2024-09-14T06:17:41.607 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078553170000000005, 2024-09-14T06:17:41.607 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.607 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.607 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.607 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.607 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.607 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.607 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11421 2024-09-14T06:17:41.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.608 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:41.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:17:41.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.609 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985718+0000", 2024-09-14T06:17:41.609 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8840000000000001e-06 2024-09-14T06:17:41.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.609 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.609 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.609 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985723+0000", 2024-09-14T06:17:41.609 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8799999999999996e-06 2024-09-14T06:17:41.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.609 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.609 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.609 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985742+0000", 2024-09-14T06:17:41.609 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8776000000000001e-05 2024-09-14T06:17:41.610 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.610 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.610 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.610 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985759+0000", 2024-09-14T06:17:41.610 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7119999999999999e-05 2024-09-14T06:17:41.610 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.610 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.610 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.610 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064259+0000", 2024-09-14T06:17:41.610 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078499418000000001 2024-09-14T06:17:41.610 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.610 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.611 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.611 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064269+0000", 2024-09-14T06:17:41.611 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:17:41.611 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.611 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.611 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.611 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.611 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.611 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11422 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.611 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:41.611 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.611123133999996, 2024-09-14T06:17:41.611 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078521540000000001, 2024-09-14T06:17:41.611 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.611 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.611 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.612 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.612 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.612 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.612 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11422 2024-09-14T06:17:41.612 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.612 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.612 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.612 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.612 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:41.612 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.612 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.612 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.612 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.612 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:17:41.613 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.613 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.613 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.613 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985787+0000", 2024-09-14T06:17:41.613 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7630000000000001e-06 2024-09-14T06:17:41.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.613 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.613 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.613 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985842+0000", 2024-09-14T06:17:41.613 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4555999999999997e-05 2024-09-14T06:17:41.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.613 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.613 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.614 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985864+0000", 2024-09-14T06:17:41.614 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2189999999999999e-05 2024-09-14T06:17:41.614 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.614 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.614 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.614 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985879+0000", 2024-09-14T06:17:41.614 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4382999999999999e-05 2024-09-14T06:17:41.614 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.614 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.614 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.614 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064298+0000", 2024-09-14T06:17:41.614 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078419121999999994 2024-09-14T06:17:41.614 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.614 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.614 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.615 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064307+0000", 2024-09-14T06:17:41.615 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5259999999999999e-06 2024-09-14T06:17:41.615 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.615 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.615 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.615 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.615 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.615 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11423 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.615 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:41.615 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.611005762000005, 2024-09-14T06:17:41.615 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078437175999999997, 2024-09-14T06:17:41.615 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.615 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.615 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.616 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.616 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.616 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.616 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11423 2024-09-14T06:17:41.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.616 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.616 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.616 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.616 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:41.616 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.616 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.616 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.616 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:17:41.617 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.617 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.617 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.617 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.617 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985905+0000", 2024-09-14T06:17:41.617 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8139999999999999e-06 2024-09-14T06:17:41.617 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.617 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.617 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.617 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985911+0000", 2024-09-14T06:17:41.617 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7230000000000001e-06 2024-09-14T06:17:41.617 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.617 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.617 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.618 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985930+0000", 2024-09-14T06:17:41.618 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9360000000000001e-05 2024-09-14T06:17:41.618 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.618 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.618 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.618 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985944+0000", 2024-09-14T06:17:41.618 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4479e-05 2024-09-14T06:17:41.618 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.618 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.618 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.618 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064332+0000", 2024-09-14T06:17:41.618 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078387686999999998 2024-09-14T06:17:41.618 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.619 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.619 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.619 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064340+0000", 2024-09-14T06:17:41.619 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1130000000000001e-06 2024-09-14T06:17:41.619 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.619 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.619 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.619 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.619 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.619 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11480 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.619 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:41.619 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.609655286000006, 2024-09-14T06:17:41.619 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077122298000000006, 2024-09-14T06:17:41.620 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.620 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.620 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.620 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.620 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.620 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.620 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11480 2024-09-14T06:17:41.620 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.620 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.620 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.620 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.620 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:41.620 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.621 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.621 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.621 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.621 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:17:41.621 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.621 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.621 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.621 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.621 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987255+0000", 2024-09-14T06:17:41.621 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:17:41.621 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.621 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.621 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.621 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987263+0000", 2024-09-14T06:17:41.622 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6499999999999996e-06 2024-09-14T06:17:41.622 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.622 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.622 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.622 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987285+0000", 2024-09-14T06:17:41.622 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1860000000000001e-05 2024-09-14T06:17:41.622 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.622 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.622 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.622 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987341+0000", 2024-09-14T06:17:41.622 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6648999999999999e-05 2024-09-14T06:17:41.622 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.622 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.622 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.623 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064367+0000", 2024-09-14T06:17:41.623 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077025622000000002 2024-09-14T06:17:41.623 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.623 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.623 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.623 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064376+0000", 2024-09-14T06:17:41.623 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7499999999999992e-06 2024-09-14T06:17:41.623 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.623 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.623 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.623 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.623 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.624 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11481 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.624 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:41.624 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.609519583999997, 2024-09-14T06:17:41.624 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077022057000000005, 2024-09-14T06:17:41.624 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.624 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.624 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.624 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.624 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.624 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.624 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11481 2024-09-14T06:17:41.624 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.624 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.624 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.625 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.625 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:41.625 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.625 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.625 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.625 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.625 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:17:41.625 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.625 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.625 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.625 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.625 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987391+0000", 2024-09-14T06:17:41.625 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4839999999999999e-06 2024-09-14T06:17:41.625 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.626 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.626 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.626 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987399+0000", 2024-09-14T06:17:41.626 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8660000000000006e-06 2024-09-14T06:17:41.626 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.626 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.626 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.626 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987419+0000", 2024-09-14T06:17:41.626 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9984e-05 2024-09-14T06:17:41.626 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.626 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.626 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.626 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987434+0000", 2024-09-14T06:17:41.626 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5396e-05 2024-09-14T06:17:41.627 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.627 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.627 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.627 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064403+0000", 2024-09-14T06:17:41.627 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076969212999999995 2024-09-14T06:17:41.627 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.627 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.627 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.627 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064411+0000", 2024-09-14T06:17:41.627 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1140000000000006e-06 2024-09-14T06:17:41.627 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.627 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.627 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.627 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.628 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.628 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11482 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.628 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:41.628 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.609434723999996, 2024-09-14T06:17:41.628 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076971172000000004, 2024-09-14T06:17:41.628 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.628 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.628 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.628 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.628 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.628 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.628 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11482 2024-09-14T06:17:41.628 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.629 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.629 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.629 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.629 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:41.629 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.629 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.629 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.629 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.629 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:17:41.629 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.629 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.629 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.629 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.630 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987476+0000", 2024-09-14T06:17:41.630 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.793e-06 2024-09-14T06:17:41.630 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.630 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.630 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.630 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987482+0000", 2024-09-14T06:17:41.630 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8270000000000003e-06 2024-09-14T06:17:41.630 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.630 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.630 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.630 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987502+0000", 2024-09-14T06:17:41.630 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0656e-05 2024-09-14T06:17:41.630 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.630 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.631 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.631 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987518+0000", 2024-09-14T06:17:41.631 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:17:41.631 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.631 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.631 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.631 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064438+0000", 2024-09-14T06:17:41.631 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076919623000000006 2024-09-14T06:17:41.631 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.631 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.631 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.631 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064445+0000", 2024-09-14T06:17:41.631 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.61e-06 2024-09-14T06:17:41.631 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.632 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.632 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.632 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.632 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.632 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11483 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.632 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:41.632 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.609366875000006, 2024-09-14T06:17:41.632 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076936562999999999, 2024-09-14T06:17:41.632 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.632 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.632 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.632 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.633 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.633 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.633 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11483 2024-09-14T06:17:41.633 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.633 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.633 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.633 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.633 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:41.633 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.633 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.633 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.633 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.633 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:17:41.633 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.634 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.634 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.634 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.634 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987544+0000", 2024-09-14T06:17:41.634 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.654e-06 2024-09-14T06:17:41.634 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.634 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.634 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.634 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987550+0000", 2024-09-14T06:17:41.634 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0299999999999999e-06 2024-09-14T06:17:41.634 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.634 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.634 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.634 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987570+0000", 2024-09-14T06:17:41.634 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0080000000000001e-05 2024-09-14T06:17:41.635 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.635 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.635 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.635 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987619+0000", 2024-09-14T06:17:41.635 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9228999999999999e-05 2024-09-14T06:17:41.635 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.635 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.635 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.635 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064471+0000", 2024-09-14T06:17:41.635 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076852086999999999 2024-09-14T06:17:41.635 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.635 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.635 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.635 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064478+0000", 2024-09-14T06:17:41.636 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4830000000000002e-06 2024-09-14T06:17:41.636 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.636 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.636 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.636 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.636 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.636 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11485 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.636 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:41.636 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.609268381999996, 2024-09-14T06:17:41.636 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076868463999999997, 2024-09-14T06:17:41.636 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.636 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.636 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.637 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.637 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.637 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.637 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11485 2024-09-14T06:17:41.637 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.637 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.637 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.637 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.637 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:41.637 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.637 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.637 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.637 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.637 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:17:41.637 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.638 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.638 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.638 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.638 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987642+0000", 2024-09-14T06:17:41.638 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4100000000000001e-06 2024-09-14T06:17:41.638 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.638 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.638 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.638 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987648+0000", 2024-09-14T06:17:41.638 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2500000000000003e-06 2024-09-14T06:17:41.638 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.638 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.638 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.638 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987683+0000", 2024-09-14T06:17:41.638 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5193e-05 2024-09-14T06:17:41.639 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.639 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.639 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.639 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987698+0000", 2024-09-14T06:17:41.639 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4893e-05 2024-09-14T06:17:41.639 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.639 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.639 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.639 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064501+0000", 2024-09-14T06:17:41.639 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076803263999999996 2024-09-14T06:17:41.639 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.639 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.640 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.640 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064509+0000", 2024-09-14T06:17:41.640 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4540000000000001e-06 2024-09-14T06:17:41.640 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.640 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.640 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.640 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.640 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.640 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11490 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.640 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:41.640 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.609118812999995, 2024-09-14T06:17:41.640 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076755043999999994, 2024-09-14T06:17:41.640 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.640 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.641 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.641 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.641 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.641 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.641 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11490 2024-09-14T06:17:41.641 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.641 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.641 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.641 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.641 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:41.641 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.641 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.642 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.642 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.642 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:17:41.642 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.642 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.642 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.642 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.642 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987791+0000", 2024-09-14T06:17:41.642 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:17:41.642 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.642 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.642 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.642 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987797+0000", 2024-09-14T06:17:41.642 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7969999999999997e-06 2024-09-14T06:17:41.642 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.643 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.643 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.643 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987817+0000", 2024-09-14T06:17:41.643 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9746e-05 2024-09-14T06:17:41.643 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.643 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.643 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.643 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987832+0000", 2024-09-14T06:17:41.643 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.452e-05 2024-09-14T06:17:41.643 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.643 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.643 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.643 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064535+0000", 2024-09-14T06:17:41.643 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076703478000000005 2024-09-14T06:17:41.643 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.644 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.644 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.644 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064545+0000", 2024-09-14T06:17:41.644 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9799999999999993e-06 2024-09-14T06:17:41.644 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.644 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.644 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.644 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.644 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.644 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11492 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.644 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:41.644 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.608991947999996, 2024-09-14T06:17:41.644 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076661759999999995, 2024-09-14T06:17:41.644 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.645 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.645 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.645 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.645 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.645 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.645 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11492 2024-09-14T06:17:41.645 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.645 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.645 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.645 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.645 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:41.645 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.645 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.645 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.646 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.646 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:17:41.646 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.646 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.646 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.646 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.646 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987918+0000", 2024-09-14T06:17:41.646 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3239999999999999e-06 2024-09-14T06:17:41.646 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.646 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.646 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.646 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987924+0000", 2024-09-14T06:17:41.646 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6930000000000004e-06 2024-09-14T06:17:41.646 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.646 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.647 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.647 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987943+0000", 2024-09-14T06:17:41.647 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9487000000000002e-05 2024-09-14T06:17:41.647 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.647 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.647 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.647 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987958+0000", 2024-09-14T06:17:41.647 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4453e-05 2024-09-14T06:17:41.647 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.647 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.647 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.647 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064571+0000", 2024-09-14T06:17:41.647 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076612988000000007 2024-09-14T06:17:41.647 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.647 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.648 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.648 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064579+0000", 2024-09-14T06:17:41.648 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8150000000000006e-06 2024-09-14T06:17:41.648 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.648 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.648 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.648 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.648 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.648 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11495 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.648 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:41.648 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.608823259000005, 2024-09-14T06:17:41.648 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076524746000000005, 2024-09-14T06:17:41.648 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.648 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.648 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.649 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.649 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.649 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.649 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11495 2024-09-14T06:17:41.649 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.649 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.649 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.649 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.649 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:41.649 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.649 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.649 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.649 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.649 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:17:41.650 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.650 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.650 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.650 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.650 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988087+0000", 2024-09-14T06:17:41.650 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0499999999999999e-06 2024-09-14T06:17:41.650 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.650 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.650 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.650 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988118+0000", 2024-09-14T06:17:41.650 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1900000000000003e-05 2024-09-14T06:17:41.650 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.651 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.651 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.651 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988138+0000", 2024-09-14T06:17:41.651 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.933e-05 2024-09-14T06:17:41.651 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.651 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.651 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.651 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988152+0000", 2024-09-14T06:17:41.651 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3889e-05 2024-09-14T06:17:41.651 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.651 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.651 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.652 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064603+0000", 2024-09-14T06:17:41.652 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076451082000000004 2024-09-14T06:17:41.652 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.652 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.652 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.652 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064610+0000", 2024-09-14T06:17:41.652 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4950000000000002e-06 2024-09-14T06:17:41.652 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.652 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.652 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.652 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.652 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.652 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11498 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.652 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:41.653 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.608737183000002, 2024-09-14T06:17:41.653 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076471883000000004, 2024-09-14T06:17:41.653 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.653 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.653 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.653 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.653 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.653 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.653 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11498 2024-09-14T06:17:41.653 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.653 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.653 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.653 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.653 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:41.654 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.654 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.654 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.654 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.654 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:17:41.654 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.654 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.654 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.654 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.654 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988173+0000", 2024-09-14T06:17:41.654 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2300000000000001e-06 2024-09-14T06:17:41.654 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.654 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.654 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.654 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988178+0000", 2024-09-14T06:17:41.655 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4229999999999996e-06 2024-09-14T06:17:41.655 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.655 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.655 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.655 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988235+0000", 2024-09-14T06:17:41.655 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6616999999999997e-05 2024-09-14T06:17:41.655 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.655 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.655 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.655 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988264+0000", 2024-09-14T06:17:41.655 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9189000000000001e-05 2024-09-14T06:17:41.655 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.655 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.655 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.656 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064636+0000", 2024-09-14T06:17:41.656 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076371573999999998 2024-09-14T06:17:41.656 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.656 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.656 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.656 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064643+0000", 2024-09-14T06:17:41.656 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8499999999999994e-06 2024-09-14T06:17:41.656 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.656 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.656 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.656 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.656 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.656 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11523 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.656 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:41.656 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.608017035000003, 2024-09-14T06:17:41.656 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076062454000000002, 2024-09-14T06:17:41.657 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.657 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.657 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.657 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.657 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.657 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.657 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11523 2024-09-14T06:17:41.657 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.657 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.657 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.657 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.657 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:41.657 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.658 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.658 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.658 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.658 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:17:41.658 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.658 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.658 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.658 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.658 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988894+0000", 2024-09-14T06:17:41.658 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.843e-06 2024-09-14T06:17:41.658 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.658 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.658 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.658 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988903+0000", 2024-09-14T06:17:41.659 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0440000000000001e-06 2024-09-14T06:17:41.659 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.659 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.659 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.659 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988929+0000", 2024-09-14T06:17:41.659 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6679e-05 2024-09-14T06:17:41.659 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.659 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.659 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.659 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988967+0000", 2024-09-14T06:17:41.659 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7453000000000002e-05 2024-09-14T06:17:41.659 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.659 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.659 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.659 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064668+0000", 2024-09-14T06:17:41.660 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075701389999999993 2024-09-14T06:17:41.660 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.660 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.660 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.660 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064954+0000", 2024-09-14T06:17:41.660 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028604500000000002 2024-09-14T06:17:41.660 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.660 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.660 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.660 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.660 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.660 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11530 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.661 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:41.661 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.607526484999994, 2024-09-14T06:17:41.661 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075620674999999998, 2024-09-14T06:17:41.661 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.661 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.661 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.661 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.661 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.661 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.661 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11530 2024-09-14T06:17:41.661 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.661 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.661 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.662 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.662 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:41.662 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.662 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.662 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.662 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.662 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:17:41.662 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.662 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.662 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.662 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.662 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989384+0000", 2024-09-14T06:17:41.662 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6160000000000001e-06 2024-09-14T06:17:41.662 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.663 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.663 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.663 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989390+0000", 2024-09-14T06:17:41.663 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7039999999999999e-06 2024-09-14T06:17:41.663 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.663 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.663 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.663 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989409+0000", 2024-09-14T06:17:41.663 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.95e-05 2024-09-14T06:17:41.663 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.663 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.663 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.663 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989429+0000", 2024-09-14T06:17:41.663 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0173000000000002e-05 2024-09-14T06:17:41.664 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.664 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.664 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.664 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064992+0000", 2024-09-14T06:17:41.664 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075562764000000004 2024-09-14T06:17:41.664 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.664 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.664 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.664 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065003+0000", 2024-09-14T06:17:41.664 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0917999999999999e-05 2024-09-14T06:17:41.664 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.664 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.664 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.665 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.665 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.665 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11532 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.665 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:41.665 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.607456088999996, 2024-09-14T06:17:41.665 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075604656000000006, 2024-09-14T06:17:41.665 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.665 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.665 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.665 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.665 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.665 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.665 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11532 2024-09-14T06:17:41.665 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.665 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.666 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.666 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.666 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:41.666 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.666 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.666 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.666 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.666 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:17:41.666 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.666 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.666 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.666 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.666 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989454+0000", 2024-09-14T06:17:41.666 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6500000000000001e-06 2024-09-14T06:17:41.666 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.667 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.667 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.667 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989460+0000", 2024-09-14T06:17:41.667 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7459999999999997e-06 2024-09-14T06:17:41.667 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.667 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.667 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.667 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989480+0000", 2024-09-14T06:17:41.667 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.948e-05 2024-09-14T06:17:41.667 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.667 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.667 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.667 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989497+0000", 2024-09-14T06:17:41.667 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7153e-05 2024-09-14T06:17:41.667 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.668 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.668 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.668 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065046+0000", 2024-09-14T06:17:41.668 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075548905999999999 2024-09-14T06:17:41.668 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.668 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.668 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.668 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065057+0000", 2024-09-14T06:17:41.668 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1721e-05 2024-09-14T06:17:41.668 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.668 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.668 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.668 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.668 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.668 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11534 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.669 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:41.669 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.607389603000001, 2024-09-14T06:17:41.669 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075574350999999998, 2024-09-14T06:17:41.669 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.669 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.669 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.669 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.669 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.669 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.669 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11534 2024-09-14T06:17:41.669 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.669 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.669 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.669 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.669 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:41.669 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.670 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.670 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.670 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.670 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:17:41.670 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.670 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.670 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.670 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.670 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989521+0000", 2024-09-14T06:17:41.670 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.533e-06 2024-09-14T06:17:41.670 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.670 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.670 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.670 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989527+0000", 2024-09-14T06:17:41.670 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0700000000000003e-06 2024-09-14T06:17:41.671 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.671 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.671 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.671 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989546+0000", 2024-09-14T06:17:41.671 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.948e-05 2024-09-14T06:17:41.671 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.671 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.671 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.671 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989561+0000", 2024-09-14T06:17:41.671 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4827000000000001e-05 2024-09-14T06:17:41.671 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.671 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.671 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.672 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065086+0000", 2024-09-14T06:17:41.672 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075524542 2024-09-14T06:17:41.672 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.672 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.672 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.672 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065094+0000", 2024-09-14T06:17:41.672 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8990000000000001e-06 2024-09-14T06:17:41.672 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.672 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.672 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.672 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.672 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.672 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11535 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.672 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:41.672 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.607322240000002, 2024-09-14T06:17:41.673 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075542466000000003, 2024-09-14T06:17:41.673 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.673 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.673 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.673 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.673 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.673 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.673 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11535 2024-09-14T06:17:41.673 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.673 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.673 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.673 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.673 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:41.673 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.674 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.674 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.674 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.674 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:17:41.674 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.674 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.674 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.674 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.674 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989588+0000", 2024-09-14T06:17:41.674 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1659999999999999e-06 2024-09-14T06:17:41.674 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.674 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.674 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.674 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989593+0000", 2024-09-14T06:17:41.675 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6230000000000002e-06 2024-09-14T06:17:41.675 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.675 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.675 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.675 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989613+0000", 2024-09-14T06:17:41.675 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9604000000000001e-05 2024-09-14T06:17:41.675 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.675 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.675 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.675 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989643+0000", 2024-09-14T06:17:41.675 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0450000000000002e-05 2024-09-14T06:17:41.675 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.675 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.675 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.675 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065119+0000", 2024-09-14T06:17:41.676 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075475930999999996 2024-09-14T06:17:41.676 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.676 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.676 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.676 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065129+0000", 2024-09-14T06:17:41.676 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6919999999999997e-06 2024-09-14T06:17:41.676 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.676 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.676 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.676 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.676 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.676 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11537 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.676 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:41.677 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.607244316999996, 2024-09-14T06:17:41.677 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075497868999999995, 2024-09-14T06:17:41.677 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.677 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.677 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.677 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.677 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.677 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.677 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11537 2024-09-14T06:17:41.677 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.677 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.677 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.677 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.677 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:41.677 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.677 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.678 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.678 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.678 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:17:41.678 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.678 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.678 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.678 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.678 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989666+0000", 2024-09-14T06:17:41.678 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.48e-06 2024-09-14T06:17:41.678 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.678 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.678 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.678 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989672+0000", 2024-09-14T06:17:41.678 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7960000000000001e-06 2024-09-14T06:17:41.678 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989690+0000", 2024-09-14T06:17:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8743e-05 2024-09-14T06:17:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989704+0000", 2024-09-14T06:17:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3913e-05 2024-09-14T06:17:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065155+0000", 2024-09-14T06:17:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075450351999999998 2024-09-14T06:17:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.680 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.680 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065162+0000", 2024-09-14T06:17:41.680 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5850000000000002e-06 2024-09-14T06:17:41.680 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.680 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.680 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.680 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.680 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.680 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11543 3.12 3.4d02a8b2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:41.680 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:41.680 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.606966752999995, 2024-09-14T06:17:41.680 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074335152000000002, 2024-09-14T06:17:41.680 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:41.680 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:41.681 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:17:41.681 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:41.681 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:41.681 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:41.681 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11543 2024-09-14T06:17:41.681 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.681 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:41.681 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.681 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:41.681 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:41.681 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.681 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.681 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.681 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:41.682 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:17:41.682 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:41.682 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.682 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.682 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:41.682 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989944+0000", 2024-09-14T06:17:41.682 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.694e-06 2024-09-14T06:17:41.682 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.682 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.682 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:41.682 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989950+0000", 2024-09-14T06:17:41.682 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3389999999999998e-06 2024-09-14T06:17:41.682 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.682 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.683 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:41.683 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989969+0000", 2024-09-14T06:17:41.683 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8963000000000001e-05 2024-09-14T06:17:41.683 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.683 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.683 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:41.683 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989985+0000", 2024-09-14T06:17:41.683 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5586999999999998e-05 2024-09-14T06:17:41.683 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.683 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.683 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:41.683 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064270+0000", 2024-09-14T06:17:41.683 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074285552000000005 2024-09-14T06:17:41.683 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:41.684 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:41.684 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:41.684 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064277+0000", 2024-09-14T06:17:41.684 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0169999999999999e-06 2024-09-14T06:17:41.684 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.684 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.684 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.684 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:41.684 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:41.684 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:41.685 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:41.685 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:41.686 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-09-14T06:17:41.686 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:17:41.727 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:41.725+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:41.792 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:41.790+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:41.858 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:41.859 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:41.859 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:41.859 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:41.873 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:17:41.919 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:41.917+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:41.994 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:41.990+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:41.999 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:41.999 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:41.999 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:41.999 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:41.999 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:42.014 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:17:42.099 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:42.096+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:42.179 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:42.179 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:42.179 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:42.179 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:42.180 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.180 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12584 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.180 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.039784+0000", 2024-09-14T06:17:42.180 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.137712190000002, 2024-09-14T06:17:42.180 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.024433808000000001, 2024-09-14T06:17:42.180 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.180 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.180 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.180 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.180 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.180 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.180 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12584 2024-09-14T06:17:42.180 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.181 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.181 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.181 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.181 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039784+0000", 2024-09-14T06:17:42.181 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.181 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.181 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.181 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.181 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039784+0000", 2024-09-14T06:17:42.181 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.181 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.181 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.181 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.181 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039784+0000", 2024-09-14T06:17:42.182 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1999999999999999e-07 2024-09-14T06:17:42.182 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.182 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.182 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.182 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039788+0000", 2024-09-14T06:17:42.182 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.41e-06 2024-09-14T06:17:42.182 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.182 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.182 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.182 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039797+0000", 2024-09-14T06:17:42.182 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7969999999999995e-06 2024-09-14T06:17:42.182 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.182 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.183 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.183 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.039804+0000", 2024-09-14T06:17:42.183 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7870000000000004e-06 2024-09-14T06:17:42.183 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.183 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.183 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.183 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.048222+0000", 2024-09-14T06:17:42.183 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0084172770000000008 2024-09-14T06:17:42.183 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.183 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.183 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.183 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.048334+0000", 2024-09-14T06:17:42.183 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000112895 2024-09-14T06:17:42.183 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.183 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.184 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.184 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064217+0000", 2024-09-14T06:17:42.184 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015882922000000001 2024-09-14T06:17:42.184 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.184 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.184 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.184 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.184 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.184 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12661 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.184 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.040519+0000", 2024-09-14T06:17:42.184 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.136976410000003, 2024-09-14T06:17:42.184 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014404233000000001, 2024-09-14T06:17:42.184 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.185 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.185 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.185 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.185 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.185 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.185 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12661 2024-09-14T06:17:42.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.185 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.185 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.185 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.185 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040519+0000", 2024-09-14T06:17:42.185 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.185 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040519+0000", 2024-09-14T06:17:42.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040520+0000", 2024-09-14T06:17:42.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6899999999999996e-07 2024-09-14T06:17:42.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040530+0000", 2024-09-14T06:17:42.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6409999999999997e-06 2024-09-14T06:17:42.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040540+0000", 2024-09-14T06:17:42.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0247e-05 2024-09-14T06:17:42.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040547+0000", 2024-09-14T06:17:42.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3830000000000003e-06 2024-09-14T06:17:42.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.054677+0000", 2024-09-14T06:17:42.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014130095000000001 2024-09-14T06:17:42.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.188 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.188 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.188 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.054786+0000", 2024-09-14T06:17:42.188 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000108441 2024-09-14T06:17:42.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.188 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.188 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.188 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.054924+0000", 2024-09-14T06:17:42.188 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013765699999999999 2024-09-14T06:17:42.188 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.188 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.188 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.188 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.189 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12662 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.189 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.040559+0000", 2024-09-14T06:17:42.189 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.136937072999999, 2024-09-14T06:17:42.189 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014659604999999999, 2024-09-14T06:17:42.189 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.189 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.189 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.189 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.189 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.189 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.189 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12662 2024-09-14T06:17:42.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.189 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.190 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.190 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040559+0000", 2024-09-14T06:17:42.190 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.190 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.190 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040559+0000", 2024-09-14T06:17:42.190 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.191 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040559+0000", 2024-09-14T06:17:42.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5799999999999998e-07 2024-09-14T06:17:42.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.191 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040562+0000", 2024-09-14T06:17:42.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9349999999999999e-06 2024-09-14T06:17:42.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.191 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040572+0000", 2024-09-14T06:17:42.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9480000000000003e-06 2024-09-14T06:17:42.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040579+0000", 2024-09-14T06:17:42.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1459999999999999e-06 2024-09-14T06:17:42.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.054963+0000", 2024-09-14T06:17:42.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014383986 2024-09-14T06:17:42.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.193 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.193 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055084+0000", 2024-09-14T06:17:42.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000120822 2024-09-14T06:17:42.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.193 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.193 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055218+0000", 2024-09-14T06:17:42.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013401 2024-09-14T06:17:42.193 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.193 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.193 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.194 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12666 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.194 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.040590+0000", 2024-09-14T06:17:42.194 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.136905924999994, 2024-09-14T06:17:42.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014895584999999999, 2024-09-14T06:17:42.194 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.194 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.194 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.194 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.194 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.194 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.194 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12666 2024-09-14T06:17:42.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.195 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040590+0000", 2024-09-14T06:17:42.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040590+0000", 2024-09-14T06:17:42.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040590+0000", 2024-09-14T06:17:42.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:17:42.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.196 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040593+0000", 2024-09-14T06:17:42.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0189999999999998e-06 2024-09-14T06:17:42.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.196 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040603+0000", 2024-09-14T06:17:42.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2820000000000002e-06 2024-09-14T06:17:42.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.196 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040610+0000", 2024-09-14T06:17:42.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1219999999999997e-06 2024-09-14T06:17:42.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055257+0000", 2024-09-14T06:17:42.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014646762000000001 2024-09-14T06:17:42.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055379+0000", 2024-09-14T06:17:42.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012186899999999999 2024-09-14T06:17:42.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.198 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.198 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055485+0000", 2024-09-14T06:17:42.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000106848 2024-09-14T06:17:42.198 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.198 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.198 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.198 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12672 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.198 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.040620+0000", 2024-09-14T06:17:42.198 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.136875442000004, 2024-09-14T06:17:42.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015140836, 2024-09-14T06:17:42.199 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.199 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.199 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.199 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.199 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.199 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.199 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12672 2024-09-14T06:17:42.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.199 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.199 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040620+0000", 2024-09-14T06:17:42.199 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.200 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.200 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.200 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040620+0000", 2024-09-14T06:17:42.200 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.200 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.200 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.200 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040621+0000", 2024-09-14T06:17:42.200 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9699999999999995e-07 2024-09-14T06:17:42.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.200 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.200 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.200 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040632+0000", 2024-09-14T06:17:42.201 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1012999999999999e-05 2024-09-14T06:17:42.201 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.201 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.201 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.201 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040642+0000", 2024-09-14T06:17:42.201 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0054999999999999e-05 2024-09-14T06:17:42.201 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.201 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.201 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.201 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.040649+0000", 2024-09-14T06:17:42.201 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3769999999999998e-06 2024-09-14T06:17:42.201 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.201 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.202 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.202 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055521+0000", 2024-09-14T06:17:42.202 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014871383 2024-09-14T06:17:42.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.202 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.202 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.202 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055634+0000", 2024-09-14T06:17:42.202 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000112757 2024-09-14T06:17:42.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.202 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.202 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.202 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.055761+0000", 2024-09-14T06:17:42.202 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012755399999999999 2024-09-14T06:17:42.202 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.203 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.203 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.203 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.203 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.203 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12674 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.203 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.041833+0000", 2024-09-14T06:17:42.203 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.135662698999994, 2024-09-14T06:17:42.203 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022511527, 2024-09-14T06:17:42.203 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.203 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.203 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.203 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.203 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.203 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.203 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12674 2024-09-14T06:17:42.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.204 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.204 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041833+0000", 2024-09-14T06:17:42.204 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.204 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041833+0000", 2024-09-14T06:17:42.204 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.205 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041834+0000", 2024-09-14T06:17:42.205 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7999999999999995e-07 2024-09-14T06:17:42.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.205 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.205 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.205 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041869+0000", 2024-09-14T06:17:42.205 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5766000000000003e-05 2024-09-14T06:17:42.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.205 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.205 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.205 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041922+0000", 2024-09-14T06:17:42.205 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2086000000000002e-05 2024-09-14T06:17:42.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.205 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.206 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.206 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041931+0000", 2024-09-14T06:17:42.206 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6160000000000008e-06 2024-09-14T06:17:42.206 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.206 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.206 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.056038+0000", 2024-09-14T06:17:42.206 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01410729 2024-09-14T06:17:42.206 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.206 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.206 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.056143+0000", 2024-09-14T06:17:42.206 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000104697 2024-09-14T06:17:42.206 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.207 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.207 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064345+0000", 2024-09-14T06:17:42.207 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0082013920000000001 2024-09-14T06:17:42.207 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.207 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.207 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.207 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.207 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.207 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12677 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.207 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.041952+0000", 2024-09-14T06:17:42.207 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.135543282, 2024-09-14T06:17:42.207 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022599910000000001, 2024-09-14T06:17:42.207 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.207 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.207 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.207 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.208 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.208 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.208 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12677 2024-09-14T06:17:42.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.208 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.208 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.208 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.208 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041952+0000", 2024-09-14T06:17:42.208 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.208 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.208 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.208 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041952+0000", 2024-09-14T06:17:42.208 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.209 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.209 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.209 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041953+0000", 2024-09-14T06:17:42.209 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1100000000000005e-07 2024-09-14T06:17:42.209 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.209 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.209 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.209 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.041958+0000", 2024-09-14T06:17:42.209 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3839999999999999e-06 2024-09-14T06:17:42.209 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.209 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.209 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.209 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042800+0000", 2024-09-14T06:17:42.209 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00084272399999999999 2024-09-14T06:17:42.209 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.210 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.210 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.210 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042817+0000", 2024-09-14T06:17:42.210 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6507e-05 2024-09-14T06:17:42.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.210 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.210 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.210 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064375+0000", 2024-09-14T06:17:42.210 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021558165000000001 2024-09-14T06:17:42.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.210 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.210 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.210 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064458+0000", 2024-09-14T06:17:42.211 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3205e-05 2024-09-14T06:17:42.211 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.211 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.211 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.211 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064552+0000", 2024-09-14T06:17:42.211 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.4114e-05 2024-09-14T06:17:42.211 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.211 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.211 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.211 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.211 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.212 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12678 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.212 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.042838+0000", 2024-09-14T06:17:42.212 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.134657653999994, 2024-09-14T06:17:42.212 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021916718000000002, 2024-09-14T06:17:42.212 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.212 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.212 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.212 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.212 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.212 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.212 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12678 2024-09-14T06:17:42.213 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.213 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.213 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.213 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.213 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042838+0000", 2024-09-14T06:17:42.213 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.213 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.213 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.213 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.213 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042838+0000", 2024-09-14T06:17:42.213 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.213 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.213 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.213 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.214 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042839+0000", 2024-09-14T06:17:42.214 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1510000000000001e-06 2024-09-14T06:17:42.214 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.214 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.214 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.214 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042845+0000", 2024-09-14T06:17:42.214 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9190000000000004e-06 2024-09-14T06:17:42.214 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.214 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.214 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.214 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042859+0000", 2024-09-14T06:17:42.214 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4022000000000001e-05 2024-09-14T06:17:42.214 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.215 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.215 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.215 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042867+0000", 2024-09-14T06:17:42.215 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9549999999999992e-06 2024-09-14T06:17:42.215 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.215 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.215 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.215 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064581+0000", 2024-09-14T06:17:42.215 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021713542999999998 2024-09-14T06:17:42.215 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.215 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.215 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.215 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064656+0000", 2024-09-14T06:17:42.215 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5564999999999999e-05 2024-09-14T06:17:42.215 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.216 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.216 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.216 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064755+0000", 2024-09-14T06:17:42.216 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.8562999999999998e-05 2024-09-14T06:17:42.216 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.216 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.216 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.216 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.216 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12687 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.216 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.042878+0000", 2024-09-14T06:17:42.216 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.134617860000006, 2024-09-14T06:17:42.216 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022072561000000001, 2024-09-14T06:17:42.216 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.216 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.216 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.217 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.217 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.217 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.217 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12687 2024-09-14T06:17:42.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.217 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.217 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.217 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.217 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042878+0000", 2024-09-14T06:17:42.217 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.217 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.217 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.217 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042878+0000", 2024-09-14T06:17:42.217 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.218 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.218 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.218 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.218 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042879+0000", 2024-09-14T06:17:42.218 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9999999999999997e-07 2024-09-14T06:17:42.218 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.218 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.218 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.218 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042882+0000", 2024-09-14T06:17:42.218 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.608e-06 2024-09-14T06:17:42.218 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.218 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.218 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.218 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042892+0000", 2024-09-14T06:17:42.218 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9729999999999992e-06 2024-09-14T06:17:42.218 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.219 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.219 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.219 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042911+0000", 2024-09-14T06:17:42.219 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9369999999999999e-05 2024-09-14T06:17:42.219 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.219 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.219 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.219 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064785+0000", 2024-09-14T06:17:42.219 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021873182000000001 2024-09-14T06:17:42.219 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.219 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.219 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.219 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064860+0000", 2024-09-14T06:17:42.219 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5403999999999998e-05 2024-09-14T06:17:42.219 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.220 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.220 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.220 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064950+0000", 2024-09-14T06:17:42.220 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0323999999999998e-05 2024-09-14T06:17:42.220 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.220 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.220 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.220 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.220 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.220 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12688 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.220 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.042922+0000", 2024-09-14T06:17:42.220 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.134573536999994, 2024-09-14T06:17:42.220 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022202118, 2024-09-14T06:17:42.220 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.220 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.221 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.221 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.221 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.221 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.221 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12688 2024-09-14T06:17:42.221 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.221 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.221 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.221 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.221 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042922+0000", 2024-09-14T06:17:42.221 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.221 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.221 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.221 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.221 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042922+0000", 2024-09-14T06:17:42.221 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.222 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.222 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.222 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.222 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042923+0000", 2024-09-14T06:17:42.222 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:17:42.222 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.222 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.222 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.222 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042937+0000", 2024-09-14T06:17:42.222 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4337e-05 2024-09-14T06:17:42.222 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.222 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.222 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.222 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042948+0000", 2024-09-14T06:17:42.222 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0569e-05 2024-09-14T06:17:42.222 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.223 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.223 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.223 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.042955+0000", 2024-09-14T06:17:42.223 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3250000000000002e-06 2024-09-14T06:17:42.223 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.223 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.223 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.223 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064981+0000", 2024-09-14T06:17:42.223 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022026046000000001 2024-09-14T06:17:42.223 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.223 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.223 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.223 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065055+0000", 2024-09-14T06:17:42.223 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4016000000000004e-05 2024-09-14T06:17:42.223 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.223 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.224 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.224 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065124+0000", 2024-09-14T06:17:42.224 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9141999999999996e-05 2024-09-14T06:17:42.224 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.224 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.224 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.224 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.224 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.224 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12690 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.224 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044605+0000", 2024-09-14T06:17:42.224 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.132890497000005, 2024-09-14T06:17:42.224 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020723715, 2024-09-14T06:17:42.224 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.224 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.224 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.224 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.225 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.225 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.225 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12690 2024-09-14T06:17:42.225 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.225 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.225 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.225 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.225 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044605+0000", 2024-09-14T06:17:42.225 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.225 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.225 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.225 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.225 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044605+0000", 2024-09-14T06:17:42.225 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.226 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.226 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044606+0000", 2024-09-14T06:17:42.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2020000000000001e-06 2024-09-14T06:17:42.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.226 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.226 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044612+0000", 2024-09-14T06:17:42.226 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9830000000000001e-06 2024-09-14T06:17:42.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.226 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.226 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.226 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044631+0000", 2024-09-14T06:17:42.227 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8280000000000001e-05 2024-09-14T06:17:42.227 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.227 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.227 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.227 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044642+0000", 2024-09-14T06:17:42.227 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1018e-05 2024-09-14T06:17:42.227 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.227 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.227 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.227 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065153+0000", 2024-09-14T06:17:42.227 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020511709999999999 2024-09-14T06:17:42.227 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.227 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.227 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.228 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065230+0000", 2024-09-14T06:17:42.228 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7003999999999997e-05 2024-09-14T06:17:42.228 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.228 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.228 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.228 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065329+0000", 2024-09-14T06:17:42.228 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.8517999999999994e-05 2024-09-14T06:17:42.228 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.228 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.228 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.228 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.228 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.228 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12692 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.229 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044659+0000", 2024-09-14T06:17:42.229 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.132836667999996, 2024-09-14T06:17:42.229 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020896346999999999, 2024-09-14T06:17:42.229 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.229 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.229 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.229 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.229 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.229 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.229 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12692 2024-09-14T06:17:42.229 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.229 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.229 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.229 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.230 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044659+0000", 2024-09-14T06:17:42.230 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.230 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.230 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.230 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.230 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044659+0000", 2024-09-14T06:17:42.230 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.230 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.230 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.230 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.230 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044660+0000", 2024-09-14T06:17:42.230 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0200000000000001e-07 2024-09-14T06:17:42.230 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.230 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.231 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.231 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044663+0000", 2024-09-14T06:17:42.231 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:17:42.231 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.231 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.231 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.231 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044674+0000", 2024-09-14T06:17:42.231 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.096e-05 2024-09-14T06:17:42.231 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.231 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.231 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.231 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044682+0000", 2024-09-14T06:17:42.231 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7149999999999998e-06 2024-09-14T06:17:42.231 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.231 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.231 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.232 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065357+0000", 2024-09-14T06:17:42.232 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020674794999999999 2024-09-14T06:17:42.232 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.232 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.232 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.232 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065435+0000", 2024-09-14T06:17:42.232 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8170999999999994e-05 2024-09-14T06:17:42.232 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.232 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.232 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.232 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065555+0000", 2024-09-14T06:17:42.232 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000120609 2024-09-14T06:17:42.232 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.232 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.232 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.232 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.233 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.233 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12694 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.233 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044692+0000", 2024-09-14T06:17:42.233 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.132803393000003, 2024-09-14T06:17:42.233 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021050059999999999, 2024-09-14T06:17:42.233 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.233 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.233 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.233 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.233 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.233 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.233 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12694 2024-09-14T06:17:42.233 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.233 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.233 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.234 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.234 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044692+0000", 2024-09-14T06:17:42.234 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.234 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.234 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.234 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.234 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044692+0000", 2024-09-14T06:17:42.234 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.234 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.234 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.234 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.234 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044693+0000", 2024-09-14T06:17:42.234 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8800000000000002e-07 2024-09-14T06:17:42.234 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.234 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.235 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.235 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044705+0000", 2024-09-14T06:17:42.235 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2252000000000001e-05 2024-09-14T06:17:42.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.235 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.235 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.235 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044716+0000", 2024-09-14T06:17:42.235 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1206000000000001e-05 2024-09-14T06:17:42.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.235 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.235 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.235 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044724+0000", 2024-09-14T06:17:42.235 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7360000000000001e-06 2024-09-14T06:17:42.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.235 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.236 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.236 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065584+0000", 2024-09-14T06:17:42.236 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.020859459 2024-09-14T06:17:42.236 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.236 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.236 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.236 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065657+0000", 2024-09-14T06:17:42.236 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3509000000000004e-05 2024-09-14T06:17:42.236 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.236 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.236 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.236 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065742+0000", 2024-09-14T06:17:42.236 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5210000000000006e-05 2024-09-14T06:17:42.236 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.237 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.237 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.237 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.237 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.237 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12702 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.237 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044734+0000", 2024-09-14T06:17:42.237 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.132761403000003, 2024-09-14T06:17:42.237 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021236848999999999, 2024-09-14T06:17:42.237 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.237 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.237 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.237 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.237 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.237 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.237 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12702 2024-09-14T06:17:42.238 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.238 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.238 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.238 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.238 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044734+0000", 2024-09-14T06:17:42.238 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.238 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.238 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.238 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.238 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044734+0000", 2024-09-14T06:17:42.238 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.238 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.238 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.238 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.238 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044735+0000", 2024-09-14T06:17:42.238 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3600000000000003e-07 2024-09-14T06:17:42.239 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.239 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.239 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.239 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044738+0000", 2024-09-14T06:17:42.239 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9189999999999999e-06 2024-09-14T06:17:42.239 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.239 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.239 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.239 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044748+0000", 2024-09-14T06:17:42.239 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0111e-05 2024-09-14T06:17:42.239 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.239 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.239 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.239 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044766+0000", 2024-09-14T06:17:42.239 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8199999999999999e-05 2024-09-14T06:17:42.239 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.240 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.240 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.240 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065771+0000", 2024-09-14T06:17:42.240 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021004683 2024-09-14T06:17:42.240 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.240 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.240 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.240 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065849+0000", 2024-09-14T06:17:42.240 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7684999999999999e-05 2024-09-14T06:17:42.240 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.240 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.240 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.240 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065971+0000", 2024-09-14T06:17:42.240 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00012251499999999999 2024-09-14T06:17:42.240 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.241 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.241 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.241 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.241 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.241 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12703 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 401408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.241 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044777+0000", 2024-09-14T06:17:42.241 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.132718776999994, 2024-09-14T06:17:42.241 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021382453999999999, 2024-09-14T06:17:42.241 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.241 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.241 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.241 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.241 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.241 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.241 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12703 2024-09-14T06:17:42.242 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.242 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.242 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.242 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.242 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044777+0000", 2024-09-14T06:17:42.242 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.242 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.242 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.242 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.242 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044777+0000", 2024-09-14T06:17:42.242 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.242 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.242 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.242 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.242 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044778+0000", 2024-09-14T06:17:42.242 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2999999999999999e-07 2024-09-14T06:17:42.243 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.243 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.243 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.243 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044787+0000", 2024-09-14T06:17:42.243 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.2429999999999994e-06 2024-09-14T06:17:42.243 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.243 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.243 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.243 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044798+0000", 2024-09-14T06:17:42.243 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0447e-05 2024-09-14T06:17:42.243 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.243 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.243 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.243 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044805+0000", 2024-09-14T06:17:42.243 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.182e-06 2024-09-14T06:17:42.243 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.244 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.244 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.244 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066001+0000", 2024-09-14T06:17:42.244 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021196079 2024-09-14T06:17:42.244 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.244 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.244 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.244 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066082+0000", 2024-09-14T06:17:42.244 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1011000000000001e-05 2024-09-14T06:17:42.244 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.244 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.244 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.244 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066159+0000", 2024-09-14T06:17:42.244 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7562e-05 2024-09-14T06:17:42.244 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.245 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.245 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.245 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.245 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.245 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12705 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.245 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044815+0000", 2024-09-14T06:17:42.245 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.132680937000003, 2024-09-14T06:17:42.245 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021573917000000001, 2024-09-14T06:17:42.245 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.245 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.245 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.245 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.245 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.245 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.245 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12705 2024-09-14T06:17:42.245 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044815+0000", 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044815+0000", 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044815+0000", 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9999999999999997e-07 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.246 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.247 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.247 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044818+0000", 2024-09-14T06:17:42.247 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7779999999999999e-06 2024-09-14T06:17:42.247 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.247 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.247 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.247 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044828+0000", 2024-09-14T06:17:42.247 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0095000000000001e-05 2024-09-14T06:17:42.247 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.247 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.247 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.247 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044835+0000", 2024-09-14T06:17:42.247 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9419999999999998e-06 2024-09-14T06:17:42.247 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.247 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.248 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.248 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066188+0000", 2024-09-14T06:17:42.248 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021352560999999999 2024-09-14T06:17:42.248 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.248 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.248 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.248 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066275+0000", 2024-09-14T06:17:42.248 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7020000000000004e-05 2024-09-14T06:17:42.248 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.248 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.248 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.248 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066389+0000", 2024-09-14T06:17:42.248 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000113921 2024-09-14T06:17:42.248 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.249 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.249 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.249 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.249 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.249 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:12707 3.b 3:d4a06803:::rbd_data.1152c88ccedd.0000000000000018:5 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:42.249 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.044844+0000", 2024-09-14T06:17:42.249 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 66.132651367999998, 2024-09-14T06:17:42.249 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021756622, 2024-09-14T06:17:42.249 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.249 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.249 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:17:42.249 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.249 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.249 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.249 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 12707 2024-09-14T06:17:42.250 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.250 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.250 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.250 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.250 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044844+0000", 2024-09-14T06:17:42.250 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.250 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.250 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.250 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.250 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044844+0000", 2024-09-14T06:17:42.250 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.250 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.250 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.251 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.251 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044845+0000", 2024-09-14T06:17:42.251 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3600000000000003e-07 2024-09-14T06:17:42.251 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.251 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.251 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.251 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044848+0000", 2024-09-14T06:17:42.251 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4710000000000001e-06 2024-09-14T06:17:42.251 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.251 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.251 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.251 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044857+0000", 2024-09-14T06:17:42.252 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5219999999999996e-06 2024-09-14T06:17:42.252 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.252 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.252 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.252 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.044864+0000", 2024-09-14T06:17:42.252 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:17:42.252 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.252 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.252 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.252 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066424+0000", 2024-09-14T06:17:42.252 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.021560461999999999 2024-09-14T06:17:42.252 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.253 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.253 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.253 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066511+0000", 2024-09-14T06:17:42.253 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6298999999999995e-05 2024-09-14T06:17:42.253 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.253 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.253 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.253 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.066601+0000", 2024-09-14T06:17:42.253 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0252000000000003e-05 2024-09-14T06:17:42.253 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.253 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.254 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.254 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.254 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.254 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_repop(client.4434.0:24231 3.c e47/23 3:335dc9f4:::rbd_data.1152db28e106.0000000000000002:head v 47'36, mlcod=46'34)", 2024-09-14T06:17:42.254 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:51.909218+0000", 2024-09-14T06:17:42.254 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 50.268277963999999, 2024-09-14T06:17:42.254 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.018625953000000001, 2024-09-14T06:17:42.254 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.254 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.254 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:42.254 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.254 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.255 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.255 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.909218+0000", 2024-09-14T06:17:42.255 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.255 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.255 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.255 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.255 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.909218+0000", 2024-09-14T06:17:42.255 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.255 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.255 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.255 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.256 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.909221+0000", 2024-09-14T06:17:42.256 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:17:42.256 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.256 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.256 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.256 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.921926+0000", 2024-09-14T06:17:42.256 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.012705602 2024-09-14T06:17:42.256 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.256 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.256 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.256 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.921949+0000", 2024-09-14T06:17:42.256 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2824e-05 2024-09-14T06:17:42.257 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.257 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.257 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.257 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.921960+0000", 2024-09-14T06:17:42.257 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0626e-05 2024-09-14T06:17:42.257 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.257 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.257 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.257 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.922035+0000", 2024-09-14T06:17:42.257 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4938999999999995e-05 2024-09-14T06:17:42.257 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.258 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.258 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.258 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.922058+0000", 2024-09-14T06:17:42.258 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2847e-05 2024-09-14T06:17:42.258 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.258 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.258 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:42.258 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.927752+0000", 2024-09-14T06:17:42.258 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056944810000000004 2024-09-14T06:17:42.258 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.258 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.258 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.258 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:51.927844+0000", 2024-09-14T06:17:42.259 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1524e-05 2024-09-14T06:17:42.259 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.259 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.259 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.259 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.259 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.259 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:27637 3.b 3:d7b02ca9:::rbd_data.1152ff705991.00000000000000a5:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:17:42.259 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:55.303878+0000", 2024-09-14T06:17:42.259 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 46.873618034000003, 2024-09-14T06:17:42.259 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.027469031000000001, 2024-09-14T06:17:42.260 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.260 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.260 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:42.260 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.260 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.260 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.260 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 27637 2024-09-14T06:17:42.260 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.260 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.260 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.260 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.260 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303878+0000", 2024-09-14T06:17:42.261 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.261 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.261 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.261 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.261 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303878+0000", 2024-09-14T06:17:42.261 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.261 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.261 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.261 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.261 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303881+0000", 2024-09-14T06:17:42.261 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8430000000000001e-06 2024-09-14T06:17:42.262 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.262 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.262 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.262 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303894+0000", 2024-09-14T06:17:42.262 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3376e-05 2024-09-14T06:17:42.262 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.262 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.262 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.262 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303926+0000", 2024-09-14T06:17:42.262 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1825000000000001e-05 2024-09-14T06:17:42.262 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.262 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.263 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.263 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.303941+0000", 2024-09-14T06:17:42.263 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5048999999999999e-05 2024-09-14T06:17:42.263 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.263 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.263 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.263 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.304106+0000", 2024-09-14T06:17:42.263 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00016504700000000001 2024-09-14T06:17:42.263 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.263 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.263 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.263 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.304357+0000", 2024-09-14T06:17:42.264 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025158899999999998 2024-09-14T06:17:42.264 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.264 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.264 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 6", 2024-09-14T06:17:42.264 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.308743+0000", 2024-09-14T06:17:42.264 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043854419999999998 2024-09-14T06:17:42.264 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.264 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.264 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:42.264 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.331026+0000", 2024-09-14T06:17:42.264 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022282999000000001 2024-09-14T06:17:42.264 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.265 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.265 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:42.265 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.331072+0000", 2024-09-14T06:17:42.265 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5694e-05 2024-09-14T06:17:42.265 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.265 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.265 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:42.265 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.331229+0000", 2024-09-14T06:17:42.265 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015776399999999999 2024-09-14T06:17:42.265 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.265 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.265 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.266 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:55.331347+0000", 2024-09-14T06:17:42.266 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000117403 2024-09-14T06:17:42.266 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.266 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.266 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.266 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.266 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.266 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:64160 3.b 3:d685a927:::rbd_data.115267b2c56c.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 8388608~9437184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:17:42.266 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:38.910513+0000", 2024-09-14T06:17:42.266 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 3.2669824059999999, 2024-09-14T06:17:42.266 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.022446348000000001, 2024-09-14T06:17:42.266 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:42.266 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:42.267 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:42.267 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:42.267 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:42.267 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.267 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 64160 2024-09-14T06:17:42.267 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.267 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:42.267 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.267 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:42.267 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:38.910513+0000", 2024-09-14T06:17:42.267 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.268 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.268 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.268 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:42.268 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:38.910513+0000", 2024-09-14T06:17:42.268 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:42.268 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.268 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.268 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:42.268 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:38.910517+0000", 2024-09-14T06:17:42.268 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7570000000000002e-06 2024-09-14T06:17:42.268 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.269 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.269 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:42.269 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:38.917868+0000", 2024-09-14T06:17:42.269 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0073514210000000003 2024-09-14T06:17:42.269 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.269 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.269 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:42.269 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:38.917910+0000", 2024-09-14T06:17:42.269 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1482e-05 2024-09-14T06:17:42.269 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.270 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.270 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.270 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:38.917936+0000", 2024-09-14T06:17:42.270 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6040000000000001e-05 2024-09-14T06:17:42.270 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.270 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.270 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.270 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:38.918123+0000", 2024-09-14T06:17:42.270 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00018717000000000001 2024-09-14T06:17:42.270 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.271 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.271 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op must wait for map", 2024-09-14T06:17:42.271 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:38.918143+0000", 2024-09-14T06:17:42.271 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9862000000000001e-05 2024-09-14T06:17:42.271 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.271 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.271 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:42.271 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:38.921387+0000", 2024-09-14T06:17:42.271 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032441700000000002 2024-09-14T06:17:42.271 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.271 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.271 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:42.271 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:38.921686+0000", 2024-09-14T06:17:42.272 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00029855700000000002 2024-09-14T06:17:42.272 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.272 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.272 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 6", 2024-09-14T06:17:42.272 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:38.922344+0000", 2024-09-14T06:17:42.272 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00065869099999999999 2024-09-14T06:17:42.272 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.272 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.272 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:42.272 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:38.931046+0000", 2024-09-14T06:17:42.272 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0087016980000000008 2024-09-14T06:17:42.272 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.273 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.273 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:42.273 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:38.932818+0000", 2024-09-14T06:17:42.273 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0017720050000000001 2024-09-14T06:17:42.273 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.273 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.273 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:42.273 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:38.932897+0000", 2024-09-14T06:17:42.273 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9072999999999993e-05 2024-09-14T06:17:42.273 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:42.273 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:42.273 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:42.274 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:38.932960+0000", 2024-09-14T06:17:42.274 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2422000000000006e-05 2024-09-14T06:17:42.274 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.274 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.274 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.274 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:42.274 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:42.274 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:42.276 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:42.272+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:42.276 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:17:42.376 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:42.376+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:42.412 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:42.412 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:42.412 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:42.412 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:42.426 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:17:42.477 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:42.474+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:42.554 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:42.554 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:42.554 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:42.554 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:42.554 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:42.569 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:17:42.578 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:42.572+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:42.679 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:42.674+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:42.753 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:42.753 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:42.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:42.753 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:42.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.753 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40393 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 868352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.753 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.708703+0000", 2024-09-14T06:17:42.754 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.041528030999999, 2024-09-14T06:17:42.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12669228199999999, 2024-09-14T06:17:42.754 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.754 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.754 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.754 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.754 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.754 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.754 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40393 2024-09-14T06:17:42.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.754 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708703+0000", 2024-09-14T06:17:42.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708703+0000", 2024-09-14T06:17:42.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708881+0000", 2024-09-14T06:17:42.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00017825 2024-09-14T06:17:42.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708885+0000", 2024-09-14T06:17:42.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.019e-06 2024-09-14T06:17:42.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708894+0000", 2024-09-14T06:17:42.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5469999999999997e-06 2024-09-14T06:17:42.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708900+0000", 2024-09-14T06:17:42.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.421e-06 2024-09-14T06:17:42.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835012+0000", 2024-09-14T06:17:42.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12611203400000001 2024-09-14T06:17:42.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835095+0000", 2024-09-14T06:17:42.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.229e-05 2024-09-14T06:17:42.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835395+0000", 2024-09-14T06:17:42.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000300721 2024-09-14T06:17:42.758 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.758 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.758 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.758 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40395 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 876544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.758 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.708914+0000", 2024-09-14T06:17:42.758 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.041317399, 2024-09-14T06:17:42.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12688781199999999, 2024-09-14T06:17:42.758 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.758 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.758 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.759 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.759 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.759 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.759 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40395 2024-09-14T06:17:42.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.759 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708914+0000", 2024-09-14T06:17:42.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708914+0000", 2024-09-14T06:17:42.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709267+0000", 2024-09-14T06:17:42.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00035336999999999999 2024-09-14T06:17:42.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709272+0000", 2024-09-14T06:17:42.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4430000000000002e-06 2024-09-14T06:17:42.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709283+0000", 2024-09-14T06:17:42.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1034e-05 2024-09-14T06:17:42.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709291+0000", 2024-09-14T06:17:42.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4739999999999999e-06 2024-09-14T06:17:42.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835428+0000", 2024-09-14T06:17:42.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126137052 2024-09-14T06:17:42.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835506+0000", 2024-09-14T06:17:42.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8492000000000007e-05 2024-09-14T06:17:42.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835801+0000", 2024-09-14T06:17:42.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00029494699999999999 2024-09-14T06:17:42.762 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.762 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.762 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.763 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40397 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.763 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.709307+0000", 2024-09-14T06:17:42.763 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.040924296, 2024-09-14T06:17:42.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126901031, 2024-09-14T06:17:42.763 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.763 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.763 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.763 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.763 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.763 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.763 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40397 2024-09-14T06:17:42.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.764 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709307+0000", 2024-09-14T06:17:42.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709307+0000", 2024-09-14T06:17:42.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709590+0000", 2024-09-14T06:17:42.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00028368499999999999 2024-09-14T06:17:42.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709596+0000", 2024-09-14T06:17:42.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3940000000000004e-06 2024-09-14T06:17:42.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709605+0000", 2024-09-14T06:17:42.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8650000000000007e-06 2024-09-14T06:17:42.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709610+0000", 2024-09-14T06:17:42.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1139999999999997e-06 2024-09-14T06:17:42.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835834+0000", 2024-09-14T06:17:42.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12622423199999999 2024-09-14T06:17:42.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835915+0000", 2024-09-14T06:17:42.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0859999999999995e-05 2024-09-14T06:17:42.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836208+0000", 2024-09-14T06:17:42.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00029288099999999999 2024-09-14T06:17:42.767 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.767 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.767 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.767 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40399 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.767 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.709624+0000", 2024-09-14T06:17:42.767 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.040606498999999, 2024-09-14T06:17:42.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.127024368, 2024-09-14T06:17:42.768 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.768 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.768 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.768 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.768 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.768 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.768 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40399 2024-09-14T06:17:42.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.768 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709624+0000", 2024-09-14T06:17:42.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709624+0000", 2024-09-14T06:17:42.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710144+0000", 2024-09-14T06:17:42.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00051909700000000005 2024-09-14T06:17:42.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710156+0000", 2024-09-14T06:17:42.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2846e-05 2024-09-14T06:17:42.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710166+0000", 2024-09-14T06:17:42.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:17:42.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710174+0000", 2024-09-14T06:17:42.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:17:42.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836240+0000", 2024-09-14T06:17:42.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12606634899999999 2024-09-14T06:17:42.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836320+0000", 2024-09-14T06:17:42.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0073000000000004e-05 2024-09-14T06:17:42.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836649+0000", 2024-09-14T06:17:42.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032834999999999999 2024-09-14T06:17:42.772 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.772 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.772 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.772 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40401 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.772 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.710189+0000", 2024-09-14T06:17:42.772 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.040042331999999, 2024-09-14T06:17:42.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126903455, 2024-09-14T06:17:42.772 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.772 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.772 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.773 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.773 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.773 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.773 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40401 2024-09-14T06:17:42.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.773 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710189+0000", 2024-09-14T06:17:42.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710189+0000", 2024-09-14T06:17:42.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712809+0000", 2024-09-14T06:17:42.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026201179999999998 2024-09-14T06:17:42.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712815+0000", 2024-09-14T06:17:42.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7960000000000001e-06 2024-09-14T06:17:42.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712825+0000", 2024-09-14T06:17:42.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0044000000000001e-05 2024-09-14T06:17:42.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712832+0000", 2024-09-14T06:17:42.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.328e-06 2024-09-14T06:17:42.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836682+0000", 2024-09-14T06:17:42.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.123850141 2024-09-14T06:17:42.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836762+0000", 2024-09-14T06:17:42.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0207e-05 2024-09-14T06:17:42.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.837092+0000", 2024-09-14T06:17:42.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000329821 2024-09-14T06:17:42.776 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.776 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.776 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.776 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40407 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.777 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:42.777 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.037348917000003, 2024-09-14T06:17:42.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.127190898, 2024-09-14T06:17:42.777 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.777 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.777 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.777 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.777 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.777 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.777 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40407 2024-09-14T06:17:42.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.777 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:42.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:42.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:42.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0900000000000002e-07 2024-09-14T06:17:42.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712884+0000", 2024-09-14T06:17:42.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5579999999999999e-06 2024-09-14T06:17:42.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712887+0000", 2024-09-14T06:17:42.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3119999999999998e-06 2024-09-14T06:17:42.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712890+0000", 2024-09-14T06:17:42.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7e-06 2024-09-14T06:17:42.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.839114+0000", 2024-09-14T06:17:42.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126223954 2024-09-14T06:17:42.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.839197+0000", 2024-09-14T06:17:42.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3250000000000004e-05 2024-09-14T06:17:42.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840073+0000", 2024-09-14T06:17:42.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000875715 2024-09-14T06:17:42.781 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.781 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.781 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.781 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40409 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.781 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:42.781 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.037337135999998, 2024-09-14T06:17:42.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12739108900000001, 2024-09-14T06:17:42.781 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.782 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.782 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.782 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.782 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.782 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.782 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40409 2024-09-14T06:17:42.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.782 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:42.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:42.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:42.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.65e-07 2024-09-14T06:17:42.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712895+0000", 2024-09-14T06:17:42.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.325e-06 2024-09-14T06:17:42.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712899+0000", 2024-09-14T06:17:42.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.219e-06 2024-09-14T06:17:42.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712901+0000", 2024-09-14T06:17:42.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7310000000000002e-06 2024-09-14T06:17:42.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.839921+0000", 2024-09-14T06:17:42.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12701995199999999 2024-09-14T06:17:42.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840006+0000", 2024-09-14T06:17:42.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5063999999999997e-05 2024-09-14T06:17:42.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840285+0000", 2024-09-14T06:17:42.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027853300000000002 2024-09-14T06:17:42.785 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.785 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.786 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.786 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40411 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.786 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712909+0000", 2024-09-14T06:17:42.786 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.037321765000002, 2024-09-14T06:17:42.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.128251319, 2024-09-14T06:17:42.786 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.786 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.786 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.786 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.786 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.786 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.787 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40411 2024-09-14T06:17:42.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.787 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.787 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712909+0000", 2024-09-14T06:17:42.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.787 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712909+0000", 2024-09-14T06:17:42.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712910+0000", 2024-09-14T06:17:42.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.39e-07 2024-09-14T06:17:42.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712911+0000", 2024-09-14T06:17:42.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.37e-06 2024-09-14T06:17:42.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712914+0000", 2024-09-14T06:17:42.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4999999999999999e-06 2024-09-14T06:17:42.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712929+0000", 2024-09-14T06:17:42.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4406000000000001e-05 2024-09-14T06:17:42.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840856+0000", 2024-09-14T06:17:42.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.127927546 2024-09-14T06:17:42.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840942+0000", 2024-09-14T06:17:42.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5232e-05 2024-09-14T06:17:42.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841161+0000", 2024-09-14T06:17:42.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021892600000000001 2024-09-14T06:17:42.790 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.790 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.790 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.791 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40413 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.791 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712938+0000", 2024-09-14T06:17:42.791 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.037292839000003, 2024-09-14T06:17:42.791 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.128612052, 2024-09-14T06:17:42.791 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.791 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.791 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.791 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.791 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.791 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.791 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40413 2024-09-14T06:17:42.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.791 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712938+0000", 2024-09-14T06:17:42.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712938+0000", 2024-09-14T06:17:42.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712939+0000", 2024-09-14T06:17:42.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1099999999999996e-07 2024-09-14T06:17:42.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712942+0000", 2024-09-14T06:17:42.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0920000000000002e-06 2024-09-14T06:17:42.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712949+0000", 2024-09-14T06:17:42.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.075e-06 2024-09-14T06:17:42.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712954+0000", 2024-09-14T06:17:42.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2229999999999998e-06 2024-09-14T06:17:42.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841192+0000", 2024-09-14T06:17:42.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12823817100000001 2024-09-14T06:17:42.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841280+0000", 2024-09-14T06:17:42.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8190999999999996e-05 2024-09-14T06:17:42.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841550+0000", 2024-09-14T06:17:42.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00026978899999999999 2024-09-14T06:17:42.795 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.795 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.795 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.795 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40415 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.795 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712962+0000", 2024-09-14T06:17:42.795 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.037268562000001, 2024-09-14T06:17:42.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12902585499999999, 2024-09-14T06:17:42.795 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.795 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.795 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.795 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.795 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.795 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.796 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40415 2024-09-14T06:17:42.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.796 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.796 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712962+0000", 2024-09-14T06:17:42.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.796 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712962+0000", 2024-09-14T06:17:42.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712963+0000", 2024-09-14T06:17:42.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:17:42.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712966+0000", 2024-09-14T06:17:42.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6649999999999999e-06 2024-09-14T06:17:42.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712972+0000", 2024-09-14T06:17:42.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4819999999999999e-06 2024-09-14T06:17:42.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712978+0000", 2024-09-14T06:17:42.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4299999999999997e-06 2024-09-14T06:17:42.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841581+0000", 2024-09-14T06:17:42.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12860382100000001 2024-09-14T06:17:42.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841665+0000", 2024-09-14T06:17:42.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3228999999999999e-05 2024-09-14T06:17:42.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841988+0000", 2024-09-14T06:17:42.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032370599999999999 2024-09-14T06:17:42.799 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.799 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.799 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.799 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40417 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.799 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712983+0000", 2024-09-14T06:17:42.799 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.037247827999998, 2024-09-14T06:17:42.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129525837, 2024-09-14T06:17:42.800 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.800 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.800 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.800 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.800 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.800 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.800 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40417 2024-09-14T06:17:42.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.800 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.800 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712983+0000", 2024-09-14T06:17:42.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712983+0000", 2024-09-14T06:17:42.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713182+0000", 2024-09-14T06:17:42.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000198569 2024-09-14T06:17:42.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713186+0000", 2024-09-14T06:17:42.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6160000000000004e-06 2024-09-14T06:17:42.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713197+0000", 2024-09-14T06:17:42.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.046e-05 2024-09-14T06:17:42.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713204+0000", 2024-09-14T06:17:42.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.515e-06 2024-09-14T06:17:42.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842021+0000", 2024-09-14T06:17:42.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12881625199999999 2024-09-14T06:17:42.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842158+0000", 2024-09-14T06:17:42.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013778099999999999 2024-09-14T06:17:42.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842509+0000", 2024-09-14T06:17:42.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000350644 2024-09-14T06:17:42.803 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.803 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.804 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.804 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40419 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.804 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.713217+0000", 2024-09-14T06:17:42.804 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.037014290999998, 2024-09-14T06:17:42.804 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12982854599999999, 2024-09-14T06:17:42.804 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.804 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.804 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.804 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.804 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.804 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.805 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40419 2024-09-14T06:17:42.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.805 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.805 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.805 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713217+0000", 2024-09-14T06:17:42.805 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.805 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.805 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713217+0000", 2024-09-14T06:17:42.805 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.806 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.806 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713831+0000", 2024-09-14T06:17:42.806 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061477700000000005 2024-09-14T06:17:42.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.806 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.806 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.806 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713841+0000", 2024-09-14T06:17:42.806 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3789999999999995e-06 2024-09-14T06:17:42.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.806 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.806 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.806 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713853+0000", 2024-09-14T06:17:42.806 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1952e-05 2024-09-14T06:17:42.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.806 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.806 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.807 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713861+0000", 2024-09-14T06:17:42.807 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4500000000000004e-06 2024-09-14T06:17:42.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.807 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.807 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842560+0000", 2024-09-14T06:17:42.807 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12869836900000001 2024-09-14T06:17:42.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.807 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.807 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842655+0000", 2024-09-14T06:17:42.807 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5617000000000003e-05 2024-09-14T06:17:42.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.807 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.808 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843045+0000", 2024-09-14T06:17:42.808 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00039000200000000002 2024-09-14T06:17:42.808 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.808 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.808 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.808 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.808 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40421 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.808 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.713882+0000", 2024-09-14T06:17:42.808 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.036348924000002, 2024-09-14T06:17:42.808 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12974492600000001, 2024-09-14T06:17:42.808 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.808 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.809 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.809 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.809 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.809 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.809 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40421 2024-09-14T06:17:42.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.809 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.809 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.809 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.809 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713882+0000", 2024-09-14T06:17:42.809 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.809 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.809 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.809 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713882+0000", 2024-09-14T06:17:42.810 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.810 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.810 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.810 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714044+0000", 2024-09-14T06:17:42.810 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016224999999999999 2024-09-14T06:17:42.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.810 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.810 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.810 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714049+0000", 2024-09-14T06:17:42.810 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0150000000000003e-06 2024-09-14T06:17:42.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.810 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.810 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.811 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714068+0000", 2024-09-14T06:17:42.811 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8749e-05 2024-09-14T06:17:42.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.811 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.811 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714077+0000", 2024-09-14T06:17:42.811 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4379999999999995e-06 2024-09-14T06:17:42.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.811 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.811 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843079+0000", 2024-09-14T06:17:42.811 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129002958 2024-09-14T06:17:42.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.812 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.812 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843290+0000", 2024-09-14T06:17:42.812 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021005899999999999 2024-09-14T06:17:42.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.812 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.812 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.812 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843627+0000", 2024-09-14T06:17:42.812 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00033745699999999999 2024-09-14T06:17:42.812 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.812 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.812 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.812 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.813 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40423 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.813 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.714090+0000", 2024-09-14T06:17:42.813 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.036140553000003, 2024-09-14T06:17:42.813 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129886166, 2024-09-14T06:17:42.813 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.813 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.813 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.813 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.813 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.813 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.813 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40423 2024-09-14T06:17:42.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.813 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.814 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.814 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714090+0000", 2024-09-14T06:17:42.814 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.814 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.814 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714090+0000", 2024-09-14T06:17:42.814 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.814 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.814 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714456+0000", 2024-09-14T06:17:42.814 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000366058 2024-09-14T06:17:42.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.815 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.815 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714461+0000", 2024-09-14T06:17:42.815 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.228e-06 2024-09-14T06:17:42.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.815 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.815 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714469+0000", 2024-09-14T06:17:42.815 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6589999999999996e-06 2024-09-14T06:17:42.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.815 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.815 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714476+0000", 2024-09-14T06:17:42.816 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:17:42.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.816 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.816 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.816 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843659+0000", 2024-09-14T06:17:42.816 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129183137 2024-09-14T06:17:42.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.816 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.816 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.816 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843745+0000", 2024-09-14T06:17:42.816 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5354999999999998e-05 2024-09-14T06:17:42.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.816 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.816 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.817 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843977+0000", 2024-09-14T06:17:42.817 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000231883 2024-09-14T06:17:42.817 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.817 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.817 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.817 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.817 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.817 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40425 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.817 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.714490+0000", 2024-09-14T06:17:42.817 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.035741358000003, 2024-09-14T06:17:42.817 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13064684600000001, 2024-09-14T06:17:42.817 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.817 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.817 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.818 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.818 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.818 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.818 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40425 2024-09-14T06:17:42.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.818 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.818 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.818 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.818 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714490+0000", 2024-09-14T06:17:42.818 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.818 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.818 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.818 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714490+0000", 2024-09-14T06:17:42.819 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.819 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.819 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.819 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714725+0000", 2024-09-14T06:17:42.819 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000235438 2024-09-14T06:17:42.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.819 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.819 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.819 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714729+0000", 2024-09-14T06:17:42.819 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8909999999999997e-06 2024-09-14T06:17:42.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.819 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.819 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.820 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714737+0000", 2024-09-14T06:17:42.820 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5329999999999994e-06 2024-09-14T06:17:42.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.820 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.820 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.820 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714742+0000", 2024-09-14T06:17:42.820 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9559999999999996e-06 2024-09-14T06:17:42.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.820 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.820 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.820 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.844184+0000", 2024-09-14T06:17:42.820 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129441954 2024-09-14T06:17:42.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.821 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.821 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.821 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.844272+0000", 2024-09-14T06:17:42.821 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7151999999999995e-05 2024-09-14T06:17:42.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.821 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.821 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.821 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845136+0000", 2024-09-14T06:17:42.821 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00086492199999999996 2024-09-14T06:17:42.821 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.821 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.821 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.821 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.822 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40426 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.822 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.714753+0000", 2024-09-14T06:17:42.822 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.035478394000002, 2024-09-14T06:17:42.822 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.130693374, 2024-09-14T06:17:42.822 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.822 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.822 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.822 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.822 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.822 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.822 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40426 2024-09-14T06:17:42.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.822 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.823 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.823 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714753+0000", 2024-09-14T06:17:42.823 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.823 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.823 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714753+0000", 2024-09-14T06:17:42.823 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.823 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.823 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715118+0000", 2024-09-14T06:17:42.823 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036512099999999999 2024-09-14T06:17:42.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.824 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.824 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715123+0000", 2024-09-14T06:17:42.824 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9910000000000002e-06 2024-09-14T06:17:42.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.824 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.824 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715141+0000", 2024-09-14T06:17:42.824 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8219999999999998e-05 2024-09-14T06:17:42.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.824 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.824 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715149+0000", 2024-09-14T06:17:42.824 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8979999999999996e-06 2024-09-14T06:17:42.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.825 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.825 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845170+0000", 2024-09-14T06:17:42.825 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13002106799999999 2024-09-14T06:17:42.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.825 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.825 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845249+0000", 2024-09-14T06:17:42.825 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9163000000000001e-05 2024-09-14T06:17:42.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.825 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.825 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845446+0000", 2024-09-14T06:17:42.825 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196913 2024-09-14T06:17:42.825 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.826 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.826 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.826 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.826 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40428 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.826 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.715166+0000", 2024-09-14T06:17:42.826 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.035065443000001, 2024-09-14T06:17:42.826 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13071775199999999, 2024-09-14T06:17:42.826 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.826 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.826 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:42.826 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.826 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.826 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.827 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40428 2024-09-14T06:17:42.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.827 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.827 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.827 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715166+0000", 2024-09-14T06:17:42.827 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.827 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.827 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715166+0000", 2024-09-14T06:17:42.827 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.828 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.828 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715361+0000", 2024-09-14T06:17:42.828 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00019536400000000001 2024-09-14T06:17:42.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.828 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.828 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.828 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715366+0000", 2024-09-14T06:17:42.828 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4589999999999997e-06 2024-09-14T06:17:42.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.828 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.828 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.828 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715387+0000", 2024-09-14T06:17:42.828 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0829e-05 2024-09-14T06:17:42.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.829 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.829 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715402+0000", 2024-09-14T06:17:42.829 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4429e-05 2024-09-14T06:17:42.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.829 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.829 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845476+0000", 2024-09-14T06:17:42.829 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13007447999999999 2024-09-14T06:17:42.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.829 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:42.830 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845557+0000", 2024-09-14T06:17:42.830 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1100000000000006e-05 2024-09-14T06:17:42.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.830 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.830 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845883+0000", 2024-09-14T06:17:42.830 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032609099999999998 2024-09-14T06:17:42.830 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.830 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.830 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.830 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40430 3.15 3.aecb2ff5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.830 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.715416+0000", 2024-09-14T06:17:42.830 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.034814978, 2024-09-14T06:17:42.831 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13113939699999999, 2024-09-14T06:17:42.831 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.831 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.831 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:42.831 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.831 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.831 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.831 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40430 2024-09-14T06:17:42.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.831 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.831 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.831 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.831 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715416+0000", 2024-09-14T06:17:42.831 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.832 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.832 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.832 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715416+0000", 2024-09-14T06:17:42.832 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.832 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.832 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.832 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715995+0000", 2024-09-14T06:17:42.832 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057881100000000004 2024-09-14T06:17:42.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.832 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.832 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.832 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716000+0000", 2024-09-14T06:17:42.832 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.677e-06 2024-09-14T06:17:42.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.833 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.833 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716013+0000", 2024-09-14T06:17:42.833 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2326e-05 2024-09-14T06:17:42.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.833 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.833 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716021+0000", 2024-09-14T06:17:42.833 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9750000000000007e-06 2024-09-14T06:17:42.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.833 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.833 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846538+0000", 2024-09-14T06:17:42.834 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13051717400000001 2024-09-14T06:17:42.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.834 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.834 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846555+0000", 2024-09-14T06:17:42.834 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7433999999999999e-05 2024-09-14T06:17:42.834 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.834 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.834 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.834 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40432 3.15 3.aecb2ff5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.834 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:42.834 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.034195339, 2024-09-14T06:17:42.834 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13055824499999999, 2024-09-14T06:17:42.835 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.835 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.835 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:42.835 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.835 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.835 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.835 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40432 2024-09-14T06:17:42.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.835 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.835 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.835 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:42.835 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.836 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.836 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:42.836 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.836 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.836 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.836 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:42.836 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9599999999999998e-07 2024-09-14T06:17:42.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.836 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.836 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.836 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716040+0000", 2024-09-14T06:17:42.836 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.399e-06 2024-09-14T06:17:42.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.836 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.837 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.837 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716048+0000", 2024-09-14T06:17:42.837 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2179999999999999e-06 2024-09-14T06:17:42.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.837 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.837 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.837 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716054+0000", 2024-09-14T06:17:42.837 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2110000000000003e-06 2024-09-14T06:17:42.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.837 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.837 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.837 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846586+0000", 2024-09-14T06:17:42.837 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.130531921 2024-09-14T06:17:42.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.837 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.838 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.838 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846594+0000", 2024-09-14T06:17:42.838 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7000000000000008e-06 2024-09-14T06:17:42.838 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.838 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.838 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.838 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.838 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40434 3.15 3.aecb2ff5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:42.838 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.716065+0000", 2024-09-14T06:17:42.838 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.034165512000001, 2024-09-14T06:17:42.838 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13055924899999999, 2024-09-14T06:17:42.838 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:42.838 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:42.839 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:42.839 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:42.839 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:42.839 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:42.839 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40434 2024-09-14T06:17:42.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.839 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:42.839 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.839 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:42.839 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716065+0000", 2024-09-14T06:17:42.839 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.839 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.839 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:42.840 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716065+0000", 2024-09-14T06:17:42.840 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:42.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.840 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.840 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:42.840 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.723984+0000", 2024-09-14T06:17:42.840 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0079189670000000007 2024-09-14T06:17:42.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.840 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.840 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:42.840 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.723991+0000", 2024-09-14T06:17:42.840 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1759999999999998e-06 2024-09-14T06:17:42.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.840 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.840 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:42.841 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.724003+0000", 2024-09-14T06:17:42.841 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1914e-05 2024-09-14T06:17:42.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.841 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.841 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:42.841 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.724013+0000", 2024-09-14T06:17:42.841 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441999999999999e-05 2024-09-14T06:17:42.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.841 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.841 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:42.841 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846617+0000", 2024-09-14T06:17:42.841 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12260452199999999 2024-09-14T06:17:42.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:42.841 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:42.842 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:42.842 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846625+0000", 2024-09-14T06:17:42.842 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2280000000000001e-06 2024-09-14T06:17:42.842 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.842 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.842 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.842 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:42.842 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:42.842 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:42.843 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:17:42.844 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:17:42.845 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:42.842+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:42.945 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:42.942+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:43.020 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:43.020 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:43.020 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:43.020 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:43.037 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:17:43.079 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:43.078+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:43.145 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:43.142+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:43.197 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:43.197 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:43.197 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:43.197 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:43.197 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:43.211 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:17:43.214 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:43.215 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:17:43.234 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-09-14T06:17:42.648844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1349972343,"num_objects":1569,"num_object_clones":268,"num_object_copies":3138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1569,"num_whiteouts":388,"num_read":99096,"num_read_kb":744202,"num_write":15529,"num_write_kb":430335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1127,"num_bytes_recovered":140497553,"num_keys_recovered":151,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12951,"ondisk_log_size":12951,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":412836,"kb_used_data":71768,"kb_used_omap":12,"kb_used_meta":340915,"kb_avail":754561884,"statfs":{"total":773094113280,"available":772671369216,"internally_reserved":0,"allocated":73490432,"data_stored":630343030,"data_compressed":17264948,"data_compressed_allocated":42393600,"data_compressed_original":606782774,"omap_allocated":12752,"internal_metadata":349097520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":57090751,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":4029,"num_read_kb":12770,"num_write":733,"num_write_kb":30595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":430,"num_bytes_recovered":49747746,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.431201"},"pg_stats":[{"pgid":"3.1f","version":"86'516","reported_seq":3279,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.624592+0000","last_change":"2024-09-14T06:17:02.195565+0000","last_active":"2024-09-14T06:17:41.624592+0000","last_peered":"2024-09-14T06:17:41.624592+0000","last_clean":"2024-09-14T06:17:41.624592+0000","last_became_active":"2024-09-14T06:17:02.193313+0000","last_became_peered":"2024-09-14T06:17:02.193313+0000","last_unstale":"2024-09-14T06:17:41.624592+0000","last_undegraded":"2024-09-14T06:17:41.624592+0000","last_fullsized":"2024-09-14T06:17:41.624592+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:51:34.761417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010466430000000001,"stat_sum":{"num_bytes":54336740,"num_objects":48,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":10,"num_read":4340,"num_read_kb":53375,"num_write":644,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"86'333","reported_seq":4121,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.624333+0000","last_change":"2024-09-14T06:16:27.310639+0000","last_active":"2024-09-14T06:17:41.624333+0000","last_peered":"2024-09-14T06:17:41.624333+0000","last_clean":"2024-09-14T06:17:41.624333+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:17:41.624333+0000","last_undegraded":"2024-09-14T06:17:41.624333+0000","last_fullsized":"2024-09-14T06:17:41.624333+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:11:42.513574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095849699999999999,"stat_sum":{"num_bytes":63404700,"num_objects":48,"num_object_clones":9,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":15,"num_read":4742,"num_read_kb":42827,"num_write":298,"num_write_kb":15475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"87'1042","reported_seq":2133,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:42.020291+0000","last_change":"2024-09-14T06:17:38.227919+0000","last_active":"2024-09-14T06:17:42.020291+0000","last_peered":"2024-09-14T06:17:42.020291+0000","last_clean":"2024-09-14T06:17:42.020291+0000","last_became_active":"2024-09-14T06:17:38.226838+0000","last_became_peered":"2024-09-14T06:17:38.226838+0000","last_unstale":"2024-09-14T06:17:42.020291+0000","last_undegraded":"2024-09-14T06:17:42.020291+0000","last_fullsized":"2024-09-14T06:17:42.020291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":1042,"log_dups_size":0,"ondisk_log_size":1042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:07:02.490072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001319401,"stat_sum":{"num_bytes":32292365,"num_objects":49,"num_object_clones":7,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":8,"num_read":4901,"num_read_kb":14188,"num_write":1503,"num_write_kb":13979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10785266,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"87'823","reported_seq":2467,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.876630+0000","last_change":"2024-09-14T06:16:22.201834+0000","last_active":"2024-09-14T06:17:41.876630+0000","last_peered":"2024-09-14T06:17:41.876630+0000","last_clean":"2024-09-14T06:17:41.876630+0000","last_became_active":"2024-09-14T06:16:20.190215+0000","last_became_peered":"2024-09-14T06:16:20.190215+0000","last_unstale":"2024-09-14T06:17:41.876630+0000","last_undegraded":"2024-09-14T06:17:41.876630+0000","last_fullsized":"2024-09-14T06:17:41.876630+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:08:24.896344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026065160000000001,"stat_sum":{"num_bytes":25862491,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":12,"num_read":5522,"num_read_kb":9572,"num_write":1222,"num_write_kb":6513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"79'291","reported_seq":3517,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.618918+0000","last_change":"2024-09-14T06:16:22.193116+0000","last_active":"2024-09-14T06:17:41.618918+0000","last_peered":"2024-09-14T06:17:41.618918+0000","last_clean":"2024-09-14T06:17:41.618918+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:17:41.618918+0000","last_undegraded":"2024-09-14T06:17:41.618918+0000","last_fullsized":"2024-09-14T06:17:41.618918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:17:00.634226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014745120000000001,"stat_sum":{"num_bytes":52171639,"num_objects":51,"num_object_clones":9,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":13,"num_read":3322,"num_read_kb":39042,"num_write":328,"num_write_kb":25723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"79'468","reported_seq":2765,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.987204+0000","last_change":"2024-09-14T06:16:22.201461+0000","last_active":"2024-09-14T06:17:41.987204+0000","last_peered":"2024-09-14T06:17:41.987204+0000","last_clean":"2024-09-14T06:17:41.987204+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:17:41.987204+0000","last_undegraded":"2024-09-14T06:17:41.987204+0000","last_fullsized":"2024-09-14T06:17:41.987204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:21:54.740165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002267542,"stat_sum":{"num_bytes":12869368,"num_objects":53,"num_object_clones":8,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":22,"num_read":3666,"num_read_kb":12739,"num_write":607,"num_write_kb":7500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"80'449","reported_seq":1765,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619096+0000","last_change":"2024-09-14T06:16:22.191773+0000","last_active":"2024-09-14T06:17:41.619096+0000","last_peered":"2024-09-14T06:17:41.619096+0000","last_clean":"2024-09-14T06:17:41.619096+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:17:41.619096+0000","last_undegraded":"2024-09-14T06:17:41.619096+0000","last_fullsized":"2024-09-14T06:17:41.619096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:45:42.492916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073833899999999997,"stat_sum":{"num_bytes":35400414,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":2666,"num_read_kb":19057,"num_write":647,"num_write_kb":12862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"80'307","reported_seq":2595,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.624167+0000","last_change":"2024-09-14T06:16:22.205296+0000","last_active":"2024-09-14T06:17:41.624167+0000","last_peered":"2024-09-14T06:17:41.624167+0000","last_clean":"2024-09-14T06:17:41.624167+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:17:41.624167+0000","last_undegraded":"2024-09-14T06:17:41.624167+0000","last_fullsized":"2024-09-14T06:17:41.624167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:57:31.710099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012474280000000001,"stat_sum":{"num_bytes":44544444,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":11,"num_read":3415,"num_read_kb":37447,"num_write":297,"num_write_kb":32929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"86'315","reported_seq":1369,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.625010+0000","last_change":"2024-09-14T06:17:37.234947+0000","last_active":"2024-09-14T06:17:41.625010+0000","last_peered":"2024-09-14T06:17:41.625010+0000","last_clean":"2024-09-14T06:17:41.625010+0000","last_became_active":"2024-09-14T06:17:37.233410+0000","last_became_peered":"2024-09-14T06:17:37.233410+0000","last_unstale":"2024-09-14T06:17:41.625010+0000","last_undegraded":"2024-09-14T06:17:41.625010+0000","last_fullsized":"2024-09-14T06:17:41.625010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:51:53.583723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001106627,"stat_sum":{"num_bytes":27316305,"num_objects":48,"num_object_clones":12,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":16,"num_read":1181,"num_read_kb":17277,"num_write":285,"num_write_kb":17182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":23467377,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"87'355","reported_seq":2005,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.621585+0000","last_change":"2024-09-14T06:17:37.236314+0000","last_active":"2024-09-14T06:17:41.621585+0000","last_peered":"2024-09-14T06:17:41.621585+0000","last_clean":"2024-09-14T06:17:41.621585+0000","last_became_active":"2024-09-14T06:17:37.235311+0000","last_became_peered":"2024-09-14T06:17:37.235311+0000","last_unstale":"2024-09-14T06:17:41.621585+0000","last_undegraded":"2024-09-14T06:17:41.621585+0000","last_fullsized":"2024-09-14T06:17:41.621585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:09.235644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001557389,"stat_sum":{"num_bytes":57848052,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":11,"num_read":2242,"num_read_kb":16285,"num_write":452,"num_write_kb":10272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":6785399,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"87'350","reported_seq":3401,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.643667+0000","last_change":"2024-09-14T06:16:22.194304+0000","last_active":"2024-09-14T06:17:41.643667+0000","last_peered":"2024-09-14T06:17:41.643667+0000","last_clean":"2024-09-14T06:17:41.643667+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:17:41.643667+0000","last_undegraded":"2024-09-14T06:17:41.643667+0000","last_fullsized":"2024-09-14T06:17:41.643667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:32:31.047615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019733379999999998,"stat_sum":{"num_bytes":71775725,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":7,"num_read":3901,"num_read_kb":64430,"num_write":448,"num_write_kb":22209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"80'246","reported_seq":1698,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.630309+0000","last_change":"2024-09-14T06:17:37.232535+0000","last_active":"2024-09-14T06:17:41.630309+0000","last_peered":"2024-09-14T06:17:41.630309+0000","last_clean":"2024-09-14T06:17:41.630309+0000","last_became_active":"2024-09-14T06:17:37.231244+0000","last_became_peered":"2024-09-14T06:17:37.231244+0000","last_unstale":"2024-09-14T06:17:41.630309+0000","last_undegraded":"2024-09-14T06:17:41.630309+0000","last_fullsized":"2024-09-14T06:17:41.630309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:22:59.070149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001465247,"stat_sum":{"num_bytes":61166345,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":10,"num_read":2671,"num_read_kb":26987,"num_write":230,"num_write_kb":15728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":32506492,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619617+0000","last_change":"2024-09-14T06:17:34.200709+0000","last_active":"2024-09-14T06:17:41.619617+0000","last_peered":"2024-09-14T06:17:41.619617+0000","last_clean":"2024-09-14T06:17:41.619617+0000","last_became_active":"2024-09-14T06:17:34.200238+0000","last_became_peered":"2024-09-14T06:17:34.200238+0000","last_unstale":"2024-09-14T06:17:41.619617+0000","last_undegraded":"2024-09-14T06:17:41.619617+0000","last_fullsized":"2024-09-14T06:17:41.619617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:34.864695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"79'364","reported_seq":914,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.618987+0000","last_change":"2024-09-14T06:16:22.192972+0000","last_active":"2024-09-14T06:17:41.618987+0000","last_peered":"2024-09-14T06:17:41.618987+0000","last_clean":"2024-09-14T06:17:41.618987+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:17:41.618987+0000","last_undegraded":"2024-09-14T06:17:41.618987+0000","last_fullsized":"2024-09-14T06:17:41.618987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:56:10.514649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086504599999999996,"stat_sum":{"num_bytes":29669358,"num_objects":48,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":10,"num_read":1315,"num_read_kb":9524,"num_write":422,"num_write_kb":9131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619067+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:17:41.619067+0000","last_peered":"2024-09-14T06:17:41.619067+0000","last_clean":"2024-09-14T06:17:41.619067+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:17:41.619067+0000","last_undegraded":"2024-09-14T06:17:41.619067+0000","last_fullsized":"2024-09-14T06:17:41.619067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"86'300","reported_seq":2898,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.632160+0000","last_change":"2024-09-14T06:16:22.192683+0000","last_active":"2024-09-14T06:17:41.632160+0000","last_peered":"2024-09-14T06:17:41.632160+0000","last_clean":"2024-09-14T06:17:41.632160+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:17:41.632160+0000","last_undegraded":"2024-09-14T06:17:41.632160+0000","last_fullsized":"2024-09-14T06:17:41.632160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:04:04.548750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001328716,"stat_sum":{"num_bytes":33183963,"num_objects":53,"num_object_clones":9,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":11,"num_read":3071,"num_read_kb":28400,"num_write":271,"num_write_kb":15212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619874+0000","last_change":"2024-09-14T06:17:34.196668+0000","last_active":"2024-09-14T06:17:41.619874+0000","last_peered":"2024-09-14T06:17:41.619874+0000","last_clean":"2024-09-14T06:17:41.619874+0000","last_became_active":"2024-09-14T06:17:34.196072+0000","last_became_peered":"2024-09-14T06:17:34.196072+0000","last_unstale":"2024-09-14T06:17:41.619874+0000","last_undegraded":"2024-09-14T06:17:41.619874+0000","last_fullsized":"2024-09-14T06:17:41.619874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:18:45.616834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"86'297","reported_seq":1619,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.627187+0000","last_change":"2024-09-14T06:16:22.213152+0000","last_active":"2024-09-14T06:17:41.627187+0000","last_peered":"2024-09-14T06:17:41.627187+0000","last_clean":"2024-09-14T06:17:41.627187+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:17:41.627187+0000","last_undegraded":"2024-09-14T06:17:41.627187+0000","last_fullsized":"2024-09-14T06:17:41.627187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:28:34.843601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057590699999999996,"stat_sum":{"num_bytes":57727010,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":8,"num_read":1567,"num_read_kb":12759,"num_write":257,"num_write_kb":11518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.624824+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:17:41.624824+0000","last_peered":"2024-09-14T06:17:41.624824+0000","last_clean":"2024-09-14T06:17:41.624824+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:17:41.624824+0000","last_undegraded":"2024-09-14T06:17:41.624824+0000","last_fullsized":"2024-09-14T06:17:41.624824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"79'215","reported_seq":1440,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.624076+0000","last_change":"2024-09-14T06:16:33.548583+0000","last_active":"2024-09-14T06:17:41.624076+0000","last_peered":"2024-09-14T06:17:41.624076+0000","last_clean":"2024-09-14T06:17:41.624076+0000","last_became_active":"2024-09-14T06:16:33.547925+0000","last_became_peered":"2024-09-14T06:16:33.547925+0000","last_unstale":"2024-09-14T06:17:41.624076+0000","last_undegraded":"2024-09-14T06:17:41.624076+0000","last_fullsized":"2024-09-14T06:17:41.624076+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:23:39.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089329700000000004,"stat_sum":{"num_bytes":69139655,"num_objects":49,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":16,"num_read":1310,"num_read_kb":21229,"num_write":226,"num_write_kb":9950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"87'228","reported_seq":702,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.956273+0000","last_change":"2024-09-14T06:16:27.282745+0000","last_active":"2024-09-14T06:17:41.956273+0000","last_peered":"2024-09-14T06:17:41.956273+0000","last_clean":"2024-09-14T06:17:41.956273+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:17:41.956273+0000","last_undegraded":"2024-09-14T06:17:41.956273+0000","last_fullsized":"2024-09-14T06:17:41.956273+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:53:56.212817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002354358,"stat_sum":{"num_bytes":38932892,"num_objects":48,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":361,"num_read_kb":4069,"num_write":165,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619498+0000","last_change":"2024-09-14T06:17:34.203977+0000","last_active":"2024-09-14T06:17:41.619498+0000","last_peered":"2024-09-14T06:17:41.619498+0000","last_clean":"2024-09-14T06:17:41.619498+0000","last_became_active":"2024-09-14T06:17:34.199244+0000","last_became_peered":"2024-09-14T06:17:34.199244+0000","last_unstale":"2024-09-14T06:17:41.619498+0000","last_undegraded":"2024-09-14T06:17:41.619498+0000","last_fullsized":"2024-09-14T06:17:41.619498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:21.934094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"80'351","reported_seq":1584,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.624008+0000","last_change":"2024-09-14T06:16:22.205051+0000","last_active":"2024-09-14T06:17:41.624008+0000","last_peered":"2024-09-14T06:17:41.624008+0000","last_clean":"2024-09-14T06:17:41.624008+0000","last_became_active":"2024-09-14T06:16:20.196215+0000","last_became_peered":"2024-09-14T06:16:20.196215+0000","last_unstale":"2024-09-14T06:17:41.624008+0000","last_undegraded":"2024-09-14T06:17:41.624008+0000","last_fullsized":"2024-09-14T06:17:41.624008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:42:24.322885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001116559,"stat_sum":{"num_bytes":33594565,"num_objects":40,"num_object_clones":3,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":9,"num_read":2026,"num_read_kb":18917,"num_write":467,"num_write_kb":11041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T06:17:38.921109+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:17:38.921109+0000","last_peered":"2024-09-14T06:17:38.921109+0000","last_clean":"2024-09-14T06:17:38.921109+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:17:38.921109+0000","last_undegraded":"2024-09-14T06:17:38.921109+0000","last_fullsized":"2024-09-14T06:17:38.921109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'484","reported_seq":1917,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.624452+0000","last_change":"2024-09-14T06:16:27.316499+0000","last_active":"2024-09-14T06:17:41.624452+0000","last_peered":"2024-09-14T06:17:41.624452+0000","last_clean":"2024-09-14T06:17:41.624452+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:17:41.624452+0000","last_undegraded":"2024-09-14T06:17:41.624452+0000","last_fullsized":"2024-09-14T06:17:41.624452+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:10:23.624742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052773999999999996,"stat_sum":{"num_bytes":17346293,"num_objects":37,"num_object_clones":4,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":9,"num_read":2816,"num_read_kb":8761,"num_write":671,"num_write_kb":8775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619379+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:17:41.619379+0000","last_peered":"2024-09-14T06:17:41.619379+0000","last_clean":"2024-09-14T06:17:41.619379+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:17:41.619379+0000","last_undegraded":"2024-09-14T06:17:41.619379+0000","last_fullsized":"2024-09-14T06:17:41.619379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"86'368","reported_seq":1546,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.618904+0000","last_change":"2024-09-14T06:16:27.302045+0000","last_active":"2024-09-14T06:17:41.618904+0000","last_peered":"2024-09-14T06:17:41.618904+0000","last_clean":"2024-09-14T06:17:41.618904+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:17:41.618904+0000","last_undegraded":"2024-09-14T06:17:41.618904+0000","last_fullsized":"2024-09-14T06:17:41.618904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:33:28.609283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060103200000000004,"stat_sum":{"num_bytes":62676633,"num_objects":52,"num_object_clones":6,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":8,"num_read":1774,"num_read_kb":22633,"num_write":296,"num_write_kb":14511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":171,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619701+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:17:41.619701+0000","last_peered":"2024-09-14T06:17:41.619701+0000","last_clean":"2024-09-14T06:17:41.619701+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:17:41.619701+0000","last_undegraded":"2024-09-14T06:17:41.619701+0000","last_fullsized":"2024-09-14T06:17:41.619701+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":241,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.616159+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:17:41.616159+0000","last_peered":"2024-09-14T06:17:41.616159+0000","last_clean":"2024-09-14T06:17:41.616159+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:17:41.616159+0000","last_undegraded":"2024-09-14T06:17:41.616159+0000","last_fullsized":"2024-09-14T06:17:41.616159+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"79'379","reported_seq":1368,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.630646+0000","last_change":"2024-09-14T06:16:22.191260+0000","last_active":"2024-09-14T06:17:41.630646+0000","last_peered":"2024-09-14T06:17:41.630646+0000","last_clean":"2024-09-14T06:17:41.630646+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:17:41.630646+0000","last_undegraded":"2024-09-14T06:17:41.630646+0000","last_fullsized":"2024-09-14T06:17:41.630646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:05:10.411581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047498199999999999,"stat_sum":{"num_bytes":12407027,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":14,"num_read":2048,"num_read_kb":6659,"num_write":539,"num_write_kb":6840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"87'344","reported_seq":1585,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619487+0000","last_change":"2024-09-14T06:17:38.225097+0000","last_active":"2024-09-14T06:17:41.619487+0000","last_peered":"2024-09-14T06:17:41.619487+0000","last_clean":"2024-09-14T06:17:41.619487+0000","last_became_active":"2024-09-14T06:17:38.224238+0000","last_became_peered":"2024-09-14T06:17:38.224238+0000","last_unstale":"2024-09-14T06:17:41.619487+0000","last_undegraded":"2024-09-14T06:17:41.619487+0000","last_fullsized":"2024-09-14T06:17:41.619487+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:54.197052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418299,"stat_sum":{"num_bytes":42339757,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":8,"num_read":2186,"num_read_kb":13625,"num_write":509,"num_write_kb":14348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8477378,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"87'311","reported_seq":2020,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.924951+0000","last_change":"2024-09-14T06:16:22.323725+0000","last_active":"2024-09-14T06:17:41.924951+0000","last_peered":"2024-09-14T06:17:41.924951+0000","last_clean":"2024-09-14T06:17:41.924951+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:17:41.924951+0000","last_undegraded":"2024-09-14T06:17:41.924951+0000","last_fullsized":"2024-09-14T06:17:41.924951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:46:42.122526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024221899999999998,"stat_sum":{"num_bytes":35230985,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":8,"num_read":2159,"num_read_kb":16053,"num_write":266,"num_write_kb":13801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"86'245","reported_seq":2529,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.471334+0000","last_change":"2024-09-14T06:16:22.219322+0000","last_active":"2024-09-14T06:17:41.471334+0000","last_peered":"2024-09-14T06:17:41.471334+0000","last_clean":"2024-09-14T06:17:41.471334+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:17:41.471334+0000","last_undegraded":"2024-09-14T06:17:41.471334+0000","last_fullsized":"2024-09-14T06:17:41.471334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:01:35.004877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045017199999999999,"stat_sum":{"num_bytes":55074078,"num_objects":55,"num_object_clones":10,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":15,"num_read":2691,"num_read_kb":16643,"num_write":224,"num_write_kb":9847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"87'139","reported_seq":1388,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.623573+0000","last_change":"2024-09-14T06:16:22.195705+0000","last_active":"2024-09-14T06:17:41.623573+0000","last_peered":"2024-09-14T06:17:41.623573+0000","last_clean":"2024-09-14T06:17:41.623573+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:17:41.623573+0000","last_undegraded":"2024-09-14T06:17:41.623573+0000","last_fullsized":"2024-09-14T06:17:41.623573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:02:15.957880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025578329999999998,"stat_sum":{"num_bytes":46202051,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":7,"num_read":1256,"num_read_kb":29057,"num_write":210,"num_write_kb":11572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"87'623","reported_seq":2318,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.667110+0000","last_change":"2024-09-14T06:17:38.228193+0000","last_active":"2024-09-14T06:17:41.667110+0000","last_peered":"2024-09-14T06:17:41.667110+0000","last_clean":"2024-09-14T06:17:41.667110+0000","last_became_active":"2024-09-14T06:17:38.227106+0000","last_became_peered":"2024-09-14T06:17:38.227106+0000","last_unstale":"2024-09-14T06:17:41.667110+0000","last_undegraded":"2024-09-14T06:17:41.667110+0000","last_fullsized":"2024-09-14T06:17:41.667110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:44.413486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001627671,"stat_sum":{"num_bytes":37374219,"num_objects":56,"num_object_clones":13,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":16,"num_read":3735,"num_read_kb":26924,"num_write":955,"num_write_kb":20272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":29390214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"87'609","reported_seq":4638,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.988152+0000","last_change":"2024-09-14T06:16:33.547557+0000","last_active":"2024-09-14T06:17:41.988152+0000","last_peered":"2024-09-14T06:17:41.988152+0000","last_clean":"2024-09-14T06:17:41.988152+0000","last_became_active":"2024-09-14T06:16:33.546796+0000","last_became_peered":"2024-09-14T06:16:33.546796+0000","last_unstale":"2024-09-14T06:17:41.988152+0000","last_undegraded":"2024-09-14T06:17:41.988152+0000","last_fullsized":"2024-09-14T06:17:41.988152+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:13:41.806382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026669160000000001,"stat_sum":{"num_bytes":48267984,"num_objects":53,"num_object_clones":7,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":12,"num_read":9212,"num_read_kb":50798,"num_write":583,"num_write_kb":16104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"80'577","reported_seq":2538,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.642399+0000","last_change":"2024-09-14T06:16:22.195247+0000","last_active":"2024-09-14T06:17:41.642399+0000","last_peered":"2024-09-14T06:17:41.642399+0000","last_clean":"2024-09-14T06:17:41.642399+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:17:41.642399+0000","last_undegraded":"2024-09-14T06:17:41.642399+0000","last_fullsized":"2024-09-14T06:17:41.642399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:12:24.464514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022870590000000001,"stat_sum":{"num_bytes":34980582,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":14,"num_read":4106,"num_read_kb":15897,"num_write":707,"num_write_kb":12738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"87'277","reported_seq":2178,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.963636+0000","last_change":"2024-09-14T06:16:22.200094+0000","last_active":"2024-09-14T06:17:41.963636+0000","last_peered":"2024-09-14T06:17:41.963636+0000","last_clean":"2024-09-14T06:17:41.963636+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:17:41.963636+0000","last_undegraded":"2024-09-14T06:17:41.963636+0000","last_fullsized":"2024-09-14T06:17:41.963636+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:37:35.508264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011343239999999999,"stat_sum":{"num_bytes":53143462,"num_objects":57,"num_object_clones":8,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":14,"num_read":2240,"num_read_kb":47484,"num_write":277,"num_write_kb":14632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"87'458","reported_seq":2540,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.944980+0000","last_change":"2024-09-14T06:17:38.225212+0000","last_active":"2024-09-14T06:17:41.944980+0000","last_peered":"2024-09-14T06:17:41.944980+0000","last_clean":"2024-09-14T06:17:41.944980+0000","last_became_active":"2024-09-14T06:17:38.224251+0000","last_became_peered":"2024-09-14T06:17:38.224251+0000","last_unstale":"2024-09-14T06:17:41.944980+0000","last_undegraded":"2024-09-14T06:17:41.944980+0000","last_fullsized":"2024-09-14T06:17:41.944980+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:31:43.282194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226324,"stat_sum":{"num_bytes":19177105,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":15,"num_read":4118,"num_read_kb":14371,"num_write":540,"num_write_kb":7654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":12239008,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"80'423","reported_seq":2649,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.624128+0000","last_change":"2024-09-14T06:17:37.232951+0000","last_active":"2024-09-14T06:17:41.624128+0000","last_peered":"2024-09-14T06:17:41.624128+0000","last_clean":"2024-09-14T06:17:41.624128+0000","last_became_active":"2024-09-14T06:17:37.232145+0000","last_became_peered":"2024-09-14T06:17:37.232145+0000","last_unstale":"2024-09-14T06:17:41.624128+0000","last_undegraded":"2024-09-14T06:17:41.624128+0000","last_fullsized":"2024-09-14T06:17:41.624128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:51:31.847884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082016299999999995,"stat_sum":{"num_bytes":17887056,"num_objects":47,"num_object_clones":6,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":14,"num_read":3083,"num_read_kb":10830,"num_write":525,"num_write_kb":8437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":10176554,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"79'389","reported_seq":3592,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619560+0000","last_change":"2024-09-14T06:16:22.322160+0000","last_active":"2024-09-14T06:17:41.619560+0000","last_peered":"2024-09-14T06:17:41.619560+0000","last_clean":"2024-09-14T06:17:41.619560+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:17:41.619560+0000","last_undegraded":"2024-09-14T06:17:41.619560+0000","last_fullsized":"2024-09-14T06:17:41.619560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:32:32.561603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011502319999999999,"stat_sum":{"num_bytes":66038693,"num_objects":56,"num_object_clones":12,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":16,"num_read":5387,"num_read_kb":16288,"num_write":355,"num_write_kb":5570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1349381956,"num_objects":1565,"num_object_clones":268,"num_object_copies":3130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1565,"num_whiteouts":388,"num_read":99030,"num_read_kb":744147,"num_write":15426,"num_write_kb":428705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1124,"num_bytes_recovered":140497534,"num_keys_recovered":151,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64811008,"data_stored":623617065,"data_compressed":17110337,"data_compressed_allocated":42094592,"data_compressed_original":603994405,"omap_allocated":0,"internal_metadata":0},"log_size":12876,"ondisk_log_size":12876,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42076,"kb_used_data":10892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329764,"statfs":{"total":96636764160,"available":96593678336,"internally_reserved":0,"allocated":11153408,"data_stored":95084645,"data_compressed":3249521,"data_compressed_allocated":7127040,"data_compressed_original":91973016,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509463,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42276,"kb_used_data":11088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329564,"statfs":{"total":96636764160,"available":96593473536,"internally_reserved":0,"allocated":11354112,"data_stored":94050392,"data_compressed":2224067,"data_compressed_allocated":6352896,"data_compressed_original":90203129,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78948,"kb_used_data":7504,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94292892,"statfs":{"total":96636764160,"available":96555921408,"internally_reserved":0,"allocated":7684096,"data_stored":64377934,"data_compressed":1692911,"data_compressed_allocated":4415488,"data_compressed_original":61953833,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63728,"kb_used_data":10336,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94308112,"statfs":{"total":96636764160,"available":96571506688,"internally_reserved":0,"allocated":10584064,"data_stored":92432051,"data_compressed":2396485,"data_compressed_allocated":6148096,"data_compressed_original":88883168,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542167,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40860,"kb_used_data":5512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330980,"statfs":{"total":96636764160,"available":96594923520,"internally_reserved":0,"allocated":5644288,"data_stored":44707210,"data_compressed":1436486,"data_compressed_allocated":2777088,"data_compressed_original":42597756,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48820,"kb_used_data":9316,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323020,"statfs":{"total":96636764160,"available":96586772480,"internally_reserved":0,"allocated":9539584,"data_stored":79799958,"data_compressed":2108388,"data_compressed_allocated":5210112,"data_compressed_original":76518299,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542166,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49088,"kb_used_data":9584,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322752,"statfs":{"total":96636764160,"available":96586498048,"internally_reserved":0,"allocated":9814016,"data_stored":92098360,"data_compressed":2507870,"data_compressed_allocated":6246400,"data_compressed_original":89532206,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47040,"kb_used_data":7536,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324800,"statfs":{"total":96636764160,"available":96588595200,"internally_reserved":0,"allocated":7716864,"data_stored":67792480,"data_compressed":1649220,"data_compressed_allocated":4116480,"data_compressed_original":65121367,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9539584,"data_stored":91866454,"data_compressed":2396079,"data_compressed_allocated":6139904,"data_compressed_original":88797152,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6602752,"data_stored":63317807,"data_compressed":1682721,"data_compressed_allocated":4374528,"data_compressed_original":61380393,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10018816,"data_stored":92379871,"data_compressed":2107433,"data_compressed_allocated":6184960,"data_compressed_original":89124224,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10067968,"data_stored":94141771,"data_compressed":3232343,"data_compressed_allocated":7081984,"data_compressed_original":91508736,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6709248,"data_stored":67328555,"data_compressed":1649220,"data_compressed_allocated":4116480,"data_compressed_original":65121367,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8695808,"data_stored":90975490,"data_compressed":2497552,"data_compressed_allocated":6205440,"data_compressed_original":88925998,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8540160,"data_stored":79363832,"data_compressed":2108503,"data_compressed_allocated":5214208,"data_compressed_original":76538779,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4636672,"data_stored":44243285,"data_compressed":1436486,"data_compressed_allocated":2777088,"data_compressed_original":42597756,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:17:43.236 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T06:17:43.236 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:17:43.236 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T06:17:43.237 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T06:17:43.246 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:43.240+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:43.344 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:43.344 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:43.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:43.344 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:43.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.344 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:17:43.344 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:17:43.344 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 99.265864837999999, 2024-09-14T06:17:43.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.56009052699999995, 2024-09-14T06:17:43.345 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.345 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.345 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:43.345 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.345 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:17:43.345 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:17:43.345 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 1 2024-09-14T06:17:43.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.345 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.345 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.345 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:17:43.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:17:43.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077065+0000", 2024-09-14T06:17:43.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6429999999999999e-06 2024-09-14T06:17:43.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077079+0000", 2024-09-14T06:17:43.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4447000000000001e-05 2024-09-14T06:17:43.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077103+0000", 2024-09-14T06:17:43.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3856e-05 2024-09-14T06:17:43.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077120+0000", 2024-09-14T06:17:43.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7337e-05 2024-09-14T06:17:43.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077255+0000", 2024-09-14T06:17:43.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000134652 2024-09-14T06:17:43.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:43.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077274+0000", 2024-09-14T06:17:43.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9003000000000001e-05 2024-09-14T06:17:43.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634272+0000", 2024-09-14T06:17:43.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.55699813099999995 2024-09-14T06:17:43.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634591+0000", 2024-09-14T06:17:43.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031910800000000002 2024-09-14T06:17:43.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:17:43.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634892+0000", 2024-09-14T06:17:43.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00030066400000000002 2024-09-14T06:17:43.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:43.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636345+0000", 2024-09-14T06:17:43.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001452896 2024-09-14T06:17:43.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:43.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636974+0000", 2024-09-14T06:17:43.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00062927800000000004 2024-09-14T06:17:43.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:43.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637071+0000", 2024-09-14T06:17:43.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7178999999999999e-05 2024-09-14T06:17:43.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637153+0000", 2024-09-14T06:17:43.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1333000000000002e-05 2024-09-14T06:17:43.350 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.350 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.350 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.351 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38947 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 159744~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.351 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:17:43.351 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662806551000003, 2024-09-14T06:17:43.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085496275999999996, 2024-09-14T06:17:43.351 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.351 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.351 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.351 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.351 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.351 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.351 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38947 2024-09-14T06:17:43.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.351 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:17:43.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:17:43.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680121+0000", 2024-09-14T06:17:43.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.82e-07 2024-09-14T06:17:43.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680123+0000", 2024-09-14T06:17:43.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5629999999999999e-06 2024-09-14T06:17:43.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680133+0000", 2024-09-14T06:17:43.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0890000000000006e-06 2024-09-14T06:17:43.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680138+0000", 2024-09-14T06:17:43.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.728e-06 2024-09-14T06:17:43.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765422+0000", 2024-09-14T06:17:43.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085283701000000003 2024-09-14T06:17:43.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765496+0000", 2024-09-14T06:17:43.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4276000000000005e-05 2024-09-14T06:17:43.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765617+0000", 2024-09-14T06:17:43.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000120437 2024-09-14T06:17:43.354 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.354 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.354 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.355 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38990 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 163840~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.355 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:17:43.355 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662780568000002, 2024-09-14T06:17:43.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085695970999999996, 2024-09-14T06:17:43.355 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.355 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.355 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.355 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.355 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.355 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.355 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38990 2024-09-14T06:17:43.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.355 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.356 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:17:43.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.356 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:17:43.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.356 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680147+0000", 2024-09-14T06:17:43.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7599999999999997e-07 2024-09-14T06:17:43.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680149+0000", 2024-09-14T06:17:43.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4700000000000001e-06 2024-09-14T06:17:43.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680156+0000", 2024-09-14T06:17:43.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.058e-06 2024-09-14T06:17:43.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680162+0000", 2024-09-14T06:17:43.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7080000000000002e-06 2024-09-14T06:17:43.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765657+0000", 2024-09-14T06:17:43.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085494566999999994 2024-09-14T06:17:43.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765741+0000", 2024-09-14T06:17:43.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4782000000000002e-05 2024-09-14T06:17:43.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765842+0000", 2024-09-14T06:17:43.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010090999999999999 2024-09-14T06:17:43.359 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.359 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.359 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.359 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38991 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 172032~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.359 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:17:43.359 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662756610999999, 2024-09-14T06:17:43.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085875878000000003, 2024-09-14T06:17:43.359 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.359 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.359 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.359 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.359 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.360 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.360 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38991 2024-09-14T06:17:43.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.360 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:17:43.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:17:43.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680171+0000", 2024-09-14T06:17:43.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4299999999999998e-07 2024-09-14T06:17:43.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680173+0000", 2024-09-14T06:17:43.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1639999999999999e-06 2024-09-14T06:17:43.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680181+0000", 2024-09-14T06:17:43.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0539999999999995e-06 2024-09-14T06:17:43.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680187+0000", 2024-09-14T06:17:43.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7529999999999998e-06 2024-09-14T06:17:43.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765875+0000", 2024-09-14T06:17:43.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085688722999999994 2024-09-14T06:17:43.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765948+0000", 2024-09-14T06:17:43.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2126999999999996e-05 2024-09-14T06:17:43.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766046+0000", 2024-09-14T06:17:43.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8614000000000001e-05 2024-09-14T06:17:43.364 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.365 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.365 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.365 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38992 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 180224~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.365 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:43.365 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662731334999997, 2024-09-14T06:17:43.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086050225999999994, 2024-09-14T06:17:43.365 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.365 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.365 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.365 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.365 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.365 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38992 2024-09-14T06:17:43.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.366 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:43.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:43.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:43.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1199999999999998e-07 2024-09-14T06:17:43.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680199+0000", 2024-09-14T06:17:43.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:17:43.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680227+0000", 2024-09-14T06:17:43.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8238999999999998e-05 2024-09-14T06:17:43.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680234+0000", 2024-09-14T06:17:43.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0550000000000002e-06 2024-09-14T06:17:43.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766077+0000", 2024-09-14T06:17:43.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085843002000000002 2024-09-14T06:17:43.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766155+0000", 2024-09-14T06:17:43.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8112e-05 2024-09-14T06:17:43.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766246+0000", 2024-09-14T06:17:43.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0785999999999994e-05 2024-09-14T06:17:43.369 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.369 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.369 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.369 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38993 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 188416~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.369 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:17:43.369 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662639984000002, 2024-09-14T06:17:43.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086166131000000007, 2024-09-14T06:17:43.370 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.370 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.370 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.370 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.370 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.370 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.370 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38993 2024-09-14T06:17:43.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.370 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.370 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.370 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:17:43.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.371 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:17:43.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.371 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680288+0000", 2024-09-14T06:17:43.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7199999999999998e-07 2024-09-14T06:17:43.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.371 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680291+0000", 2024-09-14T06:17:43.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7280000000000001e-06 2024-09-14T06:17:43.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680301+0000", 2024-09-14T06:17:43.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3360000000000001e-06 2024-09-14T06:17:43.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680308+0000", 2024-09-14T06:17:43.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1389999999999998e-06 2024-09-14T06:17:43.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766280+0000", 2024-09-14T06:17:43.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085972111000000004 2024-09-14T06:17:43.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766357+0000", 2024-09-14T06:17:43.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7477000000000004e-05 2024-09-14T06:17:43.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766453+0000", 2024-09-14T06:17:43.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5668000000000006e-05 2024-09-14T06:17:43.373 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.373 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.373 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.374 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38994 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 196608~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.374 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:43.374 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662550177, 2024-09-14T06:17:43.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086290477000000004, 2024-09-14T06:17:43.374 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.374 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.374 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.374 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.374 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.374 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.374 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38994 2024-09-14T06:17:43.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.374 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:43.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:43.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:43.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3e-07 2024-09-14T06:17:43.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680381+0000", 2024-09-14T06:17:43.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.439e-06 2024-09-14T06:17:43.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680438+0000", 2024-09-14T06:17:43.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6693000000000001e-05 2024-09-14T06:17:43.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680446+0000", 2024-09-14T06:17:43.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9160000000000001e-06 2024-09-14T06:17:43.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766490+0000", 2024-09-14T06:17:43.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086044617000000004 2024-09-14T06:17:43.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766563+0000", 2024-09-14T06:17:43.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2430999999999998e-05 2024-09-14T06:17:43.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766667+0000", 2024-09-14T06:17:43.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000104651 2024-09-14T06:17:43.378 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.378 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.378 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.378 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38995 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 204800~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.378 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:17:43.378 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662467151000001, 2024-09-14T06:17:43.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086415360999999996, 2024-09-14T06:17:43.378 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.378 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.378 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.378 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.378 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.378 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.379 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38995 2024-09-14T06:17:43.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.379 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:17:43.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:17:43.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680461+0000", 2024-09-14T06:17:43.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:17:43.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680464+0000", 2024-09-14T06:17:43.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4929999999999998e-06 2024-09-14T06:17:43.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680479+0000", 2024-09-14T06:17:43.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5322999999999999e-05 2024-09-14T06:17:43.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680496+0000", 2024-09-14T06:17:43.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6801000000000001e-05 2024-09-14T06:17:43.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766699+0000", 2024-09-14T06:17:43.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086203265000000001 2024-09-14T06:17:43.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766777+0000", 2024-09-14T06:17:43.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7284000000000001e-05 2024-09-14T06:17:43.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.382 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766875+0000", 2024-09-14T06:17:43.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8461000000000005e-05 2024-09-14T06:17:43.382 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.382 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.382 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.382 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38996 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 212992~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.382 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:17:43.382 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662419712000002, 2024-09-14T06:17:43.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086576474, 2024-09-14T06:17:43.383 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.383 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.383 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.383 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.383 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.383 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.383 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38996 2024-09-14T06:17:43.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.383 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:17:43.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:17:43.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680508+0000", 2024-09-14T06:17:43.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1799999999999995e-07 2024-09-14T06:17:43.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680511+0000", 2024-09-14T06:17:43.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9739999999999998e-06 2024-09-14T06:17:43.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680520+0000", 2024-09-14T06:17:43.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.329e-06 2024-09-14T06:17:43.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680528+0000", 2024-09-14T06:17:43.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4660000000000002e-06 2024-09-14T06:17:43.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.386 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766907+0000", 2024-09-14T06:17:43.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086379529999999996 2024-09-14T06:17:43.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.386 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.386 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766987+0000", 2024-09-14T06:17:43.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0280999999999999e-05 2024-09-14T06:17:43.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.386 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.386 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767084+0000", 2024-09-14T06:17:43.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6275999999999998e-05 2024-09-14T06:17:43.386 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.386 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.387 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.387 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38997 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 221184~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.387 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:43.387 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662390371000001, 2024-09-14T06:17:43.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086757327999999995, 2024-09-14T06:17:43.387 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.387 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.387 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.387 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.387 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.387 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.387 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38997 2024-09-14T06:17:43.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.388 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:43.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:43.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:43.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:17:43.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680540+0000", 2024-09-14T06:17:43.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6910000000000002e-06 2024-09-14T06:17:43.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680548+0000", 2024-09-14T06:17:43.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7339999999999997e-06 2024-09-14T06:17:43.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680555+0000", 2024-09-14T06:17:43.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0900000000000001e-06 2024-09-14T06:17:43.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767116+0000", 2024-09-14T06:17:43.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086561551 2024-09-14T06:17:43.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767192+0000", 2024-09-14T06:17:43.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5504000000000001e-05 2024-09-14T06:17:43.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.391 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.391 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767294+0000", 2024-09-14T06:17:43.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000102308 2024-09-14T06:17:43.391 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.391 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.391 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.391 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38998 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 229376~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.391 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:17:43.391 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662363759999998, 2024-09-14T06:17:43.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087010193999999999, 2024-09-14T06:17:43.391 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.391 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.392 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.392 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.392 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.392 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.392 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38998 2024-09-14T06:17:43.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.392 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:17:43.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:17:43.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680564+0000", 2024-09-14T06:17:43.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1800000000000001e-07 2024-09-14T06:17:43.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680566+0000", 2024-09-14T06:17:43.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.503e-06 2024-09-14T06:17:43.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680574+0000", 2024-09-14T06:17:43.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3979999999999999e-06 2024-09-14T06:17:43.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680580+0000", 2024-09-14T06:17:43.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4859999999999997e-06 2024-09-14T06:17:43.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767359+0000", 2024-09-14T06:17:43.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086779447999999995 2024-09-14T06:17:43.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767445+0000", 2024-09-14T06:17:43.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6007000000000005e-05 2024-09-14T06:17:43.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767573+0000", 2024-09-14T06:17:43.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012793400000000001 2024-09-14T06:17:43.395 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.395 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.395 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.395 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38999 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 237568~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.395 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:17:43.395 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662334895999997, 2024-09-14T06:17:43.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087211699000000004, 2024-09-14T06:17:43.396 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.396 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.396 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.396 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.396 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.396 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.396 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38999 2024-09-14T06:17:43.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.396 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.396 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.396 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:17:43.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:17:43.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680593+0000", 2024-09-14T06:17:43.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7700000000000005e-07 2024-09-14T06:17:43.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680600+0000", 2024-09-14T06:17:43.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7130000000000005e-06 2024-09-14T06:17:43.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680610+0000", 2024-09-14T06:17:43.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.55e-06 2024-09-14T06:17:43.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680616+0000", 2024-09-14T06:17:43.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3069999999999999e-06 2024-09-14T06:17:43.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767607+0000", 2024-09-14T06:17:43.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086990703000000003 2024-09-14T06:17:43.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767682+0000", 2024-09-14T06:17:43.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4944999999999995e-05 2024-09-14T06:17:43.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767804+0000", 2024-09-14T06:17:43.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012200400000000001 2024-09-14T06:17:43.399 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.399 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.399 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.399 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39000 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 245760~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.399 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:43.400 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662301274000001, 2024-09-14T06:17:43.400 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087408649000000005, 2024-09-14T06:17:43.400 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.400 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.400 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.400 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.400 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.400 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.400 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39000 2024-09-14T06:17:43.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.400 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.400 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.400 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:43.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.401 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.401 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:43.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.401 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.401 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:43.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1699999999999998e-07 2024-09-14T06:17:43.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.401 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.401 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680629+0000", 2024-09-14T06:17:43.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5469999999999999e-06 2024-09-14T06:17:43.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680636+0000", 2024-09-14T06:17:43.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.973e-06 2024-09-14T06:17:43.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680642+0000", 2024-09-14T06:17:43.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0349999999999998e-06 2024-09-14T06:17:43.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767839+0000", 2024-09-14T06:17:43.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087197067000000003 2024-09-14T06:17:43.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767913+0000", 2024-09-14T06:17:43.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3832999999999996e-05 2024-09-14T06:17:43.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768034+0000", 2024-09-14T06:17:43.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012167699999999999 2024-09-14T06:17:43.403 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.403 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.403 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.404 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39001 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 253952~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.404 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:17:43.404 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662276831, 2024-09-14T06:17:43.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087623312999999994, 2024-09-14T06:17:43.404 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.404 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.404 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.404 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.404 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.404 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.404 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39001 2024-09-14T06:17:43.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.405 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:17:43.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:17:43.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680651+0000", 2024-09-14T06:17:43.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7700000000000004e-07 2024-09-14T06:17:43.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680653+0000", 2024-09-14T06:17:43.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5739999999999998e-06 2024-09-14T06:17:43.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680702+0000", 2024-09-14T06:17:43.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8476000000000001e-05 2024-09-14T06:17:43.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680709+0000", 2024-09-14T06:17:43.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8170000000000001e-06 2024-09-14T06:17:43.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768066+0000", 2024-09-14T06:17:43.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087357852999999999 2024-09-14T06:17:43.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768163+0000", 2024-09-14T06:17:43.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6869999999999999e-05 2024-09-14T06:17:43.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768273+0000", 2024-09-14T06:17:43.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000110146 2024-09-14T06:17:43.408 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.408 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.408 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.408 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39002 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 262144~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.408 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:17:43.408 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662196725000001, 2024-09-14T06:17:43.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088150478000000004, 2024-09-14T06:17:43.408 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.408 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.408 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.408 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.408 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.408 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.408 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39002 2024-09-14T06:17:43.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.409 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:17:43.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:17:43.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680739+0000", 2024-09-14T06:17:43.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3979999999999999e-06 2024-09-14T06:17:43.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680742+0000", 2024-09-14T06:17:43.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8349999999999997e-06 2024-09-14T06:17:43.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680751+0000", 2024-09-14T06:17:43.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8599999999999999e-06 2024-09-14T06:17:43.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680758+0000", 2024-09-14T06:17:43.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4790000000000001e-06 2024-09-14T06:17:43.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768307+0000", 2024-09-14T06:17:43.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087549342000000002 2024-09-14T06:17:43.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768392+0000", 2024-09-14T06:17:43.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.441e-05 2024-09-14T06:17:43.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768881+0000", 2024-09-14T06:17:43.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000489154 2024-09-14T06:17:43.412 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.412 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.412 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.412 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39003 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 270336~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.412 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:43.412 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662144435000002, 2024-09-14T06:17:43.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088375555999999994, 2024-09-14T06:17:43.412 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.413 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.413 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.413 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39003 2024-09-14T06:17:43.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.413 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.413 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.413 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:43.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:43.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:43.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7000000000000004e-07 2024-09-14T06:17:43.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680786+0000", 2024-09-14T06:17:43.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:17:43.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680794+0000", 2024-09-14T06:17:43.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9840000000000001e-06 2024-09-14T06:17:43.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680800+0000", 2024-09-14T06:17:43.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1550000000000004e-06 2024-09-14T06:17:43.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768965+0000", 2024-09-14T06:17:43.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088165091000000001 2024-09-14T06:17:43.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769059+0000", 2024-09-14T06:17:43.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3923000000000002e-05 2024-09-14T06:17:43.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769158+0000", 2024-09-14T06:17:43.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8820000000000006e-05 2024-09-14T06:17:43.416 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.416 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.416 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.417 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39004 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 278528~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.417 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:17:43.417 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662116746999999, 2024-09-14T06:17:43.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088499624999999998, 2024-09-14T06:17:43.417 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.417 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.417 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.417 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.417 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.417 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.417 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39004 2024-09-14T06:17:43.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.418 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:17:43.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:17:43.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680811+0000", 2024-09-14T06:17:43.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:17:43.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680814+0000", 2024-09-14T06:17:43.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:17:43.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680821+0000", 2024-09-14T06:17:43.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9009999999999997e-06 2024-09-14T06:17:43.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680826+0000", 2024-09-14T06:17:43.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5350000000000004e-06 2024-09-14T06:17:43.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769189+0000", 2024-09-14T06:17:43.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088363024999999998 2024-09-14T06:17:43.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769246+0000", 2024-09-14T06:17:43.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7017e-05 2024-09-14T06:17:43.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769310+0000", 2024-09-14T06:17:43.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3541999999999996e-05 2024-09-14T06:17:43.421 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.421 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.421 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.421 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39005 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 286720~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.421 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:17:43.421 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662092663000003, 2024-09-14T06:17:43.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088681092000000003, 2024-09-14T06:17:43.421 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.421 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.421 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.422 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.422 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.422 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.422 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39005 2024-09-14T06:17:43.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.422 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:17:43.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:17:43.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680835+0000", 2024-09-14T06:17:43.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2699999999999999e-07 2024-09-14T06:17:43.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680837+0000", 2024-09-14T06:17:43.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.413e-06 2024-09-14T06:17:43.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680844+0000", 2024-09-14T06:17:43.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8820000000000003e-06 2024-09-14T06:17:43.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680850+0000", 2024-09-14T06:17:43.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5169999999999999e-06 2024-09-14T06:17:43.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769337+0000", 2024-09-14T06:17:43.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088487471999999998 2024-09-14T06:17:43.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.425 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769404+0000", 2024-09-14T06:17:43.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7111000000000002e-05 2024-09-14T06:17:43.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.425 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769515+0000", 2024-09-14T06:17:43.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011116999999999999 2024-09-14T06:17:43.425 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.425 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.425 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.425 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39006 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 294912~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.426 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:17:43.426 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662069500000001, 2024-09-14T06:17:43.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088886572999999997, 2024-09-14T06:17:43.426 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.426 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.426 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.426 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.426 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.426 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.426 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39006 2024-09-14T06:17:43.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.426 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:17:43.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:17:43.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680858+0000", 2024-09-14T06:17:43.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7e-07 2024-09-14T06:17:43.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680860+0000", 2024-09-14T06:17:43.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:17:43.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680867+0000", 2024-09-14T06:17:43.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0430000000000002e-06 2024-09-14T06:17:43.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680873+0000", 2024-09-14T06:17:43.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7389999999999996e-06 2024-09-14T06:17:43.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769547+0000", 2024-09-14T06:17:43.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088673830999999995 2024-09-14T06:17:43.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769631+0000", 2024-09-14T06:17:43.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3660999999999997e-05 2024-09-14T06:17:43.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769744+0000", 2024-09-14T06:17:43.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000113482 2024-09-14T06:17:43.429 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.430 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.430 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.430 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.430 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39007 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 303104~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:43.430 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:17:43.430 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 33.662046087999997, 2024-09-14T06:17:43.430 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.089077397000000003, 2024-09-14T06:17:43.430 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:43.430 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:43.430 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:43.430 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:43.430 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:43.431 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:43.431 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39007 2024-09-14T06:17:43.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.431 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:43.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.431 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:43.431 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:17:43.431 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.431 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:43.431 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:17:43.431 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:43.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:43.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687953+0000", 2024-09-14T06:17:43.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00707203 2024-09-14T06:17:43.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:43.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687960+0000", 2024-09-14T06:17:43.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9890000000000003e-06 2024-09-14T06:17:43.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:43.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687974+0000", 2024-09-14T06:17:43.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4226999999999999e-05 2024-09-14T06:17:43.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687983+0000", 2024-09-14T06:17:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9800000000000004e-06 2024-09-14T06:17:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769775+0000", 2024-09-14T06:17:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.081792089999999998 2024-09-14T06:17:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769849+0000", 2024-09-14T06:17:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3925999999999997e-05 2024-09-14T06:17:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769958+0000", 2024-09-14T06:17:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010915499999999999 2024-09-14T06:17:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.434 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:43.435 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:17:43.436 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:43.432+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:43.536 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:43.533+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:43.553 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:43.553 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:43.553 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:43.553 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:43.568 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:17:43.586 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:43.586 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:43.586 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 88, 2024-09-14T06:17:43.586 INFO:teuthology.orchestra.run.smithi039.stdout: "fsid": "038563b1-268f-461f-b6a6-4b4b41ad0236", 2024-09-14T06:17:43.586 INFO:teuthology.orchestra.run.smithi039.stdout: "created": "2024-09-14T06:15:49.493035+0000", 2024-09-14T06:17:43.586 INFO:teuthology.orchestra.run.smithi039.stdout: "modified": "2024-09-14T06:17:42.622710+0000", 2024-09-14T06:17:43.586 INFO:teuthology.orchestra.run.smithi039.stdout: "last_up_change": "2024-09-14T06:16:02.014989+0000", 2024-09-14T06:17:43.586 INFO:teuthology.orchestra.run.smithi039.stdout: "last_in_change": "2024-09-14T06:16:26.221200+0000", 2024-09-14T06:17:43.587 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T06:17:43.587 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_num": 5799936, 2024-09-14T06:17:43.587 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_set": [ 2024-09-14T06:17:43.587 INFO:teuthology.orchestra.run.smithi039.stdout: "pglog_hardlimit", 2024-09-14T06:17:43.587 INFO:teuthology.orchestra.run.smithi039.stdout: "purged_snapdirs", 2024-09-14T06:17:43.587 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_deletes", 2024-09-14T06:17:43.587 INFO:teuthology.orchestra.run.smithi039.stdout: "sortbitwise" 2024-09-14T06:17:43.587 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:17:43.587 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_version": 5, 2024-09-14T06:17:43.587 INFO:teuthology.orchestra.run.smithi039.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T06:17:43.587 INFO:teuthology.orchestra.run.smithi039.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T06:17:43.587 INFO:teuthology.orchestra.run.smithi039.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T06:17:43.587 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_snapshot": "", 2024-09-14T06:17:43.587 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_max": 3, 2024-09-14T06:17:43.588 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd": 8, 2024-09-14T06:17:43.588 INFO:teuthology.orchestra.run.smithi039.stdout: "require_min_compat_client": "luminous", 2024-09-14T06:17:43.588 INFO:teuthology.orchestra.run.smithi039.stdout: "min_compat_client": "luminous", 2024-09-14T06:17:43.588 INFO:teuthology.orchestra.run.smithi039.stdout: "require_osd_release": "squid", 2024-09-14T06:17:43.588 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_crimson": false, 2024-09-14T06:17:43.588 INFO:teuthology.orchestra.run.smithi039.stdout: "pools": [ 2024-09-14T06:17:43.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.588 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 1, 2024-09-14T06:17:43.588 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": ".mgr", 2024-09-14T06:17:43.588 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-09-14T06:16:02.429225+0000", 2024-09-14T06:17:43.588 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-14T06:17:43.588 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool", 2024-09-14T06:17:43.588 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-09-14T06:17:43.588 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-09-14T06:17:43.589 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-09-14T06:17:43.589 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-09-14T06:17:43.589 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:17:43.589 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:17:43.589 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:17:43.589 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:17:43.589 INFO:teuthology.orchestra.run.smithi039.stdout: "is_stretch_pool": false, 2024-09-14T06:17:43.589 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-09-14T06:17:43.589 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:17:43.589 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 1, 2024-09-14T06:17:43.589 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 1, 2024-09-14T06:17:43.589 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 1, 2024-09-14T06:17:43.589 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 1, 2024-09-14T06:17:43.590 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 1, 2024-09-14T06:17:43.590 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-09-14T06:17:43.590 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-09-14T06:17:43.590 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-09-14T06:17:43.590 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-09-14T06:17:43.590 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-09-14T06:17:43.590 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-09-14T06:17:43.590 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-09-14T06:17:43.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.590 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "18", 2024-09-14T06:17:43.590 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-09-14T06:17:43.590 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:17:43.590 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:17:43.590 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-09-14T06:17:43.590 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:17:43.591 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 0, 2024-09-14T06:17:43.591 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 0, 2024-09-14T06:17:43.591 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-09-14T06:17:43.591 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-09-14T06:17:43.591 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-09-14T06:17:43.591 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-09-14T06:17:43.591 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-09-14T06:17:43.591 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-09-14T06:17:43.591 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-09-14T06:17:43.591 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-09-14T06:17:43.591 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-09-14T06:17:43.591 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-09-14T06:17:43.591 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-09-14T06:17:43.592 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:17:43.592 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:17:43.592 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:17:43.592 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-09-14T06:17:43.592 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-09-14T06:17:43.592 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-09-14T06:17:43.592 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-09-14T06:17:43.592 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-09-14T06:17:43.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.592 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-09-14T06:17:43.592 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-09-14T06:17:43.592 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-09-14T06:17:43.592 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:17:43.592 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:17:43.593 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:17:43.593 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:17:43.593 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-09-14T06:17:43.593 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-09-14T06:17:43.593 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-09-14T06:17:43.593 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-09-14T06:17:43.593 INFO:teuthology.orchestra.run.smithi039.stdout: "options": { 2024-09-14T06:17:43.593 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_max": 32, 2024-09-14T06:17:43.593 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_min": 1 2024-09-14T06:17:43.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.593 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-09-14T06:17:43.593 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": {} 2024-09-14T06:17:43.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.593 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-09-14T06:17:43.594 INFO:teuthology.orchestra.run.smithi039.stdout: "score_type": "Fair distribution", 2024-09-14T06:17:43.594 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 6.9000000953674316, 2024-09-14T06:17:43.594 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 6.9000000953674316, 2024-09-14T06:17:43.594 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T06:17:43.594 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 2, 2024-09-14T06:17:43.594 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 2, 2024-09-14T06:17:43.594 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:17:43.594 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 1, 2024-09-14T06:17:43.594 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:17:43.594 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.594 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-14T06:17:43.594 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": "rbd", 2024-09-14T06:17:43.595 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-09-14T06:16:08.129454+0000", 2024-09-14T06:17:43.595 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 8193, 2024-09-14T06:17:43.595 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:17:43.595 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-09-14T06:17:43.595 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-09-14T06:17:43.595 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-09-14T06:17:43.595 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-09-14T06:17:43.595 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:17:43.595 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:17:43.595 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:17:43.595 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:17:43.595 INFO:teuthology.orchestra.run.smithi039.stdout: "is_stretch_pool": false, 2024-09-14T06:17:43.595 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-09-14T06:17:43.595 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:17:43.595 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 8, 2024-09-14T06:17:43.596 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 8, 2024-09-14T06:17:43.596 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 8, 2024-09-14T06:17:43.596 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 8, 2024-09-14T06:17:43.596 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 8, 2024-09-14T06:17:43.596 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-09-14T06:17:43.596 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-09-14T06:17:43.596 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-09-14T06:17:43.596 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-09-14T06:17:43.596 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-09-14T06:17:43.596 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-09-14T06:17:43.596 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-09-14T06:17:43.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.596 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "22", 2024-09-14T06:17:43.596 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-09-14T06:17:43.597 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:17:43.597 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:17:43.597 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-09-14T06:17:43.597 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:17:43.597 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 2, 2024-09-14T06:17:43.597 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 22, 2024-09-14T06:17:43.597 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-09-14T06:17:43.597 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-09-14T06:17:43.597 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-09-14T06:17:43.597 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-09-14T06:17:43.597 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-09-14T06:17:43.597 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-09-14T06:17:43.597 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-09-14T06:17:43.597 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-09-14T06:17:43.597 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-09-14T06:17:43.598 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-09-14T06:17:43.598 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-09-14T06:17:43.598 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:17:43.598 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:17:43.598 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:17:43.598 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-09-14T06:17:43.598 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-09-14T06:17:43.598 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-09-14T06:17:43.598 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-09-14T06:17:43.598 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-09-14T06:17:43.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.598 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-09-14T06:17:43.598 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-09-14T06:17:43.598 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-09-14T06:17:43.598 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:17:43.599 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:17:43.599 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:17:43.599 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:17:43.599 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-09-14T06:17:43.599 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-09-14T06:17:43.599 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-09-14T06:17:43.599 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-09-14T06:17:43.599 INFO:teuthology.orchestra.run.smithi039.stdout: "options": {}, 2024-09-14T06:17:43.599 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-09-14T06:17:43.599 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd": {} 2024-09-14T06:17:43.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.599 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-09-14T06:17:43.599 INFO:teuthology.orchestra.run.smithi039.stdout: "score_type": "Fair distribution", 2024-09-14T06:17:43.599 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 4.380000114440918, 2024-09-14T06:17:43.599 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 4.380000114440918, 2024-09-14T06:17:43.599 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 1, 2024-09-14T06:17:43.600 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 4.380000114440918, 2024-09-14T06:17:43.600 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 4.380000114440918, 2024-09-14T06:17:43.600 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:17:43.600 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 1, 2024-09-14T06:17:43.600 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:17:43.600 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.600 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.600 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 3, 2024-09-14T06:17:43.600 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": "pool_client.0", 2024-09-14T06:17:43.600 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-09-14T06:16:18.928556+0000", 2024-09-14T06:17:43.600 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 8193, 2024-09-14T06:17:43.600 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:17:43.600 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-09-14T06:17:43.601 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-09-14T06:17:43.601 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-09-14T06:17:43.601 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-09-14T06:17:43.601 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:17:43.601 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:17:43.601 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:17:43.601 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:17:43.601 INFO:teuthology.orchestra.run.smithi039.stdout: "is_stretch_pool": false, 2024-09-14T06:17:43.601 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-09-14T06:17:43.601 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:17:43.601 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 32, 2024-09-14T06:17:43.601 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 32, 2024-09-14T06:17:43.601 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 32, 2024-09-14T06:17:43.601 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 32, 2024-09-14T06:17:43.601 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 32, 2024-09-14T06:17:43.602 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-09-14T06:17:43.602 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-09-14T06:17:43.602 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-09-14T06:17:43.602 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-09-14T06:17:43.602 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-09-14T06:17:43.602 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-09-14T06:17:43.602 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-09-14T06:17:43.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.602 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "88", 2024-09-14T06:17:43.602 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-09-14T06:17:43.602 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:17:43.602 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:17:43.603 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-09-14T06:17:43.603 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:17:43.603 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 33, 2024-09-14T06:17:43.603 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 88, 2024-09-14T06:17:43.603 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-09-14T06:17:43.603 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-09-14T06:17:43.603 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-09-14T06:17:43.603 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-09-14T06:17:43.603 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-09-14T06:17:43.603 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-09-14T06:17:43.603 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-09-14T06:17:43.603 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-09-14T06:17:43.603 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-09-14T06:17:43.603 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-09-14T06:17:43.603 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-09-14T06:17:43.604 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:17:43.604 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:17:43.604 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:17:43.604 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-09-14T06:17:43.604 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-09-14T06:17:43.604 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-09-14T06:17:43.604 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-09-14T06:17:43.604 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-09-14T06:17:43.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.604 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-09-14T06:17:43.604 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-09-14T06:17:43.604 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-09-14T06:17:43.604 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:17:43.604 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:17:43.605 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:17:43.605 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:17:43.605 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-09-14T06:17:43.605 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-09-14T06:17:43.605 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-09-14T06:17:43.605 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-09-14T06:17:43.605 INFO:teuthology.orchestra.run.smithi039.stdout: "options": {}, 2024-09-14T06:17:43.605 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-09-14T06:17:43.605 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd": {} 2024-09-14T06:17:43.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.605 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-09-14T06:17:43.605 INFO:teuthology.orchestra.run.smithi039.stdout: "score_type": "Fair distribution", 2024-09-14T06:17:43.605 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 1.75, 2024-09-14T06:17:43.605 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 1.75, 2024-09-14T06:17:43.606 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 1.1399999856948853, 2024-09-14T06:17:43.606 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 2, 2024-09-14T06:17:43.606 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 2, 2024-09-14T06:17:43.606 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:17:43.606 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 1, 2024-09-14T06:17:43.606 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:17:43.606 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.606 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.606 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:17:43.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osds": [ 2024-09-14T06:17:43.606 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 0, 2024-09-14T06:17:43.606 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "c669ea76-ed19-4c12-9e56-d79d62846f95", 2024-09-14T06:17:43.606 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:17:43.607 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:17:43.607 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:17:43.607 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:17:43.607 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:17:43.607 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:17:43.607 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:17:43.607 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 84, 2024-09-14T06:17:43.607 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:17:43.607 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:17:43.607 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:17:43.607 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.607 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.607 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.607 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6808", 2024-09-14T06:17:43.608 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:17:43.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.608 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.608 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.608 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6809", 2024-09-14T06:17:43.608 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:17:43.608 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.608 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.608 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:17:43.608 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.608 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.608 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.609 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6810", 2024-09-14T06:17:43.609 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:17:43.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.609 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.609 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6811", 2024-09-14T06:17:43.609 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:17:43.609 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.609 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.609 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:17:43.609 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.609 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.609 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6814", 2024-09-14T06:17:43.609 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:17:43.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.610 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.610 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6815", 2024-09-14T06:17:43.610 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:17:43.610 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.610 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.610 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:17:43.610 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.610 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.610 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6812", 2024-09-14T06:17:43.610 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:17:43.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.611 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.611 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.611 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6813", 2024-09-14T06:17:43.611 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:17:43.611 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.611 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.611 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6809/1157559201", 2024-09-14T06:17:43.611 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6811/1157559201", 2024-09-14T06:17:43.611 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6815/1157559201", 2024-09-14T06:17:43.611 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6813/1157559201", 2024-09-14T06:17:43.612 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:17:43.612 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:17:43.612 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:17:43.612 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.612 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.612 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 1, 2024-09-14T06:17:43.612 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b", 2024-09-14T06:17:43.612 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:17:43.612 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:17:43.612 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:17:43.612 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:17:43.612 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:17:43.612 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:17:43.613 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:17:43.613 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 84, 2024-09-14T06:17:43.613 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:17:43.613 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:17:43.613 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:17:43.613 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.613 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.613 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.613 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6816", 2024-09-14T06:17:43.613 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:17:43.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.613 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.613 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.613 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6817", 2024-09-14T06:17:43.613 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:17:43.614 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.614 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.614 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:17:43.614 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.614 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.614 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.614 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6818", 2024-09-14T06:17:43.614 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:17:43.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.614 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.614 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.614 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6819", 2024-09-14T06:17:43.614 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:17:43.614 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.614 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.615 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:17:43.615 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.615 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.615 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6822", 2024-09-14T06:17:43.615 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:17:43.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.615 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.615 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6823", 2024-09-14T06:17:43.615 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:17:43.616 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.616 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.616 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:17:43.616 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.616 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.616 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6820", 2024-09-14T06:17:43.616 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:17:43.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.616 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.616 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6821", 2024-09-14T06:17:43.616 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:17:43.617 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.617 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.617 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6817/4062089212", 2024-09-14T06:17:43.617 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6819/4062089212", 2024-09-14T06:17:43.617 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6823/4062089212", 2024-09-14T06:17:43.617 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6821/4062089212", 2024-09-14T06:17:43.617 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:17:43.617 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:17:43.617 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:17:43.617 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.617 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 2, 2024-09-14T06:17:43.617 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "9bda988f-e615-4545-849b-318c408c4c42", 2024-09-14T06:17:43.618 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:17:43.618 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:17:43.618 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:17:43.618 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 0, 2024-09-14T06:17:43.618 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:17:43.618 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:17:43.618 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:17:43.618 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 84, 2024-09-14T06:17:43.618 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:17:43.618 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:17:43.618 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:17:43.618 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.618 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.618 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.618 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6824", 2024-09-14T06:17:43.618 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:17:43.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.619 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.619 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.619 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6825", 2024-09-14T06:17:43.619 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:17:43.619 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.619 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.619 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:17:43.619 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.619 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.619 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.619 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6826", 2024-09-14T06:17:43.619 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:17:43.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.620 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.620 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6827", 2024-09-14T06:17:43.620 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:17:43.620 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.620 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.620 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:17:43.620 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.620 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.620 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6830", 2024-09-14T06:17:43.620 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:17:43.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.621 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.621 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6831", 2024-09-14T06:17:43.621 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:17:43.621 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.621 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.621 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:17:43.621 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.621 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.621 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6828", 2024-09-14T06:17:43.621 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:17:43.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.622 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.622 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6829", 2024-09-14T06:17:43.622 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:17:43.622 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.622 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.622 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6825/1724773882", 2024-09-14T06:17:43.622 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6827/1724773882", 2024-09-14T06:17:43.622 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6831/1724773882", 2024-09-14T06:17:43.622 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6829/1724773882", 2024-09-14T06:17:43.622 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:17:43.622 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:17:43.623 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:17:43.623 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.623 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.623 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 3, 2024-09-14T06:17:43.623 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "8ea6cd69-2ef1-42ea-b339-7394bf8f4a87", 2024-09-14T06:17:43.623 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:17:43.623 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:17:43.623 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:17:43.623 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:17:43.623 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:17:43.623 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:17:43.623 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:17:43.623 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 85, 2024-09-14T06:17:43.623 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:17:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:17:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:17:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6800", 2024-09-14T06:17:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:17:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6801", 2024-09-14T06:17:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:17:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.625 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:17:43.625 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.625 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.625 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.625 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6802", 2024-09-14T06:17:43.625 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:17:43.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.625 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.625 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.625 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6803", 2024-09-14T06:17:43.625 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:17:43.625 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.625 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.625 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:17:43.626 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.626 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.626 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.626 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6806", 2024-09-14T06:17:43.626 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:17:43.626 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.626 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.626 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.626 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6807", 2024-09-14T06:17:43.626 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:17:43.626 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.626 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.626 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.626 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:17:43.626 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.627 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.627 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.627 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6804", 2024-09-14T06:17:43.627 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:17:43.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.627 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.627 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.627 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6805", 2024-09-14T06:17:43.627 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:17:43.627 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.627 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.627 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6801/3775031293", 2024-09-14T06:17:43.627 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6803/3775031293", 2024-09-14T06:17:43.627 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6807/3775031293", 2024-09-14T06:17:43.628 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6805/3775031293", 2024-09-14T06:17:43.628 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:17:43.628 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:17:43.628 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:17:43.628 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.628 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.628 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 4, 2024-09-14T06:17:43.628 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "fea45e8a-e8ea-4963-b93f-0651b84c9e74", 2024-09-14T06:17:43.628 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:17:43.628 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:17:43.628 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:17:43.628 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:17:43.628 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:17:43.628 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:17:43.629 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:17:43.629 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 68, 2024-09-14T06:17:43.629 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:17:43.629 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:17:43.629 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:17:43.629 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.629 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.629 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.629 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6824", 2024-09-14T06:17:43.629 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:17:43.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.629 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.629 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.629 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6825", 2024-09-14T06:17:43.629 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:17:43.630 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.630 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.630 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:17:43.630 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.630 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.630 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.630 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6826", 2024-09-14T06:17:43.630 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:17:43.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.630 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.630 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.630 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6827", 2024-09-14T06:17:43.630 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:17:43.631 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.631 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.631 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:17:43.631 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.631 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.631 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.631 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6830", 2024-09-14T06:17:43.631 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:17:43.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.631 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.631 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.631 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6831", 2024-09-14T06:17:43.631 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:17:43.632 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.632 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.632 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:17:43.632 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.632 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.632 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6828", 2024-09-14T06:17:43.632 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:17:43.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.632 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.632 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6829", 2024-09-14T06:17:43.632 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:17:43.632 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.633 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.633 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6825/2632317380", 2024-09-14T06:17:43.633 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6827/2632317380", 2024-09-14T06:17:43.633 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6831/2632317380", 2024-09-14T06:17:43.633 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6829/2632317380", 2024-09-14T06:17:43.633 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:17:43.633 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:17:43.633 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:17:43.633 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.633 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 5, 2024-09-14T06:17:43.633 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "b601e852-a180-477f-86fc-6a213fa866f0", 2024-09-14T06:17:43.634 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:17:43.634 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:17:43.634 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:17:43.634 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:17:43.634 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:17:43.634 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:17:43.634 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:17:43.634 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 84, 2024-09-14T06:17:43.634 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:17:43.634 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:17:43.634 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:17:43.634 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.634 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.634 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.634 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6816", 2024-09-14T06:17:43.635 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:17:43.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.635 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.635 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.635 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6817", 2024-09-14T06:17:43.635 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:17:43.635 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.635 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.635 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:17:43.635 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.635 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.635 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.635 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6818", 2024-09-14T06:17:43.635 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:17:43.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.636 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.636 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.636 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6819", 2024-09-14T06:17:43.636 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:17:43.636 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.636 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.636 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:17:43.636 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.636 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.636 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.636 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6822", 2024-09-14T06:17:43.637 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:17:43.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.637 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.637 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6823", 2024-09-14T06:17:43.637 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:17:43.637 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.637 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.637 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:17:43.637 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.637 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.638 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6820", 2024-09-14T06:17:43.638 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:17:43.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.638 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.638 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.638 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6821", 2024-09-14T06:17:43.638 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:17:43.638 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.638 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.638 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6817/3501664149", 2024-09-14T06:17:43.638 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6819/3501664149", 2024-09-14T06:17:43.638 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6823/3501664149", 2024-09-14T06:17:43.638 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6821/3501664149", 2024-09-14T06:17:43.639 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:17:43.639 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:17:43.639 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:17:43.639 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.639 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.639 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 6, 2024-09-14T06:17:43.639 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "1113afe0-f524-41b1-8e51-eb5164602d38", 2024-09-14T06:17:43.639 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:17:43.639 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:17:43.639 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:17:43.639 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:17:43.639 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:17:43.639 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:17:43.640 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:17:43.640 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 84, 2024-09-14T06:17:43.640 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:17:43.640 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:17:43.640 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:17:43.640 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.640 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.640 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6800", 2024-09-14T06:17:43.640 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:17:43.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.641 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.641 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6801", 2024-09-14T06:17:43.641 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:17:43.641 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.641 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.641 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:17:43.641 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.641 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.641 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6802", 2024-09-14T06:17:43.642 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:17:43.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.642 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.642 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.642 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6803", 2024-09-14T06:17:43.642 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:17:43.642 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.642 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.642 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:17:43.642 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.642 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.642 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.642 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6806", 2024-09-14T06:17:43.642 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:17:43.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.643 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.643 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6807", 2024-09-14T06:17:43.643 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:17:43.643 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.643 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.643 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:17:43.643 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.643 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.643 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6804", 2024-09-14T06:17:43.643 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:17:43.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.644 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.644 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.644 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6805", 2024-09-14T06:17:43.644 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:17:43.644 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.644 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.644 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6801/1395936437", 2024-09-14T06:17:43.644 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6803/1395936437", 2024-09-14T06:17:43.644 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6807/1395936437", 2024-09-14T06:17:43.644 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6805/1395936437", 2024-09-14T06:17:43.644 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:17:43.645 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:17:43.645 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:17:43.645 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.645 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.645 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 7, 2024-09-14T06:17:43.645 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "c9162545-318b-4591-be64-cc8ace9097fe", 2024-09-14T06:17:43.645 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:17:43.645 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:17:43.645 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:17:43.645 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:17:43.645 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:17:43.645 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:17:43.645 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:17:43.646 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 23, 2024-09-14T06:17:43.646 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:17:43.646 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:17:43.646 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:17:43.646 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.646 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.646 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.646 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6808", 2024-09-14T06:17:43.646 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:17:43.646 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.646 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.646 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.646 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6809", 2024-09-14T06:17:43.647 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:17:43.647 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.647 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.647 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:17:43.647 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.647 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.647 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.647 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6810", 2024-09-14T06:17:43.647 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:17:43.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.647 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.647 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.647 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6811", 2024-09-14T06:17:43.647 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:17:43.648 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.648 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.648 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:17:43.648 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.648 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.648 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.648 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6814", 2024-09-14T06:17:43.648 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:17:43.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.648 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.648 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.648 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6815", 2024-09-14T06:17:43.648 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:17:43.648 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.649 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.649 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:17:43.649 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:17:43.649 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.649 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:17:43.649 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6812", 2024-09-14T06:17:43.649 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:17:43.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.649 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.649 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:17:43.649 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6813", 2024-09-14T06:17:43.649 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:17:43.649 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.649 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6809/711112216", 2024-09-14T06:17:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6811/711112216", 2024-09-14T06:17:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6815/711112216", 2024-09-14T06:17:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6813/711112216", 2024-09-14T06:17:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:17:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:17:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:17:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:17:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_xinfo": [ 2024-09-14T06:17:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 0, 2024-09-14T06:17:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:17:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:17:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:17:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:17:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:17:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.705575+0000", 2024-09-14T06:17:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:17:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 1, 2024-09-14T06:17:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:17:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:17:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:17:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:17:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:17:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.806946+0000", 2024-09-14T06:17:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:17:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 2, 2024-09-14T06:17:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:17:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:17:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:17:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:17:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:17:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.853831+0000", 2024-09-14T06:17:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:17:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 3, 2024-09-14T06:17:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:17:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:17:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:17:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:17:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:17:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.690315+0000", 2024-09-14T06:17:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:17:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 4, 2024-09-14T06:17:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:17:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:17:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:17:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:17:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:17:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.494526+0000", 2024-09-14T06:17:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:17:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 5, 2024-09-14T06:17:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:17:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:17:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:17:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:17:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:17:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.546122+0000", 2024-09-14T06:17:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:17:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 6, 2024-09-14T06:17:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:17:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:17:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:17:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:17:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:17:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.173691+0000", 2024-09-14T06:17:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:17:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 7, 2024-09-14T06:17:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:17:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:17:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:17:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:17:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:17:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.148509+0000", 2024-09-14T06:17:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:17:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:17:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap": [], 2024-09-14T06:17:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap_items": [ 2024-09-14T06:17:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "pgid": "3.1f", 2024-09-14T06:17:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "mappings": [ 2024-09-14T06:17:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "from": 1, 2024-09-14T06:17:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "to": 2 2024-09-14T06:17:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:17:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap_primaries": [], 2024-09-14T06:17:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_temp": [], 2024-09-14T06:17:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_temp": [], 2024-09-14T06:17:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "blocklist": {}, 2024-09-14T06:17:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "range_blocklist": {}, 2024-09-14T06:17:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profiles": { 2024-09-14T06:17:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "default": { 2024-09-14T06:17:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-failure-domain": "osd", 2024-09-14T06:17:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "k": "2", 2024-09-14T06:17:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "m": "1", 2024-09-14T06:17:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "plugin": "jerasure", 2024-09-14T06:17:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "technique": "reed_sol_van" 2024-09-14T06:17:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps_queue": [ 2024-09-14T06:17:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 3, 2024-09-14T06:17:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "snaps": [ 2024-09-14T06:17:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "begin": 33, 2024-09-14T06:17:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "length": 1 2024-09-14T06:17:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:17:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "new_removed_snaps": [ 2024-09-14T06:17:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 3, 2024-09-14T06:17:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "snaps": [ 2024-09-14T06:17:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "begin": 33, 2024-09-14T06:17:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "length": 1 2024-09-14T06:17:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:17:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "new_purged_snaps": [], 2024-09-14T06:17:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_node_flags": {}, 2024-09-14T06:17:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "device_class_flags": {}, 2024-09-14T06:17:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode": { 2024-09-14T06:17:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode_enabled": false, 2024-09-14T06:17:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_bucket_count": 0, 2024-09-14T06:17:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "degraded_stretch_mode": 0, 2024-09-14T06:17:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "recovering_stretch_mode": 0, 2024-09-14T06:17:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode_bucket": 0 2024-09-14T06:17:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:43.659 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:43.661 INFO:tasks.thrashosds.thrasher:j is {'epoch': 88, 'fsid': '038563b1-268f-461f-b6a6-4b4b41ad0236', 'created': '2024-09-14T06:15:49.493035+0000', 'modified': '2024-09-14T06:17:42.622710+0000', 'last_up_change': '2024-09-14T06:16:02.014989+0000', 'last_in_change': '2024-09-14T06:16:26.221200+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:16:02.429225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:16:08.129454+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.380000114440918, 'score_stable': 4.380000114440918, 'optimal_score': 1, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:16:18.928556+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 88, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'c669ea76-ed19-4c12-9e56-d79d62846f95', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6808', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6809', 'nonce': 1157559201}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6810', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6811', 'nonce': 1157559201}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6814', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6815', 'nonce': 1157559201}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6812', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6813', 'nonce': 1157559201}]}, 'public_addr': '172.21.15.39:6809/1157559201', 'cluster_addr': '172.21.15.39:6811/1157559201', 'heartbeat_back_addr': '172.21.15.39:6815/1157559201', 'heartbeat_front_addr': '172.21.15.39:6813/1157559201', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6816', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6817', 'nonce': 4062089212}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6818', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6819', 'nonce': 4062089212}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6822', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6823', 'nonce': 4062089212}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6820', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6821', 'nonce': 4062089212}]}, 'public_addr': '172.21.15.39:6817/4062089212', 'cluster_addr': '172.21.15.39:6819/4062089212', 'heartbeat_back_addr': '172.21.15.39:6823/4062089212', 'heartbeat_front_addr': '172.21.15.39:6821/4062089212', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9bda988f-e615-4545-849b-318c408c4c42', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6824', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6825', 'nonce': 1724773882}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6826', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6827', 'nonce': 1724773882}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6830', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6831', 'nonce': 1724773882}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6828', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6829', 'nonce': 1724773882}]}, 'public_addr': '172.21.15.39:6825/1724773882', 'cluster_addr': '172.21.15.39:6827/1724773882', 'heartbeat_back_addr': '172.21.15.39:6831/1724773882', 'heartbeat_front_addr': '172.21.15.39:6829/1724773882', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8ea6cd69-2ef1-42ea-b339-7394bf8f4a87', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 85, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6800', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6801', 'nonce': 3775031293}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6802', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6803', 'nonce': 3775031293}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6806', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6807', 'nonce': 3775031293}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6804', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6805', 'nonce': 3775031293}]}, 'public_addr': '172.21.15.39:6801/3775031293', 'cluster_addr': '172.21.15.39:6803/3775031293', 'heartbeat_back_addr': '172.21.15.39:6807/3775031293', 'heartbeat_front_addr': '172.21.15.39:6805/3775031293', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fea45e8a-e8ea-4963-b93f-0651b84c9e74', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 68, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6824', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6825', 'nonce': 2632317380}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6826', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6827', 'nonce': 2632317380}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6830', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6831', 'nonce': 2632317380}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6828', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6829', 'nonce': 2632317380}]}, 'public_addr': '172.21.15.87:6825/2632317380', 'cluster_addr': '172.21.15.87:6827/2632317380', 'heartbeat_back_addr': '172.21.15.87:6831/2632317380', 'heartbeat_front_addr': '172.21.15.87:6829/2632317380', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b601e852-a180-477f-86fc-6a213fa866f0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6816', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6817', 'nonce': 3501664149}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6818', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6819', 'nonce': 3501664149}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6822', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6823', 'nonce': 3501664149}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6820', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6821', 'nonce': 3501664149}]}, 'public_addr': '172.21.15.87:6817/3501664149', 'cluster_addr': '172.21.15.87:6819/3501664149', 'heartbeat_back_addr': '172.21.15.87:6823/3501664149', 'heartbeat_front_addr': '172.21.15.87:6821/3501664149', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '1113afe0-f524-41b1-8e51-eb5164602d38', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6800', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6801', 'nonce': 1395936437}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6802', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6803', 'nonce': 1395936437}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6806', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6807', 'nonce': 1395936437}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6804', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6805', 'nonce': 1395936437}]}, 'public_addr': '172.21.15.87:6801/1395936437', 'cluster_addr': '172.21.15.87:6803/1395936437', 'heartbeat_back_addr': '172.21.15.87:6807/1395936437', 'heartbeat_front_addr': '172.21.15.87:6805/1395936437', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c9162545-318b-4591-be64-cc8ace9097fe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6808', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6809', 'nonce': 711112216}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6810', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6811', 'nonce': 711112216}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6814', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6815', 'nonce': 711112216}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6812', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6813', 'nonce': 711112216}]}, 'public_addr': '172.21.15.87:6809/711112216', 'cluster_addr': '172.21.15.87:6811/711112216', 'heartbeat_back_addr': '172.21.15.87:6815/711112216', 'heartbeat_front_addr': '172.21.15.87:6813/711112216', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.705575+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.806946+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.853831+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.690315+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.494526+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.546122+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.173691+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.148509+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.1f', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 33, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 33, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T06:17:43.661 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:17:43.662 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:43.656+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:43.688 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:17:43.688 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:17:43.761 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:43.756+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:43.815 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:43.815 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:43.815 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:43.835 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:17:43.879 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:43.877+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:43.887 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:43.888 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:43.905 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:43.905 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:43.922 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:43.923 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:43.940 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:43.940 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:43.962 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:43.962 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:43.963 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:43.956+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:43.963 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:43.963 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:43.980 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:17:43.983 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:43.983 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:44.005 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:44.005 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:44.026 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:44.026 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:44.042 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:44.042 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:17:44.063 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:44.060+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:44.066 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-14T06:17:43.658671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1349972343,"num_objects":1569,"num_object_clones":268,"num_object_copies":3138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1569,"num_whiteouts":388,"num_read":99096,"num_read_kb":744202,"num_write":15529,"num_write_kb":430335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1127,"num_bytes_recovered":140497553,"num_keys_recovered":151,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12951,"ondisk_log_size":12951,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":412836,"kb_used_data":71768,"kb_used_omap":12,"kb_used_meta":340915,"kb_avail":754561884,"statfs":{"total":773094113280,"available":772671369216,"internally_reserved":0,"allocated":73490432,"data_stored":630343030,"data_compressed":17264948,"data_compressed_allocated":42393600,"data_compressed_original":606782774,"omap_allocated":12752,"internal_metadata":349097520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":56056405,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":3713,"num_read_kb":7015,"num_write":649,"num_write_kb":30414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.440376"},"pg_stats":[{"pgid":"3.1f","version":"86'516","reported_seq":3279,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.624592+0000","last_change":"2024-09-14T06:17:02.195565+0000","last_active":"2024-09-14T06:17:41.624592+0000","last_peered":"2024-09-14T06:17:41.624592+0000","last_clean":"2024-09-14T06:17:41.624592+0000","last_became_active":"2024-09-14T06:17:02.193313+0000","last_became_peered":"2024-09-14T06:17:02.193313+0000","last_unstale":"2024-09-14T06:17:41.624592+0000","last_undegraded":"2024-09-14T06:17:41.624592+0000","last_fullsized":"2024-09-14T06:17:41.624592+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:51:34.761417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010466430000000001,"stat_sum":{"num_bytes":54336740,"num_objects":48,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":10,"num_read":4340,"num_read_kb":53375,"num_write":644,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"86'333","reported_seq":4121,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.624333+0000","last_change":"2024-09-14T06:16:27.310639+0000","last_active":"2024-09-14T06:17:41.624333+0000","last_peered":"2024-09-14T06:17:41.624333+0000","last_clean":"2024-09-14T06:17:41.624333+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:17:41.624333+0000","last_undegraded":"2024-09-14T06:17:41.624333+0000","last_fullsized":"2024-09-14T06:17:41.624333+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:11:42.513574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095849699999999999,"stat_sum":{"num_bytes":63404700,"num_objects":48,"num_object_clones":9,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":15,"num_read":4742,"num_read_kb":42827,"num_write":298,"num_write_kb":15475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"87'1042","reported_seq":2133,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:42.020291+0000","last_change":"2024-09-14T06:17:38.227919+0000","last_active":"2024-09-14T06:17:42.020291+0000","last_peered":"2024-09-14T06:17:42.020291+0000","last_clean":"2024-09-14T06:17:42.020291+0000","last_became_active":"2024-09-14T06:17:38.226838+0000","last_became_peered":"2024-09-14T06:17:38.226838+0000","last_unstale":"2024-09-14T06:17:42.020291+0000","last_undegraded":"2024-09-14T06:17:42.020291+0000","last_fullsized":"2024-09-14T06:17:42.020291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":1042,"log_dups_size":0,"ondisk_log_size":1042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:07:02.490072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001319401,"stat_sum":{"num_bytes":32292365,"num_objects":49,"num_object_clones":7,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":8,"num_read":4901,"num_read_kb":14188,"num_write":1503,"num_write_kb":13979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10785266,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"87'823","reported_seq":2467,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.876630+0000","last_change":"2024-09-14T06:16:22.201834+0000","last_active":"2024-09-14T06:17:41.876630+0000","last_peered":"2024-09-14T06:17:41.876630+0000","last_clean":"2024-09-14T06:17:41.876630+0000","last_became_active":"2024-09-14T06:16:20.190215+0000","last_became_peered":"2024-09-14T06:16:20.190215+0000","last_unstale":"2024-09-14T06:17:41.876630+0000","last_undegraded":"2024-09-14T06:17:41.876630+0000","last_fullsized":"2024-09-14T06:17:41.876630+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:08:24.896344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026065160000000001,"stat_sum":{"num_bytes":25862491,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":12,"num_read":5522,"num_read_kb":9572,"num_write":1222,"num_write_kb":6513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"79'291","reported_seq":3517,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.618918+0000","last_change":"2024-09-14T06:16:22.193116+0000","last_active":"2024-09-14T06:17:41.618918+0000","last_peered":"2024-09-14T06:17:41.618918+0000","last_clean":"2024-09-14T06:17:41.618918+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:17:41.618918+0000","last_undegraded":"2024-09-14T06:17:41.618918+0000","last_fullsized":"2024-09-14T06:17:41.618918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:17:00.634226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014745120000000001,"stat_sum":{"num_bytes":52171639,"num_objects":51,"num_object_clones":9,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":13,"num_read":3322,"num_read_kb":39042,"num_write":328,"num_write_kb":25723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"79'468","reported_seq":2765,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.987204+0000","last_change":"2024-09-14T06:16:22.201461+0000","last_active":"2024-09-14T06:17:41.987204+0000","last_peered":"2024-09-14T06:17:41.987204+0000","last_clean":"2024-09-14T06:17:41.987204+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:17:41.987204+0000","last_undegraded":"2024-09-14T06:17:41.987204+0000","last_fullsized":"2024-09-14T06:17:41.987204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:21:54.740165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002267542,"stat_sum":{"num_bytes":12869368,"num_objects":53,"num_object_clones":8,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":22,"num_read":3666,"num_read_kb":12739,"num_write":607,"num_write_kb":7500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"80'449","reported_seq":1765,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619096+0000","last_change":"2024-09-14T06:16:22.191773+0000","last_active":"2024-09-14T06:17:41.619096+0000","last_peered":"2024-09-14T06:17:41.619096+0000","last_clean":"2024-09-14T06:17:41.619096+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:17:41.619096+0000","last_undegraded":"2024-09-14T06:17:41.619096+0000","last_fullsized":"2024-09-14T06:17:41.619096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:45:42.492916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073833899999999997,"stat_sum":{"num_bytes":35400414,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":2666,"num_read_kb":19057,"num_write":647,"num_write_kb":12862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"80'307","reported_seq":2595,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.624167+0000","last_change":"2024-09-14T06:16:22.205296+0000","last_active":"2024-09-14T06:17:41.624167+0000","last_peered":"2024-09-14T06:17:41.624167+0000","last_clean":"2024-09-14T06:17:41.624167+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:17:41.624167+0000","last_undegraded":"2024-09-14T06:17:41.624167+0000","last_fullsized":"2024-09-14T06:17:41.624167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:57:31.710099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012474280000000001,"stat_sum":{"num_bytes":44544444,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":11,"num_read":3415,"num_read_kb":37447,"num_write":297,"num_write_kb":32929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"86'315","reported_seq":1369,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.625010+0000","last_change":"2024-09-14T06:17:37.234947+0000","last_active":"2024-09-14T06:17:41.625010+0000","last_peered":"2024-09-14T06:17:41.625010+0000","last_clean":"2024-09-14T06:17:41.625010+0000","last_became_active":"2024-09-14T06:17:37.233410+0000","last_became_peered":"2024-09-14T06:17:37.233410+0000","last_unstale":"2024-09-14T06:17:41.625010+0000","last_undegraded":"2024-09-14T06:17:41.625010+0000","last_fullsized":"2024-09-14T06:17:41.625010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:51:53.583723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001106627,"stat_sum":{"num_bytes":27316305,"num_objects":48,"num_object_clones":12,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":16,"num_read":1181,"num_read_kb":17277,"num_write":285,"num_write_kb":17182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":23467377,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"87'355","reported_seq":2005,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.621585+0000","last_change":"2024-09-14T06:17:37.236314+0000","last_active":"2024-09-14T06:17:41.621585+0000","last_peered":"2024-09-14T06:17:41.621585+0000","last_clean":"2024-09-14T06:17:41.621585+0000","last_became_active":"2024-09-14T06:17:37.235311+0000","last_became_peered":"2024-09-14T06:17:37.235311+0000","last_unstale":"2024-09-14T06:17:41.621585+0000","last_undegraded":"2024-09-14T06:17:41.621585+0000","last_fullsized":"2024-09-14T06:17:41.621585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:09.235644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001557389,"stat_sum":{"num_bytes":57848052,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":11,"num_read":2242,"num_read_kb":16285,"num_write":452,"num_write_kb":10272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":6785399,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"87'350","reported_seq":3401,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.643667+0000","last_change":"2024-09-14T06:16:22.194304+0000","last_active":"2024-09-14T06:17:41.643667+0000","last_peered":"2024-09-14T06:17:41.643667+0000","last_clean":"2024-09-14T06:17:41.643667+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:17:41.643667+0000","last_undegraded":"2024-09-14T06:17:41.643667+0000","last_fullsized":"2024-09-14T06:17:41.643667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:32:31.047615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019733379999999998,"stat_sum":{"num_bytes":71775725,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":7,"num_read":3901,"num_read_kb":64430,"num_write":448,"num_write_kb":22209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"80'246","reported_seq":1698,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.630309+0000","last_change":"2024-09-14T06:17:37.232535+0000","last_active":"2024-09-14T06:17:41.630309+0000","last_peered":"2024-09-14T06:17:41.630309+0000","last_clean":"2024-09-14T06:17:41.630309+0000","last_became_active":"2024-09-14T06:17:37.231244+0000","last_became_peered":"2024-09-14T06:17:37.231244+0000","last_unstale":"2024-09-14T06:17:41.630309+0000","last_undegraded":"2024-09-14T06:17:41.630309+0000","last_fullsized":"2024-09-14T06:17:41.630309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:22:59.070149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001465247,"stat_sum":{"num_bytes":61166345,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":10,"num_read":2671,"num_read_kb":26987,"num_write":230,"num_write_kb":15728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":32506492,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619617+0000","last_change":"2024-09-14T06:17:34.200709+0000","last_active":"2024-09-14T06:17:41.619617+0000","last_peered":"2024-09-14T06:17:41.619617+0000","last_clean":"2024-09-14T06:17:41.619617+0000","last_became_active":"2024-09-14T06:17:34.200238+0000","last_became_peered":"2024-09-14T06:17:34.200238+0000","last_unstale":"2024-09-14T06:17:41.619617+0000","last_undegraded":"2024-09-14T06:17:41.619617+0000","last_fullsized":"2024-09-14T06:17:41.619617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:34.864695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"79'364","reported_seq":914,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.618987+0000","last_change":"2024-09-14T06:16:22.192972+0000","last_active":"2024-09-14T06:17:41.618987+0000","last_peered":"2024-09-14T06:17:41.618987+0000","last_clean":"2024-09-14T06:17:41.618987+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:17:41.618987+0000","last_undegraded":"2024-09-14T06:17:41.618987+0000","last_fullsized":"2024-09-14T06:17:41.618987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:56:10.514649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086504599999999996,"stat_sum":{"num_bytes":29669358,"num_objects":48,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":10,"num_read":1315,"num_read_kb":9524,"num_write":422,"num_write_kb":9131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619067+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:17:41.619067+0000","last_peered":"2024-09-14T06:17:41.619067+0000","last_clean":"2024-09-14T06:17:41.619067+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:17:41.619067+0000","last_undegraded":"2024-09-14T06:17:41.619067+0000","last_fullsized":"2024-09-14T06:17:41.619067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"86'300","reported_seq":2898,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.632160+0000","last_change":"2024-09-14T06:16:22.192683+0000","last_active":"2024-09-14T06:17:41.632160+0000","last_peered":"2024-09-14T06:17:41.632160+0000","last_clean":"2024-09-14T06:17:41.632160+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:17:41.632160+0000","last_undegraded":"2024-09-14T06:17:41.632160+0000","last_fullsized":"2024-09-14T06:17:41.632160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:04:04.548750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001328716,"stat_sum":{"num_bytes":33183963,"num_objects":53,"num_object_clones":9,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":11,"num_read":3071,"num_read_kb":28400,"num_write":271,"num_write_kb":15212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619874+0000","last_change":"2024-09-14T06:17:34.196668+0000","last_active":"2024-09-14T06:17:41.619874+0000","last_peered":"2024-09-14T06:17:41.619874+0000","last_clean":"2024-09-14T06:17:41.619874+0000","last_became_active":"2024-09-14T06:17:34.196072+0000","last_became_peered":"2024-09-14T06:17:34.196072+0000","last_unstale":"2024-09-14T06:17:41.619874+0000","last_undegraded":"2024-09-14T06:17:41.619874+0000","last_fullsized":"2024-09-14T06:17:41.619874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:18:45.616834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"86'297","reported_seq":1619,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.627187+0000","last_change":"2024-09-14T06:16:22.213152+0000","last_active":"2024-09-14T06:17:41.627187+0000","last_peered":"2024-09-14T06:17:41.627187+0000","last_clean":"2024-09-14T06:17:41.627187+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:17:41.627187+0000","last_undegraded":"2024-09-14T06:17:41.627187+0000","last_fullsized":"2024-09-14T06:17:41.627187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:28:34.843601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057590699999999996,"stat_sum":{"num_bytes":57727010,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":8,"num_read":1567,"num_read_kb":12759,"num_write":257,"num_write_kb":11518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.624824+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:17:41.624824+0000","last_peered":"2024-09-14T06:17:41.624824+0000","last_clean":"2024-09-14T06:17:41.624824+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:17:41.624824+0000","last_undegraded":"2024-09-14T06:17:41.624824+0000","last_fullsized":"2024-09-14T06:17:41.624824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"79'215","reported_seq":1440,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.624076+0000","last_change":"2024-09-14T06:16:33.548583+0000","last_active":"2024-09-14T06:17:41.624076+0000","last_peered":"2024-09-14T06:17:41.624076+0000","last_clean":"2024-09-14T06:17:41.624076+0000","last_became_active":"2024-09-14T06:16:33.547925+0000","last_became_peered":"2024-09-14T06:16:33.547925+0000","last_unstale":"2024-09-14T06:17:41.624076+0000","last_undegraded":"2024-09-14T06:17:41.624076+0000","last_fullsized":"2024-09-14T06:17:41.624076+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:23:39.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089329700000000004,"stat_sum":{"num_bytes":69139655,"num_objects":49,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":16,"num_read":1310,"num_read_kb":21229,"num_write":226,"num_write_kb":9950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"87'228","reported_seq":702,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.956273+0000","last_change":"2024-09-14T06:16:27.282745+0000","last_active":"2024-09-14T06:17:41.956273+0000","last_peered":"2024-09-14T06:17:41.956273+0000","last_clean":"2024-09-14T06:17:41.956273+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:17:41.956273+0000","last_undegraded":"2024-09-14T06:17:41.956273+0000","last_fullsized":"2024-09-14T06:17:41.956273+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:53:56.212817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002354358,"stat_sum":{"num_bytes":38932892,"num_objects":48,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":361,"num_read_kb":4069,"num_write":165,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619498+0000","last_change":"2024-09-14T06:17:34.203977+0000","last_active":"2024-09-14T06:17:41.619498+0000","last_peered":"2024-09-14T06:17:41.619498+0000","last_clean":"2024-09-14T06:17:41.619498+0000","last_became_active":"2024-09-14T06:17:34.199244+0000","last_became_peered":"2024-09-14T06:17:34.199244+0000","last_unstale":"2024-09-14T06:17:41.619498+0000","last_undegraded":"2024-09-14T06:17:41.619498+0000","last_fullsized":"2024-09-14T06:17:41.619498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:21.934094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"80'351","reported_seq":1584,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.624008+0000","last_change":"2024-09-14T06:16:22.205051+0000","last_active":"2024-09-14T06:17:41.624008+0000","last_peered":"2024-09-14T06:17:41.624008+0000","last_clean":"2024-09-14T06:17:41.624008+0000","last_became_active":"2024-09-14T06:16:20.196215+0000","last_became_peered":"2024-09-14T06:16:20.196215+0000","last_unstale":"2024-09-14T06:17:41.624008+0000","last_undegraded":"2024-09-14T06:17:41.624008+0000","last_fullsized":"2024-09-14T06:17:41.624008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:42:24.322885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001116559,"stat_sum":{"num_bytes":33594565,"num_objects":40,"num_object_clones":3,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":9,"num_read":2026,"num_read_kb":18917,"num_write":467,"num_write_kb":11041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T06:17:38.921109+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:17:38.921109+0000","last_peered":"2024-09-14T06:17:38.921109+0000","last_clean":"2024-09-14T06:17:38.921109+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:17:38.921109+0000","last_undegraded":"2024-09-14T06:17:38.921109+0000","last_fullsized":"2024-09-14T06:17:38.921109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'484","reported_seq":1917,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.624452+0000","last_change":"2024-09-14T06:16:27.316499+0000","last_active":"2024-09-14T06:17:41.624452+0000","last_peered":"2024-09-14T06:17:41.624452+0000","last_clean":"2024-09-14T06:17:41.624452+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:17:41.624452+0000","last_undegraded":"2024-09-14T06:17:41.624452+0000","last_fullsized":"2024-09-14T06:17:41.624452+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:10:23.624742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052773999999999996,"stat_sum":{"num_bytes":17346293,"num_objects":37,"num_object_clones":4,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":9,"num_read":2816,"num_read_kb":8761,"num_write":671,"num_write_kb":8775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619379+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:17:41.619379+0000","last_peered":"2024-09-14T06:17:41.619379+0000","last_clean":"2024-09-14T06:17:41.619379+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:17:41.619379+0000","last_undegraded":"2024-09-14T06:17:41.619379+0000","last_fullsized":"2024-09-14T06:17:41.619379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"86'368","reported_seq":1546,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.618904+0000","last_change":"2024-09-14T06:16:27.302045+0000","last_active":"2024-09-14T06:17:41.618904+0000","last_peered":"2024-09-14T06:17:41.618904+0000","last_clean":"2024-09-14T06:17:41.618904+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:17:41.618904+0000","last_undegraded":"2024-09-14T06:17:41.618904+0000","last_fullsized":"2024-09-14T06:17:41.618904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:33:28.609283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060103200000000004,"stat_sum":{"num_bytes":62676633,"num_objects":52,"num_object_clones":6,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":8,"num_read":1774,"num_read_kb":22633,"num_write":296,"num_write_kb":14511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":171,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619701+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:17:41.619701+0000","last_peered":"2024-09-14T06:17:41.619701+0000","last_clean":"2024-09-14T06:17:41.619701+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:17:41.619701+0000","last_undegraded":"2024-09-14T06:17:41.619701+0000","last_fullsized":"2024-09-14T06:17:41.619701+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":241,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.616159+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:17:41.616159+0000","last_peered":"2024-09-14T06:17:41.616159+0000","last_clean":"2024-09-14T06:17:41.616159+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:17:41.616159+0000","last_undegraded":"2024-09-14T06:17:41.616159+0000","last_fullsized":"2024-09-14T06:17:41.616159+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"79'379","reported_seq":1368,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.630646+0000","last_change":"2024-09-14T06:16:22.191260+0000","last_active":"2024-09-14T06:17:41.630646+0000","last_peered":"2024-09-14T06:17:41.630646+0000","last_clean":"2024-09-14T06:17:41.630646+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:17:41.630646+0000","last_undegraded":"2024-09-14T06:17:41.630646+0000","last_fullsized":"2024-09-14T06:17:41.630646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:05:10.411581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047498199999999999,"stat_sum":{"num_bytes":12407027,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":14,"num_read":2048,"num_read_kb":6659,"num_write":539,"num_write_kb":6840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"87'344","reported_seq":1585,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619487+0000","last_change":"2024-09-14T06:17:38.225097+0000","last_active":"2024-09-14T06:17:41.619487+0000","last_peered":"2024-09-14T06:17:41.619487+0000","last_clean":"2024-09-14T06:17:41.619487+0000","last_became_active":"2024-09-14T06:17:38.224238+0000","last_became_peered":"2024-09-14T06:17:38.224238+0000","last_unstale":"2024-09-14T06:17:41.619487+0000","last_undegraded":"2024-09-14T06:17:41.619487+0000","last_fullsized":"2024-09-14T06:17:41.619487+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:54.197052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418299,"stat_sum":{"num_bytes":42339757,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":8,"num_read":2186,"num_read_kb":13625,"num_write":509,"num_write_kb":14348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8477378,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"87'311","reported_seq":2020,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.924951+0000","last_change":"2024-09-14T06:16:22.323725+0000","last_active":"2024-09-14T06:17:41.924951+0000","last_peered":"2024-09-14T06:17:41.924951+0000","last_clean":"2024-09-14T06:17:41.924951+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:17:41.924951+0000","last_undegraded":"2024-09-14T06:17:41.924951+0000","last_fullsized":"2024-09-14T06:17:41.924951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:46:42.122526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024221899999999998,"stat_sum":{"num_bytes":35230985,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":8,"num_read":2159,"num_read_kb":16053,"num_write":266,"num_write_kb":13801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"86'245","reported_seq":2529,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.471334+0000","last_change":"2024-09-14T06:16:22.219322+0000","last_active":"2024-09-14T06:17:41.471334+0000","last_peered":"2024-09-14T06:17:41.471334+0000","last_clean":"2024-09-14T06:17:41.471334+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:17:41.471334+0000","last_undegraded":"2024-09-14T06:17:41.471334+0000","last_fullsized":"2024-09-14T06:17:41.471334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:01:35.004877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045017199999999999,"stat_sum":{"num_bytes":55074078,"num_objects":55,"num_object_clones":10,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":15,"num_read":2691,"num_read_kb":16643,"num_write":224,"num_write_kb":9847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"87'139","reported_seq":1388,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.623573+0000","last_change":"2024-09-14T06:16:22.195705+0000","last_active":"2024-09-14T06:17:41.623573+0000","last_peered":"2024-09-14T06:17:41.623573+0000","last_clean":"2024-09-14T06:17:41.623573+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:17:41.623573+0000","last_undegraded":"2024-09-14T06:17:41.623573+0000","last_fullsized":"2024-09-14T06:17:41.623573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:02:15.957880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025578329999999998,"stat_sum":{"num_bytes":46202051,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":7,"num_read":1256,"num_read_kb":29057,"num_write":210,"num_write_kb":11572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"87'623","reported_seq":2318,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.667110+0000","last_change":"2024-09-14T06:17:38.228193+0000","last_active":"2024-09-14T06:17:41.667110+0000","last_peered":"2024-09-14T06:17:41.667110+0000","last_clean":"2024-09-14T06:17:41.667110+0000","last_became_active":"2024-09-14T06:17:38.227106+0000","last_became_peered":"2024-09-14T06:17:38.227106+0000","last_unstale":"2024-09-14T06:17:41.667110+0000","last_undegraded":"2024-09-14T06:17:41.667110+0000","last_fullsized":"2024-09-14T06:17:41.667110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:44.413486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001627671,"stat_sum":{"num_bytes":37374219,"num_objects":56,"num_object_clones":13,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":16,"num_read":3735,"num_read_kb":26924,"num_write":955,"num_write_kb":20272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":29390214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"87'609","reported_seq":4638,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.988152+0000","last_change":"2024-09-14T06:16:33.547557+0000","last_active":"2024-09-14T06:17:41.988152+0000","last_peered":"2024-09-14T06:17:41.988152+0000","last_clean":"2024-09-14T06:17:41.988152+0000","last_became_active":"2024-09-14T06:16:33.546796+0000","last_became_peered":"2024-09-14T06:16:33.546796+0000","last_unstale":"2024-09-14T06:17:41.988152+0000","last_undegraded":"2024-09-14T06:17:41.988152+0000","last_fullsized":"2024-09-14T06:17:41.988152+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:13:41.806382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026669160000000001,"stat_sum":{"num_bytes":48267984,"num_objects":53,"num_object_clones":7,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":12,"num_read":9212,"num_read_kb":50798,"num_write":583,"num_write_kb":16104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"80'577","reported_seq":2538,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.642399+0000","last_change":"2024-09-14T06:16:22.195247+0000","last_active":"2024-09-14T06:17:41.642399+0000","last_peered":"2024-09-14T06:17:41.642399+0000","last_clean":"2024-09-14T06:17:41.642399+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:17:41.642399+0000","last_undegraded":"2024-09-14T06:17:41.642399+0000","last_fullsized":"2024-09-14T06:17:41.642399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:12:24.464514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022870590000000001,"stat_sum":{"num_bytes":34980582,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":14,"num_read":4106,"num_read_kb":15897,"num_write":707,"num_write_kb":12738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"87'277","reported_seq":2178,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.963636+0000","last_change":"2024-09-14T06:16:22.200094+0000","last_active":"2024-09-14T06:17:41.963636+0000","last_peered":"2024-09-14T06:17:41.963636+0000","last_clean":"2024-09-14T06:17:41.963636+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:17:41.963636+0000","last_undegraded":"2024-09-14T06:17:41.963636+0000","last_fullsized":"2024-09-14T06:17:41.963636+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:37:35.508264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011343239999999999,"stat_sum":{"num_bytes":53143462,"num_objects":57,"num_object_clones":8,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":14,"num_read":2240,"num_read_kb":47484,"num_write":277,"num_write_kb":14632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"87'458","reported_seq":2540,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.944980+0000","last_change":"2024-09-14T06:17:38.225212+0000","last_active":"2024-09-14T06:17:41.944980+0000","last_peered":"2024-09-14T06:17:41.944980+0000","last_clean":"2024-09-14T06:17:41.944980+0000","last_became_active":"2024-09-14T06:17:38.224251+0000","last_became_peered":"2024-09-14T06:17:38.224251+0000","last_unstale":"2024-09-14T06:17:41.944980+0000","last_undegraded":"2024-09-14T06:17:41.944980+0000","last_fullsized":"2024-09-14T06:17:41.944980+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:31:43.282194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226324,"stat_sum":{"num_bytes":19177105,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":15,"num_read":4118,"num_read_kb":14371,"num_write":540,"num_write_kb":7654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":12239008,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"80'423","reported_seq":2649,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.624128+0000","last_change":"2024-09-14T06:17:37.232951+0000","last_active":"2024-09-14T06:17:41.624128+0000","last_peered":"2024-09-14T06:17:41.624128+0000","last_clean":"2024-09-14T06:17:41.624128+0000","last_became_active":"2024-09-14T06:17:37.232145+0000","last_became_peered":"2024-09-14T06:17:37.232145+0000","last_unstale":"2024-09-14T06:17:41.624128+0000","last_undegraded":"2024-09-14T06:17:41.624128+0000","last_fullsized":"2024-09-14T06:17:41.624128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:51:31.847884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082016299999999995,"stat_sum":{"num_bytes":17887056,"num_objects":47,"num_object_clones":6,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":14,"num_read":3083,"num_read_kb":10830,"num_write":525,"num_write_kb":8437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":10176554,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"79'389","reported_seq":3592,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T06:17:41.619560+0000","last_change":"2024-09-14T06:16:22.322160+0000","last_active":"2024-09-14T06:17:41.619560+0000","last_peered":"2024-09-14T06:17:41.619560+0000","last_clean":"2024-09-14T06:17:41.619560+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:17:41.619560+0000","last_undegraded":"2024-09-14T06:17:41.619560+0000","last_fullsized":"2024-09-14T06:17:41.619560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:32:32.561603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011502319999999999,"stat_sum":{"num_bytes":66038693,"num_objects":56,"num_object_clones":12,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":16,"num_read":5387,"num_read_kb":16288,"num_write":355,"num_write_kb":5570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1349381956,"num_objects":1565,"num_object_clones":268,"num_object_copies":3130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1565,"num_whiteouts":388,"num_read":99030,"num_read_kb":744147,"num_write":15426,"num_write_kb":428705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1124,"num_bytes_recovered":140497534,"num_keys_recovered":151,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64811008,"data_stored":623617065,"data_compressed":17110337,"data_compressed_allocated":42094592,"data_compressed_original":603994405,"omap_allocated":0,"internal_metadata":0},"log_size":12876,"ondisk_log_size":12876,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42076,"kb_used_data":10892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329764,"statfs":{"total":96636764160,"available":96593678336,"internally_reserved":0,"allocated":11153408,"data_stored":95084645,"data_compressed":3249521,"data_compressed_allocated":7127040,"data_compressed_original":91973016,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509463,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42276,"kb_used_data":11088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329564,"statfs":{"total":96636764160,"available":96593473536,"internally_reserved":0,"allocated":11354112,"data_stored":94050392,"data_compressed":2224067,"data_compressed_allocated":6352896,"data_compressed_original":90203129,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78948,"kb_used_data":7504,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94292892,"statfs":{"total":96636764160,"available":96555921408,"internally_reserved":0,"allocated":7684096,"data_stored":64377934,"data_compressed":1692911,"data_compressed_allocated":4415488,"data_compressed_original":61953833,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63728,"kb_used_data":10336,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94308112,"statfs":{"total":96636764160,"available":96571506688,"internally_reserved":0,"allocated":10584064,"data_stored":92432051,"data_compressed":2396485,"data_compressed_allocated":6148096,"data_compressed_original":88883168,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542167,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40860,"kb_used_data":5512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330980,"statfs":{"total":96636764160,"available":96594923520,"internally_reserved":0,"allocated":5644288,"data_stored":44707210,"data_compressed":1436486,"data_compressed_allocated":2777088,"data_compressed_original":42597756,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48820,"kb_used_data":9316,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323020,"statfs":{"total":96636764160,"available":96586772480,"internally_reserved":0,"allocated":9539584,"data_stored":79799958,"data_compressed":2108388,"data_compressed_allocated":5210112,"data_compressed_original":76518299,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542166,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49088,"kb_used_data":9584,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322752,"statfs":{"total":96636764160,"available":96586498048,"internally_reserved":0,"allocated":9814016,"data_stored":92098360,"data_compressed":2507870,"data_compressed_allocated":6246400,"data_compressed_original":89532206,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47040,"kb_used_data":7536,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324800,"statfs":{"total":96636764160,"available":96588595200,"internally_reserved":0,"allocated":7716864,"data_stored":67792480,"data_compressed":1649220,"data_compressed_allocated":4116480,"data_compressed_original":65121367,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9539584,"data_stored":91866454,"data_compressed":2396079,"data_compressed_allocated":6139904,"data_compressed_original":88797152,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6602752,"data_stored":63317807,"data_compressed":1682721,"data_compressed_allocated":4374528,"data_compressed_original":61380393,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10018816,"data_stored":92379871,"data_compressed":2107433,"data_compressed_allocated":6184960,"data_compressed_original":89124224,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10067968,"data_stored":94141771,"data_compressed":3232343,"data_compressed_allocated":7081984,"data_compressed_original":91508736,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6709248,"data_stored":67328555,"data_compressed":1649220,"data_compressed_allocated":4116480,"data_compressed_original":65121367,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8695808,"data_stored":90975490,"data_compressed":2497552,"data_compressed_allocated":6205440,"data_compressed_original":88925998,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8540160,"data_stored":79363832,"data_compressed":2108503,"data_compressed_allocated":5214208,"data_compressed_original":76538779,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4636672,"data_stored":44243285,"data_compressed":1436486,"data_compressed_allocated":2777088,"data_compressed_original":42597756,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:17:44.068 INFO:tasks.thrashosds.thrasher:Setting 3.1c to [4, 6] 2024-09-14T06:17:44.068 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1c', '4', '6'] 2024-09-14T06:17:44.068 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.1c 4 6 2024-09-14T06:17:44.104 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:44.104 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:44.117 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:17:44.164 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:44.160+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:44.265 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:44.260+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:44.277 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:44.278 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:44.291 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:17:44.366 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:44.361+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:44.400 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:44.400 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:44.413 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:17:44.467 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:44.461+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:44.545 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:44.545 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:44.559 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:17:44.567 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:44.565+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:44.668 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:44.665+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:44.684 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:44.685 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:44.691 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-09-14T06:17:44.691 INFO:teuthology.orchestra.run.smithi039.stderr:set 3.1c pg_upmap mapping to [4,6] 2024-09-14T06:17:44.702 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:17:44.769 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:44.765+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:44.821 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:44.821 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:44.835 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:17:44.870 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:44.865+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:44.961 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:44.962 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:44.971 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:44.968+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:44.973 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:17:45.072 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:45.069+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:45.137 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:45.137 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:45.152 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:17:45.173 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:45.169+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:45.271 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:45.271 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:45.274 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:45.269+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:45.286 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:17:45.375 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:45.368+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:45.409 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:45.409 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:45.423 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:17:45.511 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:45.508+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:45.577 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:45.572+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:45.587 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:45.587 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:45.601 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:17:45.691 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:45.688+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:45.772 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:45.772 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:45.779 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:45.772+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:45.786 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:17:45.880 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:45.877+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:45.948 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:45.948 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:45.962 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:17:46.007 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:46.004+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:46.044 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:17:46.078 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:46.079 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:46.092 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:46.089+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:46.093 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:17:46.182 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:46.176+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:46.203 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:46.204 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:46.220 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:46.220 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:46.238 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:46.238 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:46.254 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:46.255 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:46.261 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:46.261 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:46.273 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:46.273 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:46.276 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:17:46.293 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:46.293 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:46.310 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:46.311 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:46.319 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:46.316+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:46.330 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:46.330 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:46.395 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:46.393+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:46.401 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:46.401 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:46.401 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:46.401 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:46.416 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:17:46.507 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:46.505+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:46.585 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:46.585 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:46.585 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:46.585 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:46.585 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:46.585 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:46.581+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:46.599 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:17:46.688 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:46.685+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:46.720 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:17:46.777 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:46.777 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:46.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:46.777 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:46.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.778 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40393 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 868352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.778 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.708703+0000", 2024-09-14T06:17:46.778 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.066582336000003, 2024-09-14T06:17:46.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12669228199999999, 2024-09-14T06:17:46.778 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.778 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.778 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.778 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.778 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.778 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.778 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40393 2024-09-14T06:17:46.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.778 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708703+0000", 2024-09-14T06:17:46.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708703+0000", 2024-09-14T06:17:46.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708881+0000", 2024-09-14T06:17:46.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00017825 2024-09-14T06:17:46.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708885+0000", 2024-09-14T06:17:46.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.019e-06 2024-09-14T06:17:46.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708894+0000", 2024-09-14T06:17:46.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5469999999999997e-06 2024-09-14T06:17:46.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708900+0000", 2024-09-14T06:17:46.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.421e-06 2024-09-14T06:17:46.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835012+0000", 2024-09-14T06:17:46.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12611203400000001 2024-09-14T06:17:46.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835095+0000", 2024-09-14T06:17:46.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.229e-05 2024-09-14T06:17:46.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835395+0000", 2024-09-14T06:17:46.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000300721 2024-09-14T06:17:46.782 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.782 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.782 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.782 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40395 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 876544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.782 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.708914+0000", 2024-09-14T06:17:46.782 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.066371703999998, 2024-09-14T06:17:46.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12688781199999999, 2024-09-14T06:17:46.782 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.783 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.783 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.783 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.783 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.783 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.783 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40395 2024-09-14T06:17:46.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.783 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708914+0000", 2024-09-14T06:17:46.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708914+0000", 2024-09-14T06:17:46.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709267+0000", 2024-09-14T06:17:46.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00035336999999999999 2024-09-14T06:17:46.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709272+0000", 2024-09-14T06:17:46.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4430000000000002e-06 2024-09-14T06:17:46.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709283+0000", 2024-09-14T06:17:46.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1034e-05 2024-09-14T06:17:46.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709291+0000", 2024-09-14T06:17:46.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4739999999999999e-06 2024-09-14T06:17:46.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835428+0000", 2024-09-14T06:17:46.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126137052 2024-09-14T06:17:46.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835506+0000", 2024-09-14T06:17:46.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8492000000000007e-05 2024-09-14T06:17:46.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.787 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835801+0000", 2024-09-14T06:17:46.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00029494699999999999 2024-09-14T06:17:46.787 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.787 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.787 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.787 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40397 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.787 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.709307+0000", 2024-09-14T06:17:46.788 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.065978600999998, 2024-09-14T06:17:46.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126901031, 2024-09-14T06:17:46.788 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.788 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.788 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.788 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.788 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.788 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.788 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40397 2024-09-14T06:17:46.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.788 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709307+0000", 2024-09-14T06:17:46.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709307+0000", 2024-09-14T06:17:46.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709590+0000", 2024-09-14T06:17:46.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00028368499999999999 2024-09-14T06:17:46.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709596+0000", 2024-09-14T06:17:46.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3940000000000004e-06 2024-09-14T06:17:46.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709605+0000", 2024-09-14T06:17:46.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8650000000000007e-06 2024-09-14T06:17:46.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.791 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.791 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709610+0000", 2024-09-14T06:17:46.791 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1139999999999997e-06 2024-09-14T06:17:46.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835834+0000", 2024-09-14T06:17:46.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12622423199999999 2024-09-14T06:17:46.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835915+0000", 2024-09-14T06:17:46.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0859999999999995e-05 2024-09-14T06:17:46.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836208+0000", 2024-09-14T06:17:46.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00029288099999999999 2024-09-14T06:17:46.793 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.793 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.793 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.793 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40399 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.793 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.709624+0000", 2024-09-14T06:17:46.793 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.065660803999997, 2024-09-14T06:17:46.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.127024368, 2024-09-14T06:17:46.794 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.794 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.794 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.794 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.794 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.794 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.794 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40399 2024-09-14T06:17:46.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.794 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709624+0000", 2024-09-14T06:17:46.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709624+0000", 2024-09-14T06:17:46.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710144+0000", 2024-09-14T06:17:46.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00051909700000000005 2024-09-14T06:17:46.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710156+0000", 2024-09-14T06:17:46.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2846e-05 2024-09-14T06:17:46.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.796 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710166+0000", 2024-09-14T06:17:46.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:17:46.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.796 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710174+0000", 2024-09-14T06:17:46.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:17:46.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836240+0000", 2024-09-14T06:17:46.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12606634899999999 2024-09-14T06:17:46.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836320+0000", 2024-09-14T06:17:46.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0073000000000004e-05 2024-09-14T06:17:46.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836649+0000", 2024-09-14T06:17:46.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032834999999999999 2024-09-14T06:17:46.798 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.798 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.798 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.798 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40401 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.798 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.710189+0000", 2024-09-14T06:17:46.798 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.065096637000003, 2024-09-14T06:17:46.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126903455, 2024-09-14T06:17:46.799 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.799 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.799 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.799 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.799 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.799 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.799 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40401 2024-09-14T06:17:46.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.799 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710189+0000", 2024-09-14T06:17:46.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.800 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710189+0000", 2024-09-14T06:17:46.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.800 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712809+0000", 2024-09-14T06:17:46.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026201179999999998 2024-09-14T06:17:46.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712815+0000", 2024-09-14T06:17:46.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7960000000000001e-06 2024-09-14T06:17:46.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712825+0000", 2024-09-14T06:17:46.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0044000000000001e-05 2024-09-14T06:17:46.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712832+0000", 2024-09-14T06:17:46.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.328e-06 2024-09-14T06:17:46.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836682+0000", 2024-09-14T06:17:46.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.123850141 2024-09-14T06:17:46.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836762+0000", 2024-09-14T06:17:46.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0207e-05 2024-09-14T06:17:46.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.837092+0000", 2024-09-14T06:17:46.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000329821 2024-09-14T06:17:46.803 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.803 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.803 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.803 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40407 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.803 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:46.803 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.062403222, 2024-09-14T06:17:46.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.127190898, 2024-09-14T06:17:46.803 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.803 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.803 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.803 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.804 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.804 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.804 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40407 2024-09-14T06:17:46.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.804 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.804 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.804 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:46.804 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.804 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.804 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:46.804 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.805 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.805 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:46.805 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0900000000000002e-07 2024-09-14T06:17:46.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.805 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.806 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712884+0000", 2024-09-14T06:17:46.806 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5579999999999999e-06 2024-09-14T06:17:46.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.806 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.806 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.806 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712887+0000", 2024-09-14T06:17:46.806 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3119999999999998e-06 2024-09-14T06:17:46.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.806 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.806 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.806 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712890+0000", 2024-09-14T06:17:46.806 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7e-06 2024-09-14T06:17:46.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.807 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.807 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.839114+0000", 2024-09-14T06:17:46.807 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126223954 2024-09-14T06:17:46.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.807 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.807 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.839197+0000", 2024-09-14T06:17:46.807 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3250000000000004e-05 2024-09-14T06:17:46.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.808 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.808 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840073+0000", 2024-09-14T06:17:46.808 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000875715 2024-09-14T06:17:46.808 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.808 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.808 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.808 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.808 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40409 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.808 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:46.808 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.062391441000003, 2024-09-14T06:17:46.809 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12739108900000001, 2024-09-14T06:17:46.809 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.809 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.809 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.809 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.809 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.809 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.810 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40409 2024-09-14T06:17:46.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.810 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.810 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.810 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.810 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:46.810 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.811 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.811 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:46.811 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.811 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.811 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:46.811 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.65e-07 2024-09-14T06:17:46.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.812 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.812 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.812 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712895+0000", 2024-09-14T06:17:46.812 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.325e-06 2024-09-14T06:17:46.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.813 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.813 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.813 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712899+0000", 2024-09-14T06:17:46.813 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.219e-06 2024-09-14T06:17:46.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.814 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.814 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712901+0000", 2024-09-14T06:17:46.814 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7310000000000002e-06 2024-09-14T06:17:46.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.814 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.814 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.839921+0000", 2024-09-14T06:17:46.814 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12701995199999999 2024-09-14T06:17:46.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.815 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.815 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840006+0000", 2024-09-14T06:17:46.815 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5063999999999997e-05 2024-09-14T06:17:46.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.815 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.816 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840285+0000", 2024-09-14T06:17:46.816 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027853300000000002 2024-09-14T06:17:46.816 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.816 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.816 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.817 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.817 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40411 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.817 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712909+0000", 2024-09-14T06:17:46.817 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.062376069999999, 2024-09-14T06:17:46.817 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.128251319, 2024-09-14T06:17:46.818 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.818 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.818 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.818 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.818 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.818 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.818 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40411 2024-09-14T06:17:46.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.818 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.818 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.818 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.818 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712909+0000", 2024-09-14T06:17:46.819 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.819 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.819 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.819 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712909+0000", 2024-09-14T06:17:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712910+0000", 2024-09-14T06:17:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.39e-07 2024-09-14T06:17:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712911+0000", 2024-09-14T06:17:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.37e-06 2024-09-14T06:17:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.821 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.821 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712914+0000", 2024-09-14T06:17:46.821 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4999999999999999e-06 2024-09-14T06:17:46.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.821 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.821 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.821 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712929+0000", 2024-09-14T06:17:46.821 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4406000000000001e-05 2024-09-14T06:17:46.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.821 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.821 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.821 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840856+0000", 2024-09-14T06:17:46.821 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.127927546 2024-09-14T06:17:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840942+0000", 2024-09-14T06:17:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5232e-05 2024-09-14T06:17:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841161+0000", 2024-09-14T06:17:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021892600000000001 2024-09-14T06:17:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40413 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712938+0000", 2024-09-14T06:17:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.062347144, 2024-09-14T06:17:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.128612052, 2024-09-14T06:17:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40413 2024-09-14T06:17:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.824 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712938+0000", 2024-09-14T06:17:46.824 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.824 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.824 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712938+0000", 2024-09-14T06:17:46.824 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.824 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.824 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712939+0000", 2024-09-14T06:17:46.824 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1099999999999996e-07 2024-09-14T06:17:46.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.825 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.825 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712942+0000", 2024-09-14T06:17:46.825 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0920000000000002e-06 2024-09-14T06:17:46.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.825 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.825 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712949+0000", 2024-09-14T06:17:46.825 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.075e-06 2024-09-14T06:17:46.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.825 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.825 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.825 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712954+0000", 2024-09-14T06:17:46.826 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2229999999999998e-06 2024-09-14T06:17:46.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.826 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.826 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.826 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841192+0000", 2024-09-14T06:17:46.826 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12823817100000001 2024-09-14T06:17:46.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.826 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.826 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.826 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841280+0000", 2024-09-14T06:17:46.826 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8190999999999996e-05 2024-09-14T06:17:46.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.826 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.826 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.827 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841550+0000", 2024-09-14T06:17:46.827 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00026978899999999999 2024-09-14T06:17:46.827 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.827 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.827 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.827 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40415 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.827 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712962+0000", 2024-09-14T06:17:46.827 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.062322866999999, 2024-09-14T06:17:46.827 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12902585499999999, 2024-09-14T06:17:46.827 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.827 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.827 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.828 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.828 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.828 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.828 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40415 2024-09-14T06:17:46.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.828 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.828 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.828 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.828 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712962+0000", 2024-09-14T06:17:46.828 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.828 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.828 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.828 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712962+0000", 2024-09-14T06:17:46.829 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.829 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.829 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712963+0000", 2024-09-14T06:17:46.829 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:17:46.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.829 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.829 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712966+0000", 2024-09-14T06:17:46.829 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6649999999999999e-06 2024-09-14T06:17:46.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.829 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.830 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.830 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712972+0000", 2024-09-14T06:17:46.830 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4819999999999999e-06 2024-09-14T06:17:46.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.830 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.830 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712978+0000", 2024-09-14T06:17:46.830 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4299999999999997e-06 2024-09-14T06:17:46.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.830 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.830 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841581+0000", 2024-09-14T06:17:46.830 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12860382100000001 2024-09-14T06:17:46.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.830 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.831 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.831 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841665+0000", 2024-09-14T06:17:46.831 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3228999999999999e-05 2024-09-14T06:17:46.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.831 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.831 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.831 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841988+0000", 2024-09-14T06:17:46.831 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032370599999999999 2024-09-14T06:17:46.831 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.831 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.831 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.831 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.831 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40417 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.832 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712983+0000", 2024-09-14T06:17:46.832 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.062302133000003, 2024-09-14T06:17:46.832 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129525837, 2024-09-14T06:17:46.832 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.832 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.832 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.832 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.832 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.832 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.832 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40417 2024-09-14T06:17:46.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.833 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.833 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.833 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712983+0000", 2024-09-14T06:17:46.833 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.833 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.833 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712983+0000", 2024-09-14T06:17:46.833 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.834 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.834 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713182+0000", 2024-09-14T06:17:46.834 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000198569 2024-09-14T06:17:46.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.834 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.834 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713186+0000", 2024-09-14T06:17:46.834 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6160000000000004e-06 2024-09-14T06:17:46.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.834 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.834 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.834 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713197+0000", 2024-09-14T06:17:46.834 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.046e-05 2024-09-14T06:17:46.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.835 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.835 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713204+0000", 2024-09-14T06:17:46.835 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.515e-06 2024-09-14T06:17:46.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.835 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.835 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842021+0000", 2024-09-14T06:17:46.835 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12881625199999999 2024-09-14T06:17:46.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.835 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.836 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842158+0000", 2024-09-14T06:17:46.836 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013778099999999999 2024-09-14T06:17:46.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.836 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.836 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.836 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842509+0000", 2024-09-14T06:17:46.836 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000350644 2024-09-14T06:17:46.836 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.836 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.836 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.837 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.837 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40419 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.837 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.713217+0000", 2024-09-14T06:17:46.837 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.062068596000003, 2024-09-14T06:17:46.837 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12982854599999999, 2024-09-14T06:17:46.837 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.837 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.837 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.837 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.837 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.837 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.838 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40419 2024-09-14T06:17:46.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.838 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.838 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.838 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.838 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713217+0000", 2024-09-14T06:17:46.838 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.838 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.838 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.838 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713217+0000", 2024-09-14T06:17:46.838 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.839 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.839 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.839 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713831+0000", 2024-09-14T06:17:46.839 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061477700000000005 2024-09-14T06:17:46.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.839 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.839 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.839 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713841+0000", 2024-09-14T06:17:46.839 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3789999999999995e-06 2024-09-14T06:17:46.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.839 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.839 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.839 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713853+0000", 2024-09-14T06:17:46.839 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1952e-05 2024-09-14T06:17:46.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.840 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.840 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.840 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713861+0000", 2024-09-14T06:17:46.840 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4500000000000004e-06 2024-09-14T06:17:46.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.840 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.840 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.840 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842560+0000", 2024-09-14T06:17:46.840 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12869836900000001 2024-09-14T06:17:46.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.840 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.840 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.840 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842655+0000", 2024-09-14T06:17:46.840 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5617000000000003e-05 2024-09-14T06:17:46.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.841 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.841 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.841 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843045+0000", 2024-09-14T06:17:46.841 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00039000200000000002 2024-09-14T06:17:46.841 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.841 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.841 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.841 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.841 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40421 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.841 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.713882+0000", 2024-09-14T06:17:46.842 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.061403229, 2024-09-14T06:17:46.842 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12974492600000001, 2024-09-14T06:17:46.844 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.844 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.844 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.844 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.844 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.844 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.844 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40421 2024-09-14T06:17:46.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.845 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.845 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.845 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.845 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713882+0000", 2024-09-14T06:17:46.845 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.845 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.845 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.845 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713882+0000", 2024-09-14T06:17:46.845 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.845 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.845 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.846 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714044+0000", 2024-09-14T06:17:46.846 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016224999999999999 2024-09-14T06:17:46.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.846 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.846 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.846 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714049+0000", 2024-09-14T06:17:46.846 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0150000000000003e-06 2024-09-14T06:17:46.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.846 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.846 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.846 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714068+0000", 2024-09-14T06:17:46.846 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8749e-05 2024-09-14T06:17:46.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.846 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.847 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.847 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714077+0000", 2024-09-14T06:17:46.847 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4379999999999995e-06 2024-09-14T06:17:46.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.847 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.847 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.847 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843079+0000", 2024-09-14T06:17:46.847 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129002958 2024-09-14T06:17:46.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.847 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.848 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.848 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843290+0000", 2024-09-14T06:17:46.848 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021005899999999999 2024-09-14T06:17:46.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.848 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.848 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.848 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843627+0000", 2024-09-14T06:17:46.848 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00033745699999999999 2024-09-14T06:17:46.848 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.848 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.848 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.848 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.849 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40423 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.849 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.714090+0000", 2024-09-14T06:17:46.849 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.061194858, 2024-09-14T06:17:46.849 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129886166, 2024-09-14T06:17:46.849 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.849 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.849 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.849 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.849 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.849 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.850 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40423 2024-09-14T06:17:46.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.850 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.850 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.850 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.850 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714090+0000", 2024-09-14T06:17:46.850 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.850 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.850 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.850 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714090+0000", 2024-09-14T06:17:46.850 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.850 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.851 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.851 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714456+0000", 2024-09-14T06:17:46.851 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000366058 2024-09-14T06:17:46.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.851 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.851 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.851 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714461+0000", 2024-09-14T06:17:46.851 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.228e-06 2024-09-14T06:17:46.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.851 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.851 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.851 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714469+0000", 2024-09-14T06:17:46.851 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6589999999999996e-06 2024-09-14T06:17:46.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.852 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.852 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.852 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714476+0000", 2024-09-14T06:17:46.852 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:17:46.852 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.852 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.852 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.852 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843659+0000", 2024-09-14T06:17:46.852 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129183137 2024-09-14T06:17:46.852 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.852 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.852 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.852 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843745+0000", 2024-09-14T06:17:46.853 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5354999999999998e-05 2024-09-14T06:17:46.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.853 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.853 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.853 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843977+0000", 2024-09-14T06:17:46.853 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000231883 2024-09-14T06:17:46.853 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.853 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.853 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.853 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.853 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40425 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.853 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.714490+0000", 2024-09-14T06:17:46.853 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.060795663, 2024-09-14T06:17:46.854 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13064684600000001, 2024-09-14T06:17:46.854 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.854 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.854 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.854 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.854 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.854 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.854 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40425 2024-09-14T06:17:46.854 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.854 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.854 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.854 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.854 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714490+0000", 2024-09-14T06:17:46.854 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.855 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.855 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.855 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714490+0000", 2024-09-14T06:17:46.855 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.855 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.855 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.855 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714725+0000", 2024-09-14T06:17:46.855 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000235438 2024-09-14T06:17:46.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.855 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.855 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.855 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714729+0000", 2024-09-14T06:17:46.856 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8909999999999997e-06 2024-09-14T06:17:46.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.856 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.856 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.856 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714737+0000", 2024-09-14T06:17:46.856 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5329999999999994e-06 2024-09-14T06:17:46.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.856 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.856 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.856 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714742+0000", 2024-09-14T06:17:46.856 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9559999999999996e-06 2024-09-14T06:17:46.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.856 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.856 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.857 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.844184+0000", 2024-09-14T06:17:46.857 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129441954 2024-09-14T06:17:46.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.857 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.857 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.857 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.844272+0000", 2024-09-14T06:17:46.857 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7151999999999995e-05 2024-09-14T06:17:46.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.857 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.857 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.857 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845136+0000", 2024-09-14T06:17:46.858 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00086492199999999996 2024-09-14T06:17:46.858 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.858 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.858 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.858 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.858 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40426 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.858 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.714753+0000", 2024-09-14T06:17:46.858 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.060532698999999, 2024-09-14T06:17:46.858 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.130693374, 2024-09-14T06:17:46.858 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.858 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40426 2024-09-14T06:17:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714753+0000", 2024-09-14T06:17:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.859 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714753+0000", 2024-09-14T06:17:46.860 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.860 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.860 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.860 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715118+0000", 2024-09-14T06:17:46.860 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036512099999999999 2024-09-14T06:17:46.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.860 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.860 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.860 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715123+0000", 2024-09-14T06:17:46.860 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9910000000000002e-06 2024-09-14T06:17:46.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.860 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.860 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.861 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715141+0000", 2024-09-14T06:17:46.861 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8219999999999998e-05 2024-09-14T06:17:46.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.861 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.861 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.861 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715149+0000", 2024-09-14T06:17:46.861 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8979999999999996e-06 2024-09-14T06:17:46.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.861 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.861 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.861 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845170+0000", 2024-09-14T06:17:46.861 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13002106799999999 2024-09-14T06:17:46.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.861 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.862 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.862 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845249+0000", 2024-09-14T06:17:46.862 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9163000000000001e-05 2024-09-14T06:17:46.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.862 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.862 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.862 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845446+0000", 2024-09-14T06:17:46.862 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196913 2024-09-14T06:17:46.862 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.862 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.862 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.862 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.862 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40428 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.863 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.715166+0000", 2024-09-14T06:17:46.863 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.060119747999998, 2024-09-14T06:17:46.863 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13071775199999999, 2024-09-14T06:17:46.863 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.863 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.863 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:46.863 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.863 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.863 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.863 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40428 2024-09-14T06:17:46.863 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.864 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.864 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.864 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.864 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715166+0000", 2024-09-14T06:17:46.864 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.864 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.865 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.865 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.865 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715166+0000", 2024-09-14T06:17:46.865 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.865 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.865 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.865 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715361+0000", 2024-09-14T06:17:46.865 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00019536400000000001 2024-09-14T06:17:46.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.865 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.865 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.866 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715366+0000", 2024-09-14T06:17:46.866 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4589999999999997e-06 2024-09-14T06:17:46.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.866 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.866 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.866 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715387+0000", 2024-09-14T06:17:46.866 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0829e-05 2024-09-14T06:17:46.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.866 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.866 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.866 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715402+0000", 2024-09-14T06:17:46.866 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4429e-05 2024-09-14T06:17:46.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.867 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.867 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.867 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845476+0000", 2024-09-14T06:17:46.867 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13007447999999999 2024-09-14T06:17:46.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.868 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.868 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:46.868 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845557+0000", 2024-09-14T06:17:46.868 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1100000000000006e-05 2024-09-14T06:17:46.868 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.869 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.869 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.869 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845883+0000", 2024-09-14T06:17:46.869 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032609099999999998 2024-09-14T06:17:46.869 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.869 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.870 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.870 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.870 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.870 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40430 3.15 3.aecb2ff5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.870 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.715416+0000", 2024-09-14T06:17:46.870 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.059869282999998, 2024-09-14T06:17:46.870 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13113939699999999, 2024-09-14T06:17:46.870 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.870 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.871 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:46.871 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.871 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.871 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.871 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40430 2024-09-14T06:17:46.871 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.872 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.872 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.872 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.872 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715416+0000", 2024-09-14T06:17:46.873 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.873 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.873 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.873 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715416+0000", 2024-09-14T06:17:46.873 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.874 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.874 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.874 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.874 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715995+0000", 2024-09-14T06:17:46.874 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057881100000000004 2024-09-14T06:17:46.874 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.875 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.875 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.875 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716000+0000", 2024-09-14T06:17:46.875 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.677e-06 2024-09-14T06:17:46.875 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.875 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.876 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.876 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716013+0000", 2024-09-14T06:17:46.876 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2326e-05 2024-09-14T06:17:46.876 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.876 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.877 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.877 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716021+0000", 2024-09-14T06:17:46.877 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9750000000000007e-06 2024-09-14T06:17:46.877 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.877 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.877 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.878 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846538+0000", 2024-09-14T06:17:46.878 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13051717400000001 2024-09-14T06:17:46.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.878 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.878 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.878 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846555+0000", 2024-09-14T06:17:46.879 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7433999999999999e-05 2024-09-14T06:17:46.879 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.879 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.879 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.879 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.879 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.880 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40432 3.15 3.aecb2ff5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.880 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:46.880 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.059249643999998, 2024-09-14T06:17:46.880 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13055824499999999, 2024-09-14T06:17:46.880 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.881 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.881 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:46.881 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.881 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.881 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.882 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40432 2024-09-14T06:17:46.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.882 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.882 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.882 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.882 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:46.883 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.883 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.883 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.883 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:46.884 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.884 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.884 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.884 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:46.884 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9599999999999998e-07 2024-09-14T06:17:46.885 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.885 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.885 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.885 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716040+0000", 2024-09-14T06:17:46.885 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.399e-06 2024-09-14T06:17:46.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.886 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.886 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.886 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716048+0000", 2024-09-14T06:17:46.886 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2179999999999999e-06 2024-09-14T06:17:46.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.887 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.887 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.887 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716054+0000", 2024-09-14T06:17:46.887 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2110000000000003e-06 2024-09-14T06:17:46.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.887 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.888 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.888 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846586+0000", 2024-09-14T06:17:46.888 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.130531921 2024-09-14T06:17:46.888 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.888 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.888 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.889 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846594+0000", 2024-09-14T06:17:46.889 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7000000000000008e-06 2024-09-14T06:17:46.889 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.889 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.889 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.890 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.890 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.890 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40434 3.15 3.aecb2ff5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:46.890 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.716065+0000", 2024-09-14T06:17:46.890 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.059219816999999, 2024-09-14T06:17:46.890 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13055924899999999, 2024-09-14T06:17:46.891 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:46.891 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:46.891 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:46.891 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:46.891 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:46.892 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:46.892 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40434 2024-09-14T06:17:46.892 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.892 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:46.892 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.892 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:46.893 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716065+0000", 2024-09-14T06:17:46.893 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.893 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.893 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.893 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:46.894 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716065+0000", 2024-09-14T06:17:46.894 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:46.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.894 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.894 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:46.894 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.723984+0000", 2024-09-14T06:17:46.895 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0079189670000000007 2024-09-14T06:17:46.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.895 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.895 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:46.895 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.723991+0000", 2024-09-14T06:17:46.896 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1759999999999998e-06 2024-09-14T06:17:46.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.896 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.896 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:46.896 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.724003+0000", 2024-09-14T06:17:46.896 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1914e-05 2024-09-14T06:17:46.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.897 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.897 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:46.897 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.724013+0000", 2024-09-14T06:17:46.897 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441999999999999e-05 2024-09-14T06:17:46.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.897 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.898 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:46.898 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846617+0000", 2024-09-14T06:17:46.898 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12260452199999999 2024-09-14T06:17:46.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:46.898 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:46.898 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:46.898 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846625+0000", 2024-09-14T06:17:46.898 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2280000000000001e-06 2024-09-14T06:17:46.898 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.898 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.899 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.899 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:46.899 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:46.899 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:46.900 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:17:46.901 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:46.900+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:47.001 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:46.996+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:47.037 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:47.038 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:47.038 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:47.038 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:47.055 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:17:47.102 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:47.097+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:47.197 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:47.197 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:47.197 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:47.197 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:47.197 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:47.203 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:47.200+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:47.213 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:17:47.305 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:47.300+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:47.388 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:47.388 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:47.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:47.388 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:47.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.388 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:17:47.388 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:17:47.388 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 103.30793009200001, 2024-09-14T06:17:47.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.56009052699999995, 2024-09-14T06:17:47.389 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.389 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.389 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:47.389 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.389 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:17:47.389 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:17:47.389 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 1 2024-09-14T06:17:47.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.389 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:17:47.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:17:47.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077065+0000", 2024-09-14T06:17:47.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6429999999999999e-06 2024-09-14T06:17:47.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.391 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.391 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077079+0000", 2024-09-14T06:17:47.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4447000000000001e-05 2024-09-14T06:17:47.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.391 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.391 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077103+0000", 2024-09-14T06:17:47.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3856e-05 2024-09-14T06:17:47.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.391 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.391 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077120+0000", 2024-09-14T06:17:47.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7337e-05 2024-09-14T06:17:47.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077255+0000", 2024-09-14T06:17:47.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000134652 2024-09-14T06:17:47.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:47.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077274+0000", 2024-09-14T06:17:47.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9003000000000001e-05 2024-09-14T06:17:47.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634272+0000", 2024-09-14T06:17:47.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.55699813099999995 2024-09-14T06:17:47.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634591+0000", 2024-09-14T06:17:47.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031910800000000002 2024-09-14T06:17:47.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:17:47.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634892+0000", 2024-09-14T06:17:47.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00030066400000000002 2024-09-14T06:17:47.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:47.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636345+0000", 2024-09-14T06:17:47.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001452896 2024-09-14T06:17:47.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:47.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636974+0000", 2024-09-14T06:17:47.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00062927800000000004 2024-09-14T06:17:47.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:47.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637071+0000", 2024-09-14T06:17:47.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7178999999999999e-05 2024-09-14T06:17:47.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637153+0000", 2024-09-14T06:17:47.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1333000000000002e-05 2024-09-14T06:17:47.395 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.395 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.395 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.396 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38947 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 159744~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.396 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:17:47.396 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704871805000003, 2024-09-14T06:17:47.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085496275999999996, 2024-09-14T06:17:47.396 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.396 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.396 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.396 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.396 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.396 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.396 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38947 2024-09-14T06:17:47.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.396 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:17:47.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:17:47.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680121+0000", 2024-09-14T06:17:47.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.82e-07 2024-09-14T06:17:47.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680123+0000", 2024-09-14T06:17:47.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5629999999999999e-06 2024-09-14T06:17:47.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680133+0000", 2024-09-14T06:17:47.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0890000000000006e-06 2024-09-14T06:17:47.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680138+0000", 2024-09-14T06:17:47.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.728e-06 2024-09-14T06:17:47.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765422+0000", 2024-09-14T06:17:47.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085283701000000003 2024-09-14T06:17:47.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765496+0000", 2024-09-14T06:17:47.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4276000000000005e-05 2024-09-14T06:17:47.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.400 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.400 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765617+0000", 2024-09-14T06:17:47.400 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000120437 2024-09-14T06:17:47.400 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.400 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.400 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.400 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38990 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 163840~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.400 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:17:47.400 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704845822000003, 2024-09-14T06:17:47.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085695970999999996, 2024-09-14T06:17:47.401 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.401 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.401 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.401 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.401 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.401 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.401 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38990 2024-09-14T06:17:47.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.402 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:17:47.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:17:47.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680147+0000", 2024-09-14T06:17:47.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7599999999999997e-07 2024-09-14T06:17:47.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680149+0000", 2024-09-14T06:17:47.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4700000000000001e-06 2024-09-14T06:17:47.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680156+0000", 2024-09-14T06:17:47.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.058e-06 2024-09-14T06:17:47.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680162+0000", 2024-09-14T06:17:47.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7080000000000002e-06 2024-09-14T06:17:47.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765657+0000", 2024-09-14T06:17:47.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085494566999999994 2024-09-14T06:17:47.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765741+0000", 2024-09-14T06:17:47.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4782000000000002e-05 2024-09-14T06:17:47.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765842+0000", 2024-09-14T06:17:47.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010090999999999999 2024-09-14T06:17:47.405 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.405 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.405 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.405 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38991 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 172032~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.405 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:17:47.405 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704821865, 2024-09-14T06:17:47.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085875878000000003, 2024-09-14T06:17:47.405 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.405 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.405 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.406 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.406 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.406 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.406 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38991 2024-09-14T06:17:47.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.406 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:17:47.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:17:47.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680171+0000", 2024-09-14T06:17:47.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4299999999999998e-07 2024-09-14T06:17:47.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680173+0000", 2024-09-14T06:17:47.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1639999999999999e-06 2024-09-14T06:17:47.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680181+0000", 2024-09-14T06:17:47.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0539999999999995e-06 2024-09-14T06:17:47.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680187+0000", 2024-09-14T06:17:47.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7529999999999998e-06 2024-09-14T06:17:47.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765875+0000", 2024-09-14T06:17:47.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085688722999999994 2024-09-14T06:17:47.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765948+0000", 2024-09-14T06:17:47.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2126999999999996e-05 2024-09-14T06:17:47.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766046+0000", 2024-09-14T06:17:47.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8614000000000001e-05 2024-09-14T06:17:47.409 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.409 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.409 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.409 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38992 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 180224~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.409 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:47.409 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704796588999997, 2024-09-14T06:17:47.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086050225999999994, 2024-09-14T06:17:47.410 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.410 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.410 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.410 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.410 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.410 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.410 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38992 2024-09-14T06:17:47.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.410 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:47.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:47.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:47.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1199999999999998e-07 2024-09-14T06:17:47.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680199+0000", 2024-09-14T06:17:47.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:17:47.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680227+0000", 2024-09-14T06:17:47.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8238999999999998e-05 2024-09-14T06:17:47.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680234+0000", 2024-09-14T06:17:47.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0550000000000002e-06 2024-09-14T06:17:47.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766077+0000", 2024-09-14T06:17:47.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085843002000000002 2024-09-14T06:17:47.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.413 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.413 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766155+0000", 2024-09-14T06:17:47.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8112e-05 2024-09-14T06:17:47.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.413 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.413 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766246+0000", 2024-09-14T06:17:47.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0785999999999994e-05 2024-09-14T06:17:47.413 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.413 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.413 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38993 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 188416~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:17:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704705238000003, 2024-09-14T06:17:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086166131000000007, 2024-09-14T06:17:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38993 2024-09-14T06:17:47.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.415 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:17:47.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:17:47.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680288+0000", 2024-09-14T06:17:47.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7199999999999998e-07 2024-09-14T06:17:47.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680291+0000", 2024-09-14T06:17:47.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7280000000000001e-06 2024-09-14T06:17:47.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680301+0000", 2024-09-14T06:17:47.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3360000000000001e-06 2024-09-14T06:17:47.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680308+0000", 2024-09-14T06:17:47.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1389999999999998e-06 2024-09-14T06:17:47.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766280+0000", 2024-09-14T06:17:47.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085972111000000004 2024-09-14T06:17:47.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766357+0000", 2024-09-14T06:17:47.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7477000000000004e-05 2024-09-14T06:17:47.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766453+0000", 2024-09-14T06:17:47.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5668000000000006e-05 2024-09-14T06:17:47.418 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.418 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.418 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.418 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38994 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 196608~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.418 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:47.418 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704615431000001, 2024-09-14T06:17:47.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086290477000000004, 2024-09-14T06:17:47.418 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.419 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.419 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.419 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.419 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.419 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.419 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38994 2024-09-14T06:17:47.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.419 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:47.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:47.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:47.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3e-07 2024-09-14T06:17:47.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680381+0000", 2024-09-14T06:17:47.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.439e-06 2024-09-14T06:17:47.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680438+0000", 2024-09-14T06:17:47.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6693000000000001e-05 2024-09-14T06:17:47.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680446+0000", 2024-09-14T06:17:47.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9160000000000001e-06 2024-09-14T06:17:47.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766490+0000", 2024-09-14T06:17:47.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086044617000000004 2024-09-14T06:17:47.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766563+0000", 2024-09-14T06:17:47.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2430999999999998e-05 2024-09-14T06:17:47.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766667+0000", 2024-09-14T06:17:47.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000104651 2024-09-14T06:17:47.422 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.422 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.422 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.422 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38995 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 204800~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.423 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:17:47.423 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704532405000002, 2024-09-14T06:17:47.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086415360999999996, 2024-09-14T06:17:47.423 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.423 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.423 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.423 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.423 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.423 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.423 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38995 2024-09-14T06:17:47.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.423 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:17:47.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:17:47.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680461+0000", 2024-09-14T06:17:47.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:17:47.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.425 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680464+0000", 2024-09-14T06:17:47.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4929999999999998e-06 2024-09-14T06:17:47.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.425 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680479+0000", 2024-09-14T06:17:47.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5322999999999999e-05 2024-09-14T06:17:47.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.425 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680496+0000", 2024-09-14T06:17:47.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6801000000000001e-05 2024-09-14T06:17:47.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.426 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766699+0000", 2024-09-14T06:17:47.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086203265000000001 2024-09-14T06:17:47.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.426 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766777+0000", 2024-09-14T06:17:47.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7284000000000001e-05 2024-09-14T06:17:47.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.426 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766875+0000", 2024-09-14T06:17:47.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8461000000000005e-05 2024-09-14T06:17:47.426 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.427 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.427 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.427 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38996 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 212992~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.427 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:17:47.427 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704484966000003, 2024-09-14T06:17:47.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086576474, 2024-09-14T06:17:47.427 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.427 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.427 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.427 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.428 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.428 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.428 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38996 2024-09-14T06:17:47.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.428 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:17:47.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:17:47.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680508+0000", 2024-09-14T06:17:47.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1799999999999995e-07 2024-09-14T06:17:47.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680511+0000", 2024-09-14T06:17:47.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9739999999999998e-06 2024-09-14T06:17:47.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.430 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.430 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680520+0000", 2024-09-14T06:17:47.430 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.329e-06 2024-09-14T06:17:47.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.430 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.430 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.430 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680528+0000", 2024-09-14T06:17:47.430 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4660000000000002e-06 2024-09-14T06:17:47.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.430 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.430 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.430 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766907+0000", 2024-09-14T06:17:47.431 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086379529999999996 2024-09-14T06:17:47.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.431 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.431 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766987+0000", 2024-09-14T06:17:47.431 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0280999999999999e-05 2024-09-14T06:17:47.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.431 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.431 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767084+0000", 2024-09-14T06:17:47.431 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6275999999999998e-05 2024-09-14T06:17:47.431 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.431 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.432 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.432 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38997 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 221184~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.432 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:47.432 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704455625000001, 2024-09-14T06:17:47.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086757327999999995, 2024-09-14T06:17:47.432 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.432 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.432 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.432 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.432 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.432 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.432 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38997 2024-09-14T06:17:47.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.433 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:47.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:47.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:47.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:17:47.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.434 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.434 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680540+0000", 2024-09-14T06:17:47.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6910000000000002e-06 2024-09-14T06:17:47.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.434 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.434 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680548+0000", 2024-09-14T06:17:47.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7339999999999997e-06 2024-09-14T06:17:47.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.434 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.434 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680555+0000", 2024-09-14T06:17:47.435 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0900000000000001e-06 2024-09-14T06:17:47.435 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.435 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.435 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.435 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767116+0000", 2024-09-14T06:17:47.435 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086561551 2024-09-14T06:17:47.435 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.435 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.435 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.435 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767192+0000", 2024-09-14T06:17:47.435 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5504000000000001e-05 2024-09-14T06:17:47.435 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.435 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.435 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.435 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767294+0000", 2024-09-14T06:17:47.436 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000102308 2024-09-14T06:17:47.436 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.436 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.436 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.436 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38998 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 229376~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.436 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:17:47.436 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704429013999999, 2024-09-14T06:17:47.436 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087010193999999999, 2024-09-14T06:17:47.436 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.436 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.436 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.436 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.437 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.437 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.437 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38998 2024-09-14T06:17:47.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.437 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.437 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.437 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.437 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:17:47.437 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.437 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.437 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.438 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:17:47.438 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.438 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.438 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680564+0000", 2024-09-14T06:17:47.438 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1800000000000001e-07 2024-09-14T06:17:47.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.438 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.438 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680566+0000", 2024-09-14T06:17:47.438 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.503e-06 2024-09-14T06:17:47.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.439 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.439 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680574+0000", 2024-09-14T06:17:47.439 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3979999999999999e-06 2024-09-14T06:17:47.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.439 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.439 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.439 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680580+0000", 2024-09-14T06:17:47.439 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4859999999999997e-06 2024-09-14T06:17:47.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.439 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.439 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.439 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767359+0000", 2024-09-14T06:17:47.439 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086779447999999995 2024-09-14T06:17:47.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.440 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.440 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.440 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767445+0000", 2024-09-14T06:17:47.440 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6007000000000005e-05 2024-09-14T06:17:47.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.440 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.440 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.440 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767573+0000", 2024-09-14T06:17:47.440 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012793400000000001 2024-09-14T06:17:47.440 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.440 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.441 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.441 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.441 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.441 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38999 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 237568~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.441 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:17:47.441 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704400149999998, 2024-09-14T06:17:47.441 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087211699000000004, 2024-09-14T06:17:47.441 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.441 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.441 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.441 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.441 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.441 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.442 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38999 2024-09-14T06:17:47.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.442 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.442 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.442 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.442 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:17:47.442 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.442 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.442 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.442 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:17:47.442 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.443 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.443 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.443 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680593+0000", 2024-09-14T06:17:47.443 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7700000000000005e-07 2024-09-14T06:17:47.443 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.443 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.443 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.443 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680600+0000", 2024-09-14T06:17:47.443 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7130000000000005e-06 2024-09-14T06:17:47.443 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.443 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.443 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.443 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680610+0000", 2024-09-14T06:17:47.444 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.55e-06 2024-09-14T06:17:47.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.444 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.444 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.444 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680616+0000", 2024-09-14T06:17:47.444 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3069999999999999e-06 2024-09-14T06:17:47.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.444 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.444 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.444 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767607+0000", 2024-09-14T06:17:47.444 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086990703000000003 2024-09-14T06:17:47.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.444 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.445 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.445 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767682+0000", 2024-09-14T06:17:47.445 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4944999999999995e-05 2024-09-14T06:17:47.445 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.445 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.445 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.445 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767804+0000", 2024-09-14T06:17:47.445 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012200400000000001 2024-09-14T06:17:47.445 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.445 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.445 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.445 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.445 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.446 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39000 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 245760~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.446 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:47.446 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704366528000001, 2024-09-14T06:17:47.446 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087408649000000005, 2024-09-14T06:17:47.446 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.446 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.446 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.446 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.446 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.446 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.446 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39000 2024-09-14T06:17:47.446 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.446 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.447 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.447 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.447 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:47.447 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.447 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.447 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.447 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:47.447 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.447 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.447 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.447 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:47.448 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1699999999999998e-07 2024-09-14T06:17:47.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.448 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.448 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.448 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680629+0000", 2024-09-14T06:17:47.448 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5469999999999999e-06 2024-09-14T06:17:47.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.448 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.448 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.448 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680636+0000", 2024-09-14T06:17:47.448 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.973e-06 2024-09-14T06:17:47.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.448 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.449 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.449 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680642+0000", 2024-09-14T06:17:47.449 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0349999999999998e-06 2024-09-14T06:17:47.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.449 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.449 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.449 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767839+0000", 2024-09-14T06:17:47.449 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087197067000000003 2024-09-14T06:17:47.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.449 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.449 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.449 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767913+0000", 2024-09-14T06:17:47.450 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3832999999999996e-05 2024-09-14T06:17:47.450 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.450 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.450 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.450 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768034+0000", 2024-09-14T06:17:47.450 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012167699999999999 2024-09-14T06:17:47.450 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.450 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.450 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.450 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.450 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.450 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39001 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 253952~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.450 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:17:47.450 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704342085, 2024-09-14T06:17:47.451 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087623312999999994, 2024-09-14T06:17:47.451 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.451 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.451 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.451 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.451 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.451 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.451 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39001 2024-09-14T06:17:47.451 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.451 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.451 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.451 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.451 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:17:47.451 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.451 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.452 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.452 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.452 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:17:47.452 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.452 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.452 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.452 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.452 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680651+0000", 2024-09-14T06:17:47.452 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7700000000000004e-07 2024-09-14T06:17:47.452 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.452 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.452 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.452 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680653+0000", 2024-09-14T06:17:47.453 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5739999999999998e-06 2024-09-14T06:17:47.453 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.453 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.453 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.453 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680702+0000", 2024-09-14T06:17:47.453 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8476000000000001e-05 2024-09-14T06:17:47.453 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.453 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.453 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.453 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680709+0000", 2024-09-14T06:17:47.453 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8170000000000001e-06 2024-09-14T06:17:47.453 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.453 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.453 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.454 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768066+0000", 2024-09-14T06:17:47.454 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087357852999999999 2024-09-14T06:17:47.454 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.454 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.454 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.454 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768163+0000", 2024-09-14T06:17:47.454 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6869999999999999e-05 2024-09-14T06:17:47.454 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.454 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.454 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.454 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768273+0000", 2024-09-14T06:17:47.454 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000110146 2024-09-14T06:17:47.454 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.455 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.455 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.455 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.455 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.455 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39002 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 262144~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.455 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:17:47.455 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704261979000002, 2024-09-14T06:17:47.455 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088150478000000004, 2024-09-14T06:17:47.455 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.455 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.455 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.455 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.455 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.456 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.456 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39002 2024-09-14T06:17:47.456 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.456 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.456 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.456 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.456 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:17:47.456 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.456 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.456 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.456 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.456 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:17:47.457 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.457 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.457 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.457 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680739+0000", 2024-09-14T06:17:47.457 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3979999999999999e-06 2024-09-14T06:17:47.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.457 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.457 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.457 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680742+0000", 2024-09-14T06:17:47.457 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8349999999999997e-06 2024-09-14T06:17:47.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.458 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.458 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.458 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680751+0000", 2024-09-14T06:17:47.458 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8599999999999999e-06 2024-09-14T06:17:47.458 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.460 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.460 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.460 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680758+0000", 2024-09-14T06:17:47.461 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4790000000000001e-06 2024-09-14T06:17:47.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.461 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.461 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.461 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768307+0000", 2024-09-14T06:17:47.461 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087549342000000002 2024-09-14T06:17:47.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.461 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.461 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.461 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768392+0000", 2024-09-14T06:17:47.461 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.441e-05 2024-09-14T06:17:47.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.461 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768881+0000", 2024-09-14T06:17:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000489154 2024-09-14T06:17:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39003 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 270336~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704209689000002, 2024-09-14T06:17:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088375555999999994, 2024-09-14T06:17:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39003 2024-09-14T06:17:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.464 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.464 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:47.464 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.464 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.464 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.464 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.464 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:47.464 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7000000000000004e-07 2024-09-14T06:17:47.464 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.464 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.464 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.464 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680786+0000", 2024-09-14T06:17:47.464 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:17:47.465 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.465 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.465 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.465 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680794+0000", 2024-09-14T06:17:47.465 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9840000000000001e-06 2024-09-14T06:17:47.465 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.465 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.465 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.465 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680800+0000", 2024-09-14T06:17:47.465 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1550000000000004e-06 2024-09-14T06:17:47.465 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.465 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.465 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.466 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768965+0000", 2024-09-14T06:17:47.466 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088165091000000001 2024-09-14T06:17:47.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.466 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.466 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.466 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769059+0000", 2024-09-14T06:17:47.466 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3923000000000002e-05 2024-09-14T06:17:47.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.466 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.466 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.466 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769158+0000", 2024-09-14T06:17:47.466 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8820000000000006e-05 2024-09-14T06:17:47.466 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.466 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.467 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.467 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.467 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.467 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39004 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 278528~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.467 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:17:47.467 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704182001, 2024-09-14T06:17:47.467 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088499624999999998, 2024-09-14T06:17:47.467 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.467 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.467 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.467 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.467 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.467 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.467 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39004 2024-09-14T06:17:47.468 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.468 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.468 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.468 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.468 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:17:47.468 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.468 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.468 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.468 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.468 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:17:47.468 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.468 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.468 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.468 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.469 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680811+0000", 2024-09-14T06:17:47.469 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:17:47.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.469 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.469 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.469 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680814+0000", 2024-09-14T06:17:47.469 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:17:47.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.469 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.469 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.469 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680821+0000", 2024-09-14T06:17:47.469 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9009999999999997e-06 2024-09-14T06:17:47.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.470 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.470 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.470 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680826+0000", 2024-09-14T06:17:47.470 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5350000000000004e-06 2024-09-14T06:17:47.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.470 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.470 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.470 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769189+0000", 2024-09-14T06:17:47.470 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088363024999999998 2024-09-14T06:17:47.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.470 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.470 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.471 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769246+0000", 2024-09-14T06:17:47.471 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7017e-05 2024-09-14T06:17:47.471 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.471 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.471 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.471 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769310+0000", 2024-09-14T06:17:47.471 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3541999999999996e-05 2024-09-14T06:17:47.471 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.471 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.471 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.471 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.471 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.471 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39005 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 286720~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.471 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:17:47.472 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704157917000003, 2024-09-14T06:17:47.472 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088681092000000003, 2024-09-14T06:17:47.472 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.472 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.472 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.472 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.472 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.472 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.472 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39005 2024-09-14T06:17:47.472 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.472 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.472 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.472 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.472 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:17:47.473 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.473 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.473 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.473 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.473 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:17:47.473 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.473 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.473 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.473 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.473 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680835+0000", 2024-09-14T06:17:47.473 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2699999999999999e-07 2024-09-14T06:17:47.474 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.474 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.474 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.474 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680837+0000", 2024-09-14T06:17:47.474 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.413e-06 2024-09-14T06:17:47.474 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.474 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.474 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.474 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680844+0000", 2024-09-14T06:17:47.474 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8820000000000003e-06 2024-09-14T06:17:47.474 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.474 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.475 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.475 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680850+0000", 2024-09-14T06:17:47.475 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5169999999999999e-06 2024-09-14T06:17:47.475 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.475 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.475 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.475 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769337+0000", 2024-09-14T06:17:47.475 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088487471999999998 2024-09-14T06:17:47.475 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.475 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.475 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.475 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769404+0000", 2024-09-14T06:17:47.475 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7111000000000002e-05 2024-09-14T06:17:47.475 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.476 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.476 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.476 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769515+0000", 2024-09-14T06:17:47.476 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011116999999999999 2024-09-14T06:17:47.476 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.476 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.476 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.476 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.476 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.476 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39006 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 294912~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.476 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:17:47.476 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704134754000002, 2024-09-14T06:17:47.476 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088886572999999997, 2024-09-14T06:17:47.476 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.477 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.477 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.477 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.477 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.477 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.477 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39006 2024-09-14T06:17:47.477 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.477 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.477 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.477 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.477 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:17:47.477 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.477 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.477 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.478 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.478 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:17:47.478 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.478 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.478 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.478 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.478 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680858+0000", 2024-09-14T06:17:47.478 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7e-07 2024-09-14T06:17:47.478 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.478 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.478 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.478 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680860+0000", 2024-09-14T06:17:47.478 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:17:47.478 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.479 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.479 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.479 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680867+0000", 2024-09-14T06:17:47.479 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0430000000000002e-06 2024-09-14T06:17:47.479 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.479 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.479 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.479 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680873+0000", 2024-09-14T06:17:47.479 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7389999999999996e-06 2024-09-14T06:17:47.479 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.479 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.479 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.479 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769547+0000", 2024-09-14T06:17:47.479 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088673830999999995 2024-09-14T06:17:47.480 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.480 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.480 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.480 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769631+0000", 2024-09-14T06:17:47.480 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3660999999999997e-05 2024-09-14T06:17:47.480 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.480 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.480 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.480 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769744+0000", 2024-09-14T06:17:47.480 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000113482 2024-09-14T06:17:47.480 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.480 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.480 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.480 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.481 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.481 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39007 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 303104~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:47.481 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:17:47.481 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.704111341999997, 2024-09-14T06:17:47.481 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.089077397000000003, 2024-09-14T06:17:47.481 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.481 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.481 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.481 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.481 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.481 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.482 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39007 2024-09-14T06:17:47.482 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.482 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.482 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.482 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.482 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:17:47.482 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.482 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.482 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.482 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.482 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:17:47.482 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.482 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.482 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.483 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.483 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687953+0000", 2024-09-14T06:17:47.483 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00707203 2024-09-14T06:17:47.483 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.483 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.483 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.483 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687960+0000", 2024-09-14T06:17:47.483 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9890000000000003e-06 2024-09-14T06:17:47.483 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.483 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.483 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.483 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687974+0000", 2024-09-14T06:17:47.483 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4226999999999999e-05 2024-09-14T06:17:47.483 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.483 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.484 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.484 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687983+0000", 2024-09-14T06:17:47.484 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9800000000000004e-06 2024-09-14T06:17:47.484 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.484 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.484 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.484 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769775+0000", 2024-09-14T06:17:47.484 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.081792089999999998 2024-09-14T06:17:47.484 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.484 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.484 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.484 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769849+0000", 2024-09-14T06:17:47.484 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3925999999999997e-05 2024-09-14T06:17:47.484 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.485 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.485 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.485 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769958+0000", 2024-09-14T06:17:47.485 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010915499999999999 2024-09-14T06:17:47.486 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.486 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.486 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.486 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.486 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.486 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:47.489 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:17:47.489 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:47.485+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:47.589 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:47.585+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:47.664 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:47.664 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:47.664 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:47.665 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:47.679 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:17:47.713 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-09-14T06:17:47.714 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:47.709+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:47.790 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:47.784+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:47.841 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:47.841 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:47.841 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:47.841 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:47.841 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:47.856 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:17:47.899 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:47.897+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:47.984 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:47.984 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:47.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:47.985 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:47.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.985 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3916 3.16 3:6ab8b1d8:::rbd_data.1152bd92e04f.0000000000000002:head [stat,truncate 1590169] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:17:47.985 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:17:47.985 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 80.403643372000005, 2024-09-14T06:17:47.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022958078999999999, 2024-09-14T06:17:47.985 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.985 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:47.985 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.986 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.986 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.986 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3916 2024-09-14T06:17:47.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.986 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:17:47.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:17:47.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579203+0000", 2024-09-14T06:17:47.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.748e-06 2024-09-14T06:17:47.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579208+0000", 2024-09-14T06:17:47.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:17:47.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579223+0000", 2024-09-14T06:17:47.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.508e-05 2024-09-14T06:17:47.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579232+0000", 2024-09-14T06:17:47.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9719999999999998e-06 2024-09-14T06:17:47.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579307+0000", 2024-09-14T06:17:47.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5028000000000001e-05 2024-09-14T06:17:47.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579412+0000", 2024-09-14T06:17:47.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010525400000000001 2024-09-14T06:17:47.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:17:47.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579535+0000", 2024-09-14T06:17:47.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122726 2024-09-14T06:17:47.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:47.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.581415+0000", 2024-09-14T06:17:47.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018800360000000001 2024-09-14T06:17:47.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:47.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602017+0000", 2024-09-14T06:17:47.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.020602425000000001 2024-09-14T06:17:47.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:47.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602096+0000", 2024-09-14T06:17:47.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8968999999999996e-05 2024-09-14T06:17:47.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602159+0000", 2024-09-14T06:17:47.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2824999999999996e-05 2024-09-14T06:17:47.991 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.991 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.991 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.991 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:7151 3.e 3:7793b17d:::rbd_data.1152f87cbac6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:17:47.991 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:17:47.991 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 76.653247504999996, 2024-09-14T06:17:47.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.029556783, 2024-09-14T06:17:47.991 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.991 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.992 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:47.992 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:47.992 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:47.992 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:47.992 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 7151 2024-09-14T06:17:47.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.992 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:17:47.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:17:47.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329599+0000", 2024-09-14T06:17:47.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5270000000000001e-06 2024-09-14T06:17:47.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329610+0000", 2024-09-14T06:17:47.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0431000000000001e-05 2024-09-14T06:17:47.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329634+0000", 2024-09-14T06:17:47.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3617000000000001e-05 2024-09-14T06:17:47.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:47.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329644+0000", 2024-09-14T06:17:47.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.095e-05 2024-09-14T06:17:47.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:47.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329712+0000", 2024-09-14T06:17:47.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7067e-05 2024-09-14T06:17:47.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:47.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329827+0000", 2024-09-14T06:17:47.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011498 2024-09-14T06:17:47.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:17:47.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.334271+0000", 2024-09-14T06:17:47.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044445839999999997 2024-09-14T06:17:47.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:47.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.358967+0000", 2024-09-14T06:17:47.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024696356999999999 2024-09-14T06:17:47.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:47.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359027+0000", 2024-09-14T06:17:47.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0006000000000001e-05 2024-09-14T06:17:47.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:47.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359099+0000", 2024-09-14T06:17:47.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1052999999999999e-05 2024-09-14T06:17:47.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:47.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359154+0000", 2024-09-14T06:17:47.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5210999999999997e-05 2024-09-14T06:17:47.997 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.997 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:47.997 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:47.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.997 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:7151 3.e e32/29 ondisk, result = 0)", 2024-09-14T06:17:47.997 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:17:47.997 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 76.633303071, 2024-09-14T06:17:47.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0096251340000000005, 2024-09-14T06:17:47.997 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:47.997 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:47.997 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:47.998 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:47.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:47.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:17:47.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:47.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:17:47.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:47.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:47.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349543+0000", 2024-09-14T06:17:47.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:17:47.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:47.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349552+0000", 2024-09-14T06:17:47.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1039999999999996e-06 2024-09-14T06:17:47.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:47.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:47.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349579+0000", 2024-09-14T06:17:47.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6449999999999999e-05 2024-09-14T06:17:47.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:47.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349593+0000", 2024-09-14T06:17:48.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4099e-05 2024-09-14T06:17:48.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359003+0000", 2024-09-14T06:17:48.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0094096119999999995 2024-09-14T06:17:48.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359014+0000", 2024-09-14T06:17:48.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0871e-05 2024-09-14T06:17:48.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.001 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.001 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359167+0000", 2024-09-14T06:17:48.001 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015300700000000001 2024-09-14T06:17:48.001 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.001 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.001 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.001 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15619 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'124, mlcod=36'119)", 2024-09-14T06:17:48.001 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:48.001 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 71.684989908999995, 2024-09-14T06:17:48.001 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061545389999999997, 2024-09-14T06:17:48.002 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.002 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.002 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.002 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:48.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:48.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:48.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3399999999999998e-07 2024-09-14T06:17:48.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297861+0000", 2024-09-14T06:17:48.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7640000000000002e-06 2024-09-14T06:17:48.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297876+0000", 2024-09-14T06:17:48.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4881e-05 2024-09-14T06:17:48.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297884+0000", 2024-09-14T06:17:48.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.588e-06 2024-09-14T06:17:48.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298294+0000", 2024-09-14T06:17:48.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00041011200000000002 2024-09-14T06:17:48.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.005 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.005 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298317+0000", 2024-09-14T06:17:48.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3417999999999998e-05 2024-09-14T06:17:48.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.005 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.005 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303972+0000", 2024-09-14T06:17:48.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056548680000000004 2024-09-14T06:17:48.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.005 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.005 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304009+0000", 2024-09-14T06:17:48.006 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7073999999999998e-05 2024-09-14T06:17:48.006 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.006 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.006 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.006 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.006 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15621 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'125, mlcod=36'119)", 2024-09-14T06:17:48.006 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:48.006 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 71.683478891999997, 2024-09-14T06:17:48.006 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0069496169999999999, 2024-09-14T06:17:48.006 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.006 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.006 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.006 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:48.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:48.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:48.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2200000000000003e-07 2024-09-14T06:17:48.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299372+0000", 2024-09-14T06:17:48.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3850000000000001e-06 2024-09-14T06:17:48.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299387+0000", 2024-09-14T06:17:48.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4905e-05 2024-09-14T06:17:48.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299393+0000", 2024-09-14T06:17:48.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9090000000000003e-06 2024-09-14T06:17:48.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.009 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.009 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303591+0000", 2024-09-14T06:17:48.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041976640000000003 2024-09-14T06:17:48.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.009 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.009 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303619+0000", 2024-09-14T06:17:48.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7651e-05 2024-09-14T06:17:48.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.009 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.009 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306287+0000", 2024-09-14T06:17:48.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026686689999999998 2024-09-14T06:17:48.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.010 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.010 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306315+0000", 2024-09-14T06:17:48.010 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7712000000000002e-05 2024-09-14T06:17:48.010 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.010 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.010 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.010 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:18539 3.1e e40/29 3:7e28f783:::rbd_data.1152227be94f.0000000000000000:head v 40'59, mlcod=40'57)", 2024-09-14T06:17:48.010 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:17:48.010 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 65.699803540999994, 2024-09-14T06:17:48.011 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0059097430000000003, 2024-09-14T06:17:48.011 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.011 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.011 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.011 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.011 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.011 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:17:48.011 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.011 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.011 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:17:48.011 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.012 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283043+0000", 2024-09-14T06:17:48.012 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4499999999999998e-06 2024-09-14T06:17:48.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.012 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285878+0000", 2024-09-14T06:17:48.012 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028348039999999998 2024-09-14T06:17:48.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.012 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285908+0000", 2024-09-14T06:17:48.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9756000000000001e-05 2024-09-14T06:17:48.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.013 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.013 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285922+0000", 2024-09-14T06:17:48.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.43e-05 2024-09-14T06:17:48.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.013 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.013 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.286030+0000", 2024-09-14T06:17:48.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000107954 2024-09-14T06:17:48.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.014 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.014 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.286058+0000", 2024-09-14T06:17:48.014 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7796e-05 2024-09-14T06:17:48.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.014 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.014 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.014 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.288888+0000", 2024-09-14T06:17:48.014 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028295799999999999 2024-09-14T06:17:48.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.014 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.014 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.014 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.288951+0000", 2024-09-14T06:17:48.014 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3102999999999995e-05 2024-09-14T06:17:48.015 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.015 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.015 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.015 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.015 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:24229 3.5 e47/34 3:a30fd30b:::rbd_data.1152db28e106.0000000000000001:head v 47'55, mlcod=45'53)", 2024-09-14T06:17:48.015 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:17:48.015 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 56.072833293999999, 2024-09-14T06:17:48.015 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0092924199999999992, 2024-09-14T06:17:48.015 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.015 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.015 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.015 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.015 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.016 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.016 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:17:48.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.016 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.016 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.016 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:17:48.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.016 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.016 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.016 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910013+0000", 2024-09-14T06:17:48.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.812e-06 2024-09-14T06:17:48.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915374+0000", 2024-09-14T06:17:48.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053609240000000004 2024-09-14T06:17:48.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915395+0000", 2024-09-14T06:17:48.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1118e-05 2024-09-14T06:17:48.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915406+0000", 2024-09-14T06:17:48.018 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1303e-05 2024-09-14T06:17:48.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.018 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.018 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.018 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915490+0000", 2024-09-14T06:17:48.018 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4017000000000006e-05 2024-09-14T06:17:48.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.018 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.018 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.018 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915515+0000", 2024-09-14T06:17:48.018 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4712e-05 2024-09-14T06:17:48.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.018 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.018 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.019 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919240+0000", 2024-09-14T06:17:48.019 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037247690000000002 2024-09-14T06:17:48.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.019 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.019 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.019 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919304+0000", 2024-09-14T06:17:48.019 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3764999999999997e-05 2024-09-14T06:17:48.019 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.019 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.019 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.019 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.019 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:head, version: 31'47, data_included: [0~98304,360448~278528], data_size: 376832, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:head@31'47, size: 638976, copy_subset: [(0, 638976)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 638976, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:17:48.019 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:17:48.020 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.772570348999999, 2024-09-14T06:17:48.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062487460000000003, 2024-09-14T06:17:48.020 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.020 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.020 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:48.020 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.020 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:17:48.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.020 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:17:48.021 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.021 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.021 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.021 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210275+0000", 2024-09-14T06:17:48.021 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:17:48.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.021 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.021 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.021 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210752+0000", 2024-09-14T06:17:48.021 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047772100000000002 2024-09-14T06:17:48.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.021 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.021 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210781+0000", 2024-09-14T06:17:48.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8391999999999999e-05 2024-09-14T06:17:48.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.022 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210805+0000", 2024-09-14T06:17:48.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4142999999999999e-05 2024-09-14T06:17:48.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.022 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.213641+0000", 2024-09-14T06:17:48.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002835827 2024-09-14T06:17:48.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.023 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.023 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.213663+0000", 2024-09-14T06:17:48.023 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.243e-05 2024-09-14T06:17:48.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.023 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.023 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.216523+0000", 2024-09-14T06:17:48.023 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028597729999999999 2024-09-14T06:17:48.023 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.023 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.023 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.023 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:50564 3.16 3:689e8220:::rbd_header.1152757de986:head [call rbd.snapshot_add in=15b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-14T06:17:48.024 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:13.319284+0000", 2024-09-14T06:17:48.024 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 34.663559968999998, 2024-09-14T06:17:48.024 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.030881613, 2024-09-14T06:17:48.024 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.024 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.024 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.024 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.024 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.024 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.024 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50564 2024-09-14T06:17:48.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.024 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.024 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319284+0000", 2024-09-14T06:17:48.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319284+0000", 2024-09-14T06:17:48.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319286+0000", 2024-09-14T06:17:48.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.849e-06 2024-09-14T06:17:48.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.026 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.026 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319293+0000", 2024-09-14T06:17:48.026 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.72e-06 2024-09-14T06:17:48.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.026 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.026 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.026 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319313+0000", 2024-09-14T06:17:48.026 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9833000000000001e-05 2024-09-14T06:17:48.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.026 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.026 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.026 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319325+0000", 2024-09-14T06:17:48.026 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1924e-05 2024-09-14T06:17:48.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.027 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.027 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.027 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319441+0000", 2024-09-14T06:17:48.027 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000115952 2024-09-14T06:17:48.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.027 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.027 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op must wait for map", 2024-09-14T06:17:48.027 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319453+0000", 2024-09-14T06:17:48.027 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1810999999999999e-05 2024-09-14T06:17:48.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.027 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.027 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.027 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322905+0000", 2024-09-14T06:17:48.027 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034519680000000001 2024-09-14T06:17:48.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.028 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.028 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:48.028 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322917+0000", 2024-09-14T06:17:48.028 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1946999999999999e-05 2024-09-14T06:17:48.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.028 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.028 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.028 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.342215+0000", 2024-09-14T06:17:48.028 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0192988300000001 2024-09-14T06:17:48.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.028 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.028 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.028 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.342451+0000", 2024-09-14T06:17:48.028 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00023546 2024-09-14T06:17:48.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.029 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:48.029 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.342959+0000", 2024-09-14T06:17:48.029 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050845999999999997 2024-09-14T06:17:48.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.029 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:48.029 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.347472+0000", 2024-09-14T06:17:48.029 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045125310000000002 2024-09-14T06:17:48.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.029 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:48.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.349544+0000", 2024-09-14T06:17:48.030 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020721730000000001 2024-09-14T06:17:48.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.030 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.030 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.349727+0000", 2024-09-14T06:17:48.030 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000182776 2024-09-14T06:17:48.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.030 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.030 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.350166+0000", 2024-09-14T06:17:48.030 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000439379 2024-09-14T06:17:48.030 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.030 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.030 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.031 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.031 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:50565 3.16 3:689e8220:::rbd_header.1152757de986:head [watch reconnect cookie 140175519011472 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-14T06:17:48.031 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:13.319351+0000", 2024-09-14T06:17:48.031 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 34.663493391000003, 2024-09-14T06:17:48.031 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0352714430000001, 2024-09-14T06:17:48.031 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.031 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.031 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:48.031 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.031 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.031 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.031 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50565 2024-09-14T06:17:48.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.031 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.032 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.032 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.032 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319351+0000", 2024-09-14T06:17:48.032 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.032 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.032 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.032 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319351+0000", 2024-09-14T06:17:48.032 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.032 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.032 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.032 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319352+0000", 2024-09-14T06:17:48.032 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0500000000000003e-07 2024-09-14T06:17:48.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.033 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.033 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.033 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319355+0000", 2024-09-14T06:17:48.033 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3960000000000002e-06 2024-09-14T06:17:48.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.033 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.033 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.033 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319367+0000", 2024-09-14T06:17:48.033 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1518999999999999e-05 2024-09-14T06:17:48.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.033 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.033 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.033 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319373+0000", 2024-09-14T06:17:48.033 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7800000000000003e-06 2024-09-14T06:17:48.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.034 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.034 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319517+0000", 2024-09-14T06:17:48.034 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000143719 2024-09-14T06:17:48.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.034 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting_for_map not empty", 2024-09-14T06:17:48.034 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319520+0000", 2024-09-14T06:17:48.034 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1990000000000002e-06 2024-09-14T06:17:48.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.034 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.034 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322963+0000", 2024-09-14T06:17:48.035 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034424920000000001 2024-09-14T06:17:48.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.035 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.035 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:48.035 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322971+0000", 2024-09-14T06:17:48.035 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0860000000000002e-06 2024-09-14T06:17:48.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.035 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.035 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.035 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343565+0000", 2024-09-14T06:17:48.035 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0205935530000001 2024-09-14T06:17:48.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.035 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.035 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for rw locks", 2024-09-14T06:17:48.035 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343675+0000", 2024-09-14T06:17:48.036 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011048900000000001 2024-09-14T06:17:48.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.036 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.036 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.036 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.350295+0000", 2024-09-14T06:17:48.036 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006620094 2024-09-14T06:17:48.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.036 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.036 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.036 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.350584+0000", 2024-09-14T06:17:48.036 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00028850000000000002 2024-09-14T06:17:48.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.036 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.036 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.037 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.354623+0000", 2024-09-14T06:17:48.037 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040389110000000001 2024-09-14T06:17:48.037 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.037 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.037 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.037 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.037 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:50566 3.16 3:689e8220:::rbd_header.1152757de986:head [watch ping cookie 140175519011472 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-14T06:17:48.037 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:13.960177+0000", 2024-09-14T06:17:48.037 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 34.022667243999997, 2024-09-14T06:17:48.037 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.39354602500000002, 2024-09-14T06:17:48.037 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.037 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.037 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:48.038 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.038 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.038 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.038 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50566 2024-09-14T06:17:48.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.038 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.038 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.038 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.038 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960177+0000", 2024-09-14T06:17:48.038 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.038 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.038 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.038 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960177+0000", 2024-09-14T06:17:48.039 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.039 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.039 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.039 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960178+0000", 2024-09-14T06:17:48.039 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.136e-06 2024-09-14T06:17:48.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.039 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.039 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.039 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960186+0000", 2024-09-14T06:17:48.039 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0299999999999994e-06 2024-09-14T06:17:48.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.039 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.039 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.040 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960202+0000", 2024-09-14T06:17:48.040 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5238999999999999e-05 2024-09-14T06:17:48.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.040 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.040 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.040 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960218+0000", 2024-09-14T06:17:48.040 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.668e-05 2024-09-14T06:17:48.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.040 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.040 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.040 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960350+0000", 2024-09-14T06:17:48.040 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013201 2024-09-14T06:17:48.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.041 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.041 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:48.041 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960368+0000", 2024-09-14T06:17:48.041 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7526000000000001e-05 2024-09-14T06:17:48.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.041 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.041 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.041 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343805+0000", 2024-09-14T06:17:48.041 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.38343735400000001 2024-09-14T06:17:48.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.041 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.041 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for rw locks", 2024-09-14T06:17:48.041 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343907+0000", 2024-09-14T06:17:48.041 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010133599999999999 2024-09-14T06:17:48.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.042 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.042 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.042 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.352997+0000", 2024-09-14T06:17:48.042 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0090907409999999994 2024-09-14T06:17:48.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.042 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.042 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.042 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.353352+0000", 2024-09-14T06:17:48.042 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00035436000000000002 2024-09-14T06:17:48.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.042 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.042 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.042 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.353723+0000", 2024-09-14T06:17:48.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00037161300000000002 2024-09-14T06:17:48.043 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.043 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.043 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.043 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:51037 3.11 3:89b51a5c:::rbd_data.11522fd7a0c5.0000000000000059:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-14T06:17:48.043 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:15.038049+0000", 2024-09-14T06:17:48.043 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 32.944795902000003, 2024-09-14T06:17:48.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.075311116999999997, 2024-09-14T06:17:48.043 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.043 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.043 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.043 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.044 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.044 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.044 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 51037 2024-09-14T06:17:48.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.044 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.044 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.044 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.044 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038049+0000", 2024-09-14T06:17:48.044 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.044 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.044 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.044 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038049+0000", 2024-09-14T06:17:48.044 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.045 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.045 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038051+0000", 2024-09-14T06:17:48.045 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:17:48.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.045 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.045 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038065+0000", 2024-09-14T06:17:48.045 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3576e-05 2024-09-14T06:17:48.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.045 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.045 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038097+0000", 2024-09-14T06:17:48.046 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2566e-05 2024-09-14T06:17:48.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.046 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.046 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038121+0000", 2024-09-14T06:17:48.046 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3207e-05 2024-09-14T06:17:48.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.046 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.046 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038291+0000", 2024-09-14T06:17:48.046 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016996900000000001 2024-09-14T06:17:48.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.046 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.046 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038543+0000", 2024-09-14T06:17:48.047 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00025233999999999999 2024-09-14T06:17:48.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.047 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.047 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:48.047 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.053218+0000", 2024-09-14T06:17:48.047 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014674957000000001 2024-09-14T06:17:48.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.047 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.047 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:48.047 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113195+0000", 2024-09-14T06:17:48.047 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.059976697000000002 2024-09-14T06:17:48.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.047 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.047 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:48.048 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113212+0000", 2024-09-14T06:17:48.048 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7299000000000002e-05 2024-09-14T06:17:48.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.048 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.048 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.048 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113267+0000", 2024-09-14T06:17:48.048 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4632999999999997e-05 2024-09-14T06:17:48.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.048 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.048 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.048 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113360+0000", 2024-09-14T06:17:48.048 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3129999999999998e-05 2024-09-14T06:17:48.048 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.049 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.049 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.049 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:51037 3.11 e67/66 ondisk, result = 0)", 2024-09-14T06:17:48.049 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:15.093818+0000", 2024-09-14T06:17:48.049 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 32.889026111, 2024-09-14T06:17:48.049 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.019384384000000001, 2024-09-14T06:17:48.049 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.049 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.049 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:48.049 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.049 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.050 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093818+0000", 2024-09-14T06:17:48.050 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.050 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.050 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.050 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093818+0000", 2024-09-14T06:17:48.050 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.050 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.050 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.050 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093820+0000", 2024-09-14T06:17:48.050 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0169999999999999e-06 2024-09-14T06:17:48.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.050 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.051 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.051 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093829+0000", 2024-09-14T06:17:48.051 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6500000000000002e-06 2024-09-14T06:17:48.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.051 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.051 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.051 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093860+0000", 2024-09-14T06:17:48.051 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1275999999999997e-05 2024-09-14T06:17:48.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.051 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.051 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.051 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093883+0000", 2024-09-14T06:17:48.051 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2464999999999999e-05 2024-09-14T06:17:48.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.051 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.052 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.052 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113178+0000", 2024-09-14T06:17:48.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.019295643000000001 2024-09-14T06:17:48.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.052 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.052 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.052 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113189+0000", 2024-09-14T06:17:48.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0285999999999999e-05 2024-09-14T06:17:48.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.052 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.052 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.052 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113203+0000", 2024-09-14T06:17:48.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4047e-05 2024-09-14T06:17:48.052 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.052 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.053 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.053 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.053 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:51156 3.16 3:6daa8e8a:::rbd_data.11522fd7a0c5.00000000000000bc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-14T06:17:48.053 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:15.298652+0000", 2024-09-14T06:17:48.053 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 32.684192474, 2024-09-14T06:17:48.053 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0057204090000000001, 2024-09-14T06:17:48.053 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.053 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.053 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.054 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 51156 2024-09-14T06:17:48.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.054 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.054 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.054 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.298652+0000", 2024-09-14T06:17:48.054 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.054 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.054 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.298652+0000", 2024-09-14T06:17:48.054 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.055 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.055 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.298655+0000", 2024-09-14T06:17:48.055 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3069999999999998e-06 2024-09-14T06:17:48.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.055 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.055 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.298671+0000", 2024-09-14T06:17:48.055 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5415999999999999e-05 2024-09-14T06:17:48.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.055 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.055 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.298714+0000", 2024-09-14T06:17:48.055 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3161999999999998e-05 2024-09-14T06:17:48.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.056 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.056 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.298740+0000", 2024-09-14T06:17:48.056 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6582999999999999e-05 2024-09-14T06:17:48.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.056 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.056 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.056 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.298936+0000", 2024-09-14T06:17:48.056 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000195212 2024-09-14T06:17:48.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.056 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.056 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.056 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.299200+0000", 2024-09-14T06:17:48.056 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00026452600000000001 2024-09-14T06:17:48.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.056 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.057 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:48.057 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.299532+0000", 2024-09-14T06:17:48.057 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00033223200000000001 2024-09-14T06:17:48.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.057 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.057 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:48.057 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.301497+0000", 2024-09-14T06:17:48.057 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001964772 2024-09-14T06:17:48.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.057 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.057 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:48.057 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.303904+0000", 2024-09-14T06:17:48.057 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024067469999999999 2024-09-14T06:17:48.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.058 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.058 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.058 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.304072+0000", 2024-09-14T06:17:48.058 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016849899999999999 2024-09-14T06:17:48.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.058 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.058 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.058 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.304372+0000", 2024-09-14T06:17:48.058 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00029995299999999999 2024-09-14T06:17:48.058 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.058 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.058 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.058 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.059 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:52268 3.e e70/34 3:7013111b:::rbd_data.1152cd930a87.0000000000000001:head v 70'315, mlcod=70'313)", 2024-09-14T06:17:48.059 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:17.975434+0000", 2024-09-14T06:17:48.059 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 30.007410913000001, 2024-09-14T06:17:48.059 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014816765000000001, 2024-09-14T06:17:48.059 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.059 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.059 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.059 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.059 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.059 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.975434+0000", 2024-09-14T06:17:48.059 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.059 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.060 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.975434+0000", 2024-09-14T06:17:48.060 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.060 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.060 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.060 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.975436+0000", 2024-09-14T06:17:48.060 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3010000000000001e-06 2024-09-14T06:17:48.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.060 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.060 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.060 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982453+0000", 2024-09-14T06:17:48.060 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0070167909999999997 2024-09-14T06:17:48.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.061 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.061 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.061 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982489+0000", 2024-09-14T06:17:48.061 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6603000000000002e-05 2024-09-14T06:17:48.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.061 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.061 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.061 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982507+0000", 2024-09-14T06:17:48.061 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8074e-05 2024-09-14T06:17:48.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.061 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.061 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.061 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982610+0000", 2024-09-14T06:17:48.061 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010224 2024-09-14T06:17:48.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.062 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.062 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.062 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982639+0000", 2024-09-14T06:17:48.062 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9009999999999998e-05 2024-09-14T06:17:48.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.062 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.062 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.062 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.990188+0000", 2024-09-14T06:17:48.062 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0075497150000000002 2024-09-14T06:17:48.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.062 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.062 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.062 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.990250+0000", 2024-09-14T06:17:48.063 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2031000000000003e-05 2024-09-14T06:17:48.063 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.063 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.063 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.063 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.063 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:52269 3.1f e70/56 3:fd802c48:::rbd_data.1152cd930a87.0000000000000002:head v 70'321, mlcod=70'318)", 2024-09-14T06:17:48.063 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:17.982671+0000", 2024-09-14T06:17:48.063 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 30.000173930999999, 2024-09-14T06:17:48.063 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016453552, 2024-09-14T06:17:48.063 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.063 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.063 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.063 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.063 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.064 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.064 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982671+0000", 2024-09-14T06:17:48.064 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.064 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.064 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.064 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982671+0000", 2024-09-14T06:17:48.064 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.064 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.064 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.064 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982672+0000", 2024-09-14T06:17:48.064 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.246e-06 2024-09-14T06:17:48.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991501+0000", 2024-09-14T06:17:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0088292590000000008 2024-09-14T06:17:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991521+0000", 2024-09-14T06:17:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0211e-05 2024-09-14T06:17:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991531+0000", 2024-09-14T06:17:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0207000000000001e-05 2024-09-14T06:17:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991605+0000", 2024-09-14T06:17:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3066999999999997e-05 2024-09-14T06:17:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991633+0000", 2024-09-14T06:17:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8136e-05 2024-09-14T06:17:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.999052+0000", 2024-09-14T06:17:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0074192320000000004 2024-09-14T06:17:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.999124+0000", 2024-09-14T06:17:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2193999999999994e-05 2024-09-14T06:17:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:59899 3.9 e75/68 3:96b79e96:::rbd_data.115214b4562.0000000000000004:head v 75'319, mlcod=75'317)", 2024-09-14T06:17:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:24.822693+0000", 2024-09-14T06:17:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.160151079999999, 2024-09-14T06:17:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014246999, 2024-09-14T06:17:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.822693+0000", 2024-09-14T06:17:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.822693+0000", 2024-09-14T06:17:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.822695+0000", 2024-09-14T06:17:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.762e-06 2024-09-14T06:17:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831397+0000", 2024-09-14T06:17:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0087016230000000003 2024-09-14T06:17:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831424+0000", 2024-09-14T06:17:48.070 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7480000000000001e-05 2024-09-14T06:17:48.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.070 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.070 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.070 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831435+0000", 2024-09-14T06:17:48.070 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0943e-05 2024-09-14T06:17:48.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.070 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.070 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.070 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831509+0000", 2024-09-14T06:17:48.070 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3596000000000005e-05 2024-09-14T06:17:48.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.070 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.070 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.070 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831531+0000", 2024-09-14T06:17:48.071 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2628000000000001e-05 2024-09-14T06:17:48.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.071 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.071 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.071 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.836873+0000", 2024-09-14T06:17:48.071 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053417849999999999 2024-09-14T06:17:48.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.071 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.071 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.071 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.836940+0000", 2024-09-14T06:17:48.071 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7181999999999995e-05 2024-09-14T06:17:48.071 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.071 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.072 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.072 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.072 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:64148 3.2 e86/29 3:41d40d1e:::rbd_data.115267b2c56c.0000000000000000:head v 86'276, mlcod=80'274)", 2024-09-14T06:17:48.072 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:38.879703+0000", 2024-09-14T06:17:48.072 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.1031414519999991, 2024-09-14T06:17:48.072 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014921665000000001, 2024-09-14T06:17:48.072 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.072 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.072 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.072 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.072 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.072 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.072 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.879703+0000", 2024-09-14T06:17:48.073 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.073 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.073 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.073 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.879703+0000", 2024-09-14T06:17:48.073 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.073 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.073 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.073 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.879704+0000", 2024-09-14T06:17:48.073 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0219999999999999e-06 2024-09-14T06:17:48.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.073 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.073 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.074 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885314+0000", 2024-09-14T06:17:48.074 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056102119999999998 2024-09-14T06:17:48.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.074 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.074 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.074 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885331+0000", 2024-09-14T06:17:48.074 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6696000000000001e-05 2024-09-14T06:17:48.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.074 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.074 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.074 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885342+0000", 2024-09-14T06:17:48.074 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0886999999999999e-05 2024-09-14T06:17:48.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.074 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.075 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.075 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885468+0000", 2024-09-14T06:17:48.075 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012655399999999999 2024-09-14T06:17:48.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.075 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.075 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.075 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885517+0000", 2024-09-14T06:17:48.075 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9057999999999997e-05 2024-09-14T06:17:48.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.075 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.075 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.075 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.894572+0000", 2024-09-14T06:17:48.075 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0090541130000000008 2024-09-14T06:17:48.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.076 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.076 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.076 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.894625+0000", 2024-09-14T06:17:48.076 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3122999999999999e-05 2024-09-14T06:17:48.076 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.076 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.076 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.076 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.076 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:64152 3.3 e86/29 3:c1c8d963:::rbd_data.115267b2c56c.0000000000000002:head v 86'180, mlcod=79'178)", 2024-09-14T06:17:48.076 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:38.884954+0000", 2024-09-14T06:17:48.076 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0978899900000005, 2024-09-14T06:17:48.076 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011982222000000001, 2024-09-14T06:17:48.076 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.077 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.077 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.077 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.077 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.077 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.884954+0000", 2024-09-14T06:17:48.077 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.077 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.077 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.884954+0000", 2024-09-14T06:17:48.077 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.078 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.078 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.884956+0000", 2024-09-14T06:17:48.078 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1179999999999999e-06 2024-09-14T06:17:48.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.078 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.078 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.078 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886783+0000", 2024-09-14T06:17:48.078 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018269289999999999 2024-09-14T06:17:48.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.078 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.078 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.078 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886795+0000", 2024-09-14T06:17:48.078 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1975e-05 2024-09-14T06:17:48.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.078 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.079 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.079 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886802+0000", 2024-09-14T06:17:48.079 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.824e-06 2024-09-14T06:17:48.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.079 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.079 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.079 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886881+0000", 2024-09-14T06:17:48.079 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8702000000000006e-05 2024-09-14T06:17:48.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.079 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.079 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.079 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886902+0000", 2024-09-14T06:17:48.079 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0854000000000001e-05 2024-09-14T06:17:48.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.080 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.080 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.080 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.896887+0000", 2024-09-14T06:17:48.080 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0099853700000000004 2024-09-14T06:17:48.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.080 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.080 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.080 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.896937+0000", 2024-09-14T06:17:48.080 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9450000000000003e-05 2024-09-14T06:17:48.080 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.080 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.080 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.080 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.081 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:64155 3.1 e86/29 3:86008c25:::rbd_data.115267b2c56c.0000000000000003:head v 86'451, mlcod=78'449)", 2024-09-14T06:17:48.081 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:17:48.081 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 9.0797889040000008, 2024-09-14T06:17:48.081 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006333038, 2024-09-14T06:17:48.081 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.081 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.081 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.081 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.081 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.081 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.081 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:17:48.081 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.081 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.082 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.082 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:17:48.082 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.082 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.082 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.082 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:17:48.082 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:17:48.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.082 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.082 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.082 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906826+0000", 2024-09-14T06:17:48.082 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037703260000000001 2024-09-14T06:17:48.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.083 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.083 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.083 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906845+0000", 2024-09-14T06:17:48.083 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8488999999999999e-05 2024-09-14T06:17:48.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.083 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.083 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.083 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906857+0000", 2024-09-14T06:17:48.083 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2571e-05 2024-09-14T06:17:48.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.083 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.083 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.083 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906966+0000", 2024-09-14T06:17:48.084 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010832000000000001 2024-09-14T06:17:48.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.084 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.084 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.084 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906984+0000", 2024-09-14T06:17:48.084 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8068999999999999e-05 2024-09-14T06:17:48.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.084 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.084 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.084 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.909306+0000", 2024-09-14T06:17:48.084 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023221050000000001 2024-09-14T06:17:48.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.084 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.084 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.085 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.909389+0000", 2024-09-14T06:17:48.085 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2865e-05 2024-09-14T06:17:48.085 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.085 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.085 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.085 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.085 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.085 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:48.087 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:17:48.087 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:48.081+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:48.187 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:48.184+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:48.262 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:48.262 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:48.262 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:48.262 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:48.277 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:17:48.323 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:48.321+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:48.348 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:17:48.388 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:48.388+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:48.397 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:48.397 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:48.397 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:48.397 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:48.397 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:48.412 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:17:48.489 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:48.484+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:48.492 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:48.493 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:48.509 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:48.509 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:48.528 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:48.528 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:48.537 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:48.537 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:48.538 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:48.538 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:48.538 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.538 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:48.538 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:48.538 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 72.543657347999996, 2024-09-14T06:17:48.538 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:17:48.538 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.538 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.538 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:48.538 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.538 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.539 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.539 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:17:48.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.539 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.539 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.539 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.539 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:48.539 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.539 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.539 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.539 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:48.539 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.540 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.540 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.540 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:48.540 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:17:48.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.540 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.540 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.540 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:17:48.540 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:17:48.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.540 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.541 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.541 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:17:48.541 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:17:48.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.541 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.541 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.541 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:17:48.541 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:17:48.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.541 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.541 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.541 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:17:48.541 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:17:48.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.542 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.542 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.542 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:17:48.542 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:17:48.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.542 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.542 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.542 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:17:48.542 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:17:48.542 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.542 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.542 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.542 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.542 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:48.543 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:48.543 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 72.543644275999995, 2024-09-14T06:17:48.543 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:17:48.543 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.543 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.543 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:48.543 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.543 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.543 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.543 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:17:48.543 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.543 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.543 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.544 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.544 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:48.544 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.544 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.544 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.544 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:48.544 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.544 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.544 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.544 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:17:48.544 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:17:48.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.545 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.545 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.545 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:17:48.545 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:17:48.545 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.545 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.545 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.545 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:17:48.545 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:17:48.545 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.545 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.545 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.545 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:17:48.545 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:17:48.546 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.546 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.546 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.546 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:17:48.546 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:17:48.546 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.546 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.546 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.546 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:17:48.546 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:17:48.546 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.546 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.546 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.546 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:17:48.546 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:17:48.547 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.547 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.547 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.547 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.547 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.547 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:48.547 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:48.547 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 72.543626844000002, 2024-09-14T06:17:48.547 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:17:48.547 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.547 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.547 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:48.547 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.547 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.547 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.548 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:17:48.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.548 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.548 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.548 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.548 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:48.548 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.548 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.548 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.548 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:48.548 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.548 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.549 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.549 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:48.549 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:17:48.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.549 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.549 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.549 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:17:48.549 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:17:48.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.549 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.549 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.550 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:17:48.550 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:17:48.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.550 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.550 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.550 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:17:48.550 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:17:48.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.550 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.551 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.551 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:17:48.551 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:17:48.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.551 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.551 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.551 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:17:48.551 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:17:48.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.551 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.551 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.551 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:17:48.551 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:17:48.551 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.552 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.552 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.552 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.552 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.552 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:48.552 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:48.552 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 72.543605739, 2024-09-14T06:17:48.552 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:17:48.552 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.552 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.552 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:48.552 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.552 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.552 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.553 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:17:48.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.553 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.553 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.553 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.553 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:48.553 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.553 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.553 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.553 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:48.553 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.553 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.554 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.554 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:48.554 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:17:48.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.554 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.554 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.554 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:17:48.554 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:17:48.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.554 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.554 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.554 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:17:48.554 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:17:48.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.555 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.555 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.555 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:17:48.555 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:17:48.555 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.555 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.555 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.555 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:17:48.555 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:17:48.555 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.555 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.555 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.555 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:17:48.555 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:17:48.556 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.556 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.556 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.556 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:17:48.556 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:17:48.556 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.556 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.556 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.556 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.556 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.556 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:48.556 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:48.556 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 72.543584631000002, 2024-09-14T06:17:48.556 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:17:48.557 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.557 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.557 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:48.557 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.557 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.557 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.557 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:17:48.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.557 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.557 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.557 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.557 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:48.557 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.558 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.558 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.558 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:48.558 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.558 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.558 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.558 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:48.558 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:17:48.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.559 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.559 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.559 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:17:48.559 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:17:48.559 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.559 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.559 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.559 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:17:48.559 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:17:48.559 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.560 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.560 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.560 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:17:48.560 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:17:48.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.560 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.560 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.560 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:17:48.560 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:17:48.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.560 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.560 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.560 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:17:48.560 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:17:48.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.561 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.561 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.561 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:17:48.561 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:17:48.561 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.561 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.561 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.561 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.561 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48234 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:48.561 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:48.561 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.502313338999997, 2024-09-14T06:17:48.561 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051730353999999999, 2024-09-14T06:17:48.562 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.562 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.562 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.562 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.562 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.562 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.562 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48234 2024-09-14T06:17:48.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.562 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.562 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.562 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.562 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:48.562 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.563 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.563 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.563 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:48.563 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.563 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.563 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.563 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032937+0000", 2024-09-14T06:17:48.563 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:17:48.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.563 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.563 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.563 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032940+0000", 2024-09-14T06:17:48.563 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0580000000000002e-06 2024-09-14T06:17:48.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.564 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.564 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.564 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032946+0000", 2024-09-14T06:17:48.564 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2960000000000004e-06 2024-09-14T06:17:48.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.564 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.564 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.564 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032951+0000", 2024-09-14T06:17:48.564 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.938e-06 2024-09-14T06:17:48.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.564 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.564 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.564 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074419+0000", 2024-09-14T06:17:48.565 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041467979000000002 2024-09-14T06:17:48.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.565 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.565 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.565 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074490+0000", 2024-09-14T06:17:48.565 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1324999999999999e-05 2024-09-14T06:17:48.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.565 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.565 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:48.565 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074859+0000", 2024-09-14T06:17:48.565 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036914899999999999 2024-09-14T06:17:48.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.565 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.565 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:48.565 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078464+0000", 2024-09-14T06:17:48.566 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036043170000000001 2024-09-14T06:17:48.566 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.566 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.566 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:48.566 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084547+0000", 2024-09-14T06:17:48.566 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006083097 2024-09-14T06:17:48.566 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.566 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.566 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.566 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084614+0000", 2024-09-14T06:17:48.566 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7509999999999996e-05 2024-09-14T06:17:48.566 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.566 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.567 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.567 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084667+0000", 2024-09-14T06:17:48.567 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:17:48.567 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.567 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.567 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.567 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.567 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.567 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48237 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:48.567 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:48.567 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.502233595, 2024-09-14T06:17:48.567 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051772830999999998, 2024-09-14T06:17:48.567 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.567 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.568 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.568 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.568 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.568 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.568 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48237 2024-09-14T06:17:48.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.568 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.568 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.568 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.568 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:48.568 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.568 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.568 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.569 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:48.569 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.569 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.569 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.569 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:48.569 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0499999999999999e-07 2024-09-14T06:17:48.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.569 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.569 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.569 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033018+0000", 2024-09-14T06:17:48.569 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0949999999999998e-06 2024-09-14T06:17:48.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.570 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.570 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.570 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033023+0000", 2024-09-14T06:17:48.570 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1620000000000001e-06 2024-09-14T06:17:48.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.570 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.570 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.570 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033026+0000", 2024-09-14T06:17:48.570 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0120000000000001e-06 2024-09-14T06:17:48.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.570 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.570 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.570 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075349+0000", 2024-09-14T06:17:48.571 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042323373999999997 2024-09-14T06:17:48.571 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.571 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.571 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.571 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075394+0000", 2024-09-14T06:17:48.571 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4861000000000003e-05 2024-09-14T06:17:48.571 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.571 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.571 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:48.571 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075578+0000", 2024-09-14T06:17:48.571 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00018414899999999999 2024-09-14T06:17:48.571 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.571 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.571 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:48.572 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078894+0000", 2024-09-14T06:17:48.572 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033158480000000001 2024-09-14T06:17:48.572 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.572 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.572 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:48.572 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084677+0000", 2024-09-14T06:17:48.572 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0057832120000000002 2024-09-14T06:17:48.572 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.572 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.572 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.572 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084752+0000", 2024-09-14T06:17:48.572 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5123000000000006e-05 2024-09-14T06:17:48.572 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.573 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.573 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.573 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084789+0000", 2024-09-14T06:17:48.573 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6690000000000003e-05 2024-09-14T06:17:48.573 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.573 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.573 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.573 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.573 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48243 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:48.573 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:48.573 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.502152469999999, 2024-09-14T06:17:48.573 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051780221000000001, 2024-09-14T06:17:48.573 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.574 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.574 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.574 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.574 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.574 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.574 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48243 2024-09-14T06:17:48.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.574 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.574 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.574 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.574 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:48.574 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.574 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:17:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033100+0000", 2024-09-14T06:17:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:17:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033108+0000", 2024-09-14T06:17:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1760000000000004e-06 2024-09-14T06:17:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033111+0000", 2024-09-14T06:17:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0790000000000001e-06 2024-09-14T06:17:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075903+0000", 2024-09-14T06:17:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042791942999999999 2024-09-14T06:17:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075987+0000", 2024-09-14T06:17:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4467000000000003e-05 2024-09-14T06:17:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.076259+0000", 2024-09-14T06:17:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027201900000000003 2024-09-14T06:17:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078947+0000", 2024-09-14T06:17:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026875879999999999 2024-09-14T06:17:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084797+0000", 2024-09-14T06:17:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0058500380000000001 2024-09-14T06:17:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084843+0000", 2024-09-14T06:17:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6298999999999998e-05 2024-09-14T06:17:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084877+0000", 2024-09-14T06:17:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4378999999999997e-05 2024-09-14T06:17:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48287 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.502112670000002, 2024-09-14T06:17:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051924706000000001, 2024-09-14T06:17:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48287 2024-09-14T06:17:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033138+0000", 2024-09-14T06:17:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6599999999999996e-07 2024-09-14T06:17:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033141+0000", 2024-09-14T06:17:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0879999999999999e-06 2024-09-14T06:17:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033147+0000", 2024-09-14T06:17:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2249999999999997e-06 2024-09-14T06:17:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033153+0000", 2024-09-14T06:17:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.066e-06 2024-09-14T06:17:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077790+0000", 2024-09-14T06:17:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.044637255000000001 2024-09-14T06:17:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077845+0000", 2024-09-14T06:17:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4410999999999998e-05 2024-09-14T06:17:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078062+0000", 2024-09-14T06:17:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021755599999999999 2024-09-14T06:17:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.080991+0000", 2024-09-14T06:17:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029282969999999998 2024-09-14T06:17:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084885+0000", 2024-09-14T06:17:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003894495 2024-09-14T06:17:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085020+0000", 2024-09-14T06:17:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013500799999999999 2024-09-14T06:17:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085062+0000", 2024-09-14T06:17:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1638999999999999e-05 2024-09-14T06:17:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48288 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.502088970999999, 2024-09-14T06:17:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051994018000000003, 2024-09-14T06:17:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48288 2024-09-14T06:17:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.89e-07 2024-09-14T06:17:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033164+0000", 2024-09-14T06:17:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.779e-06 2024-09-14T06:17:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033171+0000", 2024-09-14T06:17:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6279999999999999e-06 2024-09-14T06:17:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033176+0000", 2024-09-14T06:17:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1610000000000002e-06 2024-09-14T06:17:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078991+0000", 2024-09-14T06:17:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045815348999999998 2024-09-14T06:17:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079086+0000", 2024-09-14T06:17:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5282e-05 2024-09-14T06:17:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079397+0000", 2024-09-14T06:17:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031055999999999998 2024-09-14T06:17:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084432+0000", 2024-09-14T06:17:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050355599999999997 2024-09-14T06:17:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085073+0000", 2024-09-14T06:17:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064015199999999995 2024-09-14T06:17:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085120+0000", 2024-09-14T06:17:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7107000000000001e-05 2024-09-14T06:17:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085155+0000", 2024-09-14T06:17:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4950999999999998e-05 2024-09-14T06:17:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48416 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.499848559, 2024-09-14T06:17:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053690144000000002, 2024-09-14T06:17:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48416 2024-09-14T06:17:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035402+0000", 2024-09-14T06:17:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7999999999999995e-07 2024-09-14T06:17:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035405+0000", 2024-09-14T06:17:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3960000000000002e-06 2024-09-14T06:17:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035413+0000", 2024-09-14T06:17:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2209999999999997e-06 2024-09-14T06:17:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035420+0000", 2024-09-14T06:17:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1539999999999996e-06 2024-09-14T06:17:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081021+0000", 2024-09-14T06:17:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045600389999999998 2024-09-14T06:17:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081091+0000", 2024-09-14T06:17:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0325000000000002e-05 2024-09-14T06:17:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081434+0000", 2024-09-14T06:17:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000342519 2024-09-14T06:17:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085254+0000", 2024-09-14T06:17:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038201609999999999 2024-09-14T06:17:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088977+0000", 2024-09-14T06:17:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003722805 2024-09-14T06:17:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089048+0000", 2024-09-14T06:17:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1266999999999994e-05 2024-09-14T06:17:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089091+0000", 2024-09-14T06:17:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3325999999999998e-05 2024-09-14T06:17:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48417 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.499819889999998, 2024-09-14T06:17:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053805032000000003, 2024-09-14T06:17:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48417 2024-09-14T06:17:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9500000000000003e-07 2024-09-14T06:17:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035433+0000", 2024-09-14T06:17:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.796e-06 2024-09-14T06:17:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035442+0000", 2024-09-14T06:17:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7120000000000006e-06 2024-09-14T06:17:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035448+0000", 2024-09-14T06:17:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.015e-06 2024-09-14T06:17:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081855+0000", 2024-09-14T06:17:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.046407634000000003 2024-09-14T06:17:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081938+0000", 2024-09-14T06:17:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2687000000000003e-05 2024-09-14T06:17:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082313+0000", 2024-09-14T06:17:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000374872 2024-09-14T06:17:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087616+0000", 2024-09-14T06:17:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053032670000000004 2024-09-14T06:17:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089140+0000", 2024-09-14T06:17:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0015234350000000001 2024-09-14T06:17:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089184+0000", 2024-09-14T06:17:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4804e-05 2024-09-14T06:17:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089235+0000", 2024-09-14T06:17:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0315000000000002e-05 2024-09-14T06:17:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48418 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.499775751000001, 2024-09-14T06:17:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053936425000000003, 2024-09-14T06:17:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48418 2024-09-14T06:17:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035475+0000", 2024-09-14T06:17:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7000000000000003e-07 2024-09-14T06:17:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035479+0000", 2024-09-14T06:17:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2350000000000001e-06 2024-09-14T06:17:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035488+0000", 2024-09-14T06:17:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8820000000000007e-06 2024-09-14T06:17:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035494+0000", 2024-09-14T06:17:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3060000000000003e-06 2024-09-14T06:17:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082826+0000", 2024-09-14T06:17:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.047332008000000002 2024-09-14T06:17:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082883+0000", 2024-09-14T06:17:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7124999999999999e-05 2024-09-14T06:17:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.083385+0000", 2024-09-14T06:17:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050134099999999998 2024-09-14T06:17:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088888+0000", 2024-09-14T06:17:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005503127 2024-09-14T06:17:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089294+0000", 2024-09-14T06:17:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040669700000000003 2024-09-14T06:17:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089348+0000", 2024-09-14T06:17:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3729e-05 2024-09-14T06:17:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089410+0000", 2024-09-14T06:17:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2105000000000003e-05 2024-09-14T06:17:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48419 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.499698121000002, 2024-09-14T06:17:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054007545999999997, 2024-09-14T06:17:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48419 2024-09-14T06:17:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035552+0000", 2024-09-14T06:17:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4099999999999998e-07 2024-09-14T06:17:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035557+0000", 2024-09-14T06:17:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1819999999999997e-06 2024-09-14T06:17:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035566+0000", 2024-09-14T06:17:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1120000000000002e-06 2024-09-14T06:17:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035572+0000", 2024-09-14T06:17:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0430000000000004e-06 2024-09-14T06:17:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085304+0000", 2024-09-14T06:17:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049731707 2024-09-14T06:17:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085371+0000", 2024-09-14T06:17:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6771000000000002e-05 2024-09-14T06:17:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085601+0000", 2024-09-14T06:17:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022968000000000001 2024-09-14T06:17:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088893+0000", 2024-09-14T06:17:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032927780000000001 2024-09-14T06:17:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089467+0000", 2024-09-14T06:17:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057330399999999996 2024-09-14T06:17:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089520+0000", 2024-09-14T06:17:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3133000000000001e-05 2024-09-14T06:17:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089559+0000", 2024-09-14T06:17:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9195e-05 2024-09-14T06:17:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48420 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.499608891000001, 2024-09-14T06:17:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056401932000000002, 2024-09-14T06:17:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48420 2024-09-14T06:17:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035642+0000", 2024-09-14T06:17:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1519999999999999e-06 2024-09-14T06:17:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035647+0000", 2024-09-14T06:17:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2299999999999999e-06 2024-09-14T06:17:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035660+0000", 2024-09-14T06:17:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3166999999999999e-05 2024-09-14T06:17:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035666+0000", 2024-09-14T06:17:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1720000000000004e-06 2024-09-14T06:17:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086104+0000", 2024-09-14T06:17:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050437956999999999 2024-09-14T06:17:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086164+0000", 2024-09-14T06:17:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9592999999999997e-05 2024-09-14T06:17:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086402+0000", 2024-09-14T06:17:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000238314 2024-09-14T06:17:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089103+0000", 2024-09-14T06:17:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027011600000000002 2024-09-14T06:17:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091962+0000", 2024-09-14T06:17:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028581259999999999 2024-09-14T06:17:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092005+0000", 2024-09-14T06:17:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3111000000000001e-05 2024-09-14T06:17:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092043+0000", 2024-09-14T06:17:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7950000000000001e-05 2024-09-14T06:17:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48424 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.499344946000001, 2024-09-14T06:17:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056273173000000003, 2024-09-14T06:17:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48424 2024-09-14T06:17:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2900000000000003e-07 2024-09-14T06:17:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035909+0000", 2024-09-14T06:17:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6490000000000001e-06 2024-09-14T06:17:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035918+0000", 2024-09-14T06:17:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5089999999999999e-06 2024-09-14T06:17:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035924+0000", 2024-09-14T06:17:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7089999999999999e-06 2024-09-14T06:17:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087662+0000", 2024-09-14T06:17:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737621999999997 2024-09-14T06:17:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087744+0000", 2024-09-14T06:17:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2174000000000003e-05 2024-09-14T06:17:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088106+0000", 2024-09-14T06:17:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000361947 2024-09-14T06:17:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092053+0000", 2024-09-14T06:17:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0039473080000000001 2024-09-14T06:17:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092087+0000", 2024-09-14T06:17:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4153000000000003e-05 2024-09-14T06:17:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092138+0000", 2024-09-14T06:17:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0804999999999999e-05 2024-09-14T06:17:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092178+0000", 2024-09-14T06:17:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9668e-05 2024-09-14T06:17:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48508 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.494269865, 2024-09-14T06:17:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054906017000000001, 2024-09-14T06:17:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48508 2024-09-14T06:17:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040981+0000", 2024-09-14T06:17:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7700000000000004e-07 2024-09-14T06:17:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040984+0000", 2024-09-14T06:17:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1480000000000002e-06 2024-09-14T06:17:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040991+0000", 2024-09-14T06:17:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4580000000000004e-06 2024-09-14T06:17:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040997+0000", 2024-09-14T06:17:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7799999999999997e-06 2024-09-14T06:17:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089601+0000", 2024-09-14T06:17:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048604264000000001 2024-09-14T06:17:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089708+0000", 2024-09-14T06:17:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010681200000000001 2024-09-14T06:17:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090384+0000", 2024-09-14T06:17:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00067557499999999998 2024-09-14T06:17:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093879+0000", 2024-09-14T06:17:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034954679999999998 2024-09-14T06:17:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095764+0000", 2024-09-14T06:17:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018848070000000001 2024-09-14T06:17:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095837+0000", 2024-09-14T06:17:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3433e-05 2024-09-14T06:17:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095886+0000", 2024-09-14T06:17:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8495000000000002e-05 2024-09-14T06:17:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48537 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.492906095000002, 2024-09-14T06:17:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053681833999999998, 2024-09-14T06:17:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48537 2024-09-14T06:17:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2299999999999999e-07 2024-09-14T06:17:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042350+0000", 2024-09-14T06:17:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5690000000000004e-06 2024-09-14T06:17:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042361+0000", 2024-09-14T06:17:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1337e-05 2024-09-14T06:17:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042369+0000", 2024-09-14T06:17:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8359999999999992e-06 2024-09-14T06:17:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090914+0000", 2024-09-14T06:17:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048544945999999999 2024-09-14T06:17:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090995+0000", 2024-09-14T06:17:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0548000000000002e-05 2024-09-14T06:17:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091508+0000", 2024-09-14T06:17:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000513723 2024-09-14T06:17:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093900+0000", 2024-09-14T06:17:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023920429999999999 2024-09-14T06:17:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095944+0000", 2024-09-14T06:17:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020437379999999998 2024-09-14T06:17:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095987+0000", 2024-09-14T06:17:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2521999999999997e-05 2024-09-14T06:17:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.096025+0000", 2024-09-14T06:17:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8649000000000002e-05 2024-09-14T06:17:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:63810 3.1d 3.c887d99d (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-14T06:17:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:17:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 14.955148209000001, 2024-09-14T06:17:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.61346978900000004, 2024-09-14T06:17:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 63810 2024-09-14T06:17:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:17:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:17:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580104+0000", 2024-09-14T06:17:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4650000000000001e-06 2024-09-14T06:17:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580112+0000", 2024-09-14T06:17:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9219999999999997e-06 2024-09-14T06:17:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580130+0000", 2024-09-14T06:17:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8128999999999999e-05 2024-09-14T06:17:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580140+0000", 2024-09-14T06:17:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0285000000000001e-05 2024-09-14T06:17:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580201+0000", 2024-09-14T06:17:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0517999999999999e-05 2024-09-14T06:17:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580209+0000", 2024-09-14T06:17:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6430000000000001e-06 2024-09-14T06:17:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:34.193554+0000", 2024-09-14T06:17:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.61334482899999998 2024-09-14T06:17:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:34.193571+0000", 2024-09-14T06:17:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6997999999999999e-05 2024-09-14T06:17:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:63832 3.1d 3:b99be113:::rbd_object_map.11525ae33f05:head [read 0~52] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T06:17:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:17:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 11.297151823, 2024-09-14T06:17:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.99066972200000003, 2024-09-14T06:17:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 63832 2024-09-14T06:17:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:17:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:17:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238100+0000", 2024-09-14T06:17:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0849999999999999e-06 2024-09-14T06:17:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238108+0000", 2024-09-14T06:17:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.089e-06 2024-09-14T06:17:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238123+0000", 2024-09-14T06:17:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5418e-05 2024-09-14T06:17:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238136+0000", 2024-09-14T06:17:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2204e-05 2024-09-14T06:17:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238913+0000", 2024-09-14T06:17:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00077720399999999998 2024-09-14T06:17:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238934+0000", 2024-09-14T06:17:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1311999999999999e-05 2024-09-14T06:17:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228134+0000", 2024-09-14T06:17:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.98919991200000001 2024-09-14T06:17:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228261+0000", 2024-09-14T06:17:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000126742 2024-09-14T06:17:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228768+0000", 2024-09-14T06:17:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050675600000000003 2024-09-14T06:17:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:48.647 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:48.648 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:48.649 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:48.650 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:17:48.650 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:48.651 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:48.651 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:48.651 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:48.651 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:48.652 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:48.652 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:48.652 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:48.653 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:48.644+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:48.749 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:48.744+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:48.790 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:48.790 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:48.804 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:17:48.850 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:48.845+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:48.951 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:48.945+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:48.978 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:48.978 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:48.994 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:17:49.052 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:49.048+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:49.115 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:49.116 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:49.130 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:17:49.175 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:49.172+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:49.253 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:49.249+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:49.312 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:49.313 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:49.328 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:17:49.371 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:49.368+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:49.454 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:49.455 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:49.455 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:49.448+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:49.469 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:17:49.556 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:49.552+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:49.633 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:49.634 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:49.647 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:17:49.657 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:49.653+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:49.723 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:17:49.723 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:17:49.723 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-14T06:17:49.723 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-14T06:17:49.748 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:17:49.768 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:49.765+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:49.820 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:49.820 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:49.836 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:17:49.879 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:49.876+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:49.958 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:49.958 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:49.959 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:49.952+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:49.972 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:17:50.043 INFO:teuthology.orchestra.run.smithi039.stdout:moved 0 / 82 (0%) 2024-09-14T06:17:50.043 INFO:teuthology.orchestra.run.smithi039.stdout:avg 10.25 2024-09-14T06:17:50.043 INFO:teuthology.orchestra.run.smithi039.stdout:stddev 2.63391 -> 2.63391 (expected baseline 2.99479) 2024-09-14T06:17:50.044 INFO:teuthology.orchestra.run.smithi039.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-14T06:17:50.044 INFO:teuthology.orchestra.run.smithi039.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-14T06:17:50.044 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:50.044 INFO:teuthology.orchestra.run.smithi039.stdout:oload 1000 2024-09-14T06:17:50.044 INFO:teuthology.orchestra.run.smithi039.stdout:max_change 0.05 2024-09-14T06:17:50.044 INFO:teuthology.orchestra.run.smithi039.stdout:max_change_osds 4 2024-09-14T06:17:50.044 INFO:teuthology.orchestra.run.smithi039.stdout:average_utilization 0.0005 2024-09-14T06:17:50.044 INFO:teuthology.orchestra.run.smithi039.stdout:overload_utilization 0.0054 2024-09-14T06:17:50.044 INFO:teuthology.orchestra.run.smithi039.stderr:no change 2024-09-14T06:17:50.063 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-14T06:17:50.064 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-14T06:17:50.064 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:50.060+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:50.140 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:50.140 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:50.154 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:17:50.183 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-09-14T06:17:50.195 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:50.192+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:50.261 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:50.256+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:50.323 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:50.323 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:50.338 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:17:50.363 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:50.357+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:50.398 INFO:teuthology.orchestra.run.smithi039.stdout:moved 0 / 82 (0%) 2024-09-14T06:17:50.398 INFO:teuthology.orchestra.run.smithi039.stdout:avg 10.25 2024-09-14T06:17:50.398 INFO:teuthology.orchestra.run.smithi039.stdout:stddev 2.63391 -> 2.63391 (expected baseline 2.99479) 2024-09-14T06:17:50.398 INFO:teuthology.orchestra.run.smithi039.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-14T06:17:50.398 INFO:teuthology.orchestra.run.smithi039.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-14T06:17:50.398 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:50.399 INFO:teuthology.orchestra.run.smithi039.stdout:oload 1000 2024-09-14T06:17:50.399 INFO:teuthology.orchestra.run.smithi039.stdout:max_change 3 2024-09-14T06:17:50.399 INFO:teuthology.orchestra.run.smithi039.stdout:max_change_osds 4 2024-09-14T06:17:50.399 INFO:teuthology.orchestra.run.smithi039.stdout:average_utilization 0.0005 2024-09-14T06:17:50.399 INFO:teuthology.orchestra.run.smithi039.stdout:overload_utilization 0.0054 2024-09-14T06:17:50.399 INFO:teuthology.orchestra.run.smithi039.stderr:no change 2024-09-14T06:17:50.419 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-14T06:17:50.419 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-14T06:17:50.465 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:50.460+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:50.506 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:50.506 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:50.520 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:17:50.564 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:50.561+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:50.655 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:17:50.683 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:50.683 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:50.697 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:17:50.700 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:50.697+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:50.766 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:50.760+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:50.791 INFO:teuthology.orchestra.run.smithi039.stdout:moved 0 / 82 (0%) 2024-09-14T06:17:50.791 INFO:teuthology.orchestra.run.smithi039.stdout:avg 10.25 2024-09-14T06:17:50.792 INFO:teuthology.orchestra.run.smithi039.stdout:stddev 2.63391 -> 2.63391 (expected baseline 2.99479) 2024-09-14T06:17:50.792 INFO:teuthology.orchestra.run.smithi039.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-14T06:17:50.792 INFO:teuthology.orchestra.run.smithi039.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-14T06:17:50.792 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:50.792 INFO:teuthology.orchestra.run.smithi039.stdout:oload 1000 2024-09-14T06:17:50.792 INFO:teuthology.orchestra.run.smithi039.stdout:max_change 0.05 2024-09-14T06:17:50.792 INFO:teuthology.orchestra.run.smithi039.stdout:max_change_osds 4 2024-09-14T06:17:50.792 INFO:teuthology.orchestra.run.smithi039.stdout:average_utilization 0.0005 2024-09-14T06:17:50.792 INFO:teuthology.orchestra.run.smithi039.stdout:overload_utilization 0.0054 2024-09-14T06:17:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:no change 2024-09-14T06:17:50.808 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:50.809 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:50.811 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-14T06:17:50.812 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-14T06:17:50.822 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:50.822 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:50.822 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:50.822 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:50.826 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:50.826 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:50.838 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:17:50.843 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:50.844 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:50.861 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:50.862 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:50.866 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:50.860+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:50.883 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:50.883 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:50.904 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:50.904 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:50.925 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:50.925 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:50.947 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:50.948 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:50.967 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:50.960+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:50.967 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:50.967 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:50.967 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:50.982 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:17:51.068 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:51.068+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:51.119 INFO:teuthology.orchestra.run.smithi039.stdout:moved 0 / 82 (0%) 2024-09-14T06:17:51.119 INFO:teuthology.orchestra.run.smithi039.stdout:avg 10.25 2024-09-14T06:17:51.119 INFO:teuthology.orchestra.run.smithi039.stdout:stddev 2.63391 -> 2.63391 (expected baseline 2.99479) 2024-09-14T06:17:51.119 INFO:teuthology.orchestra.run.smithi039.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-14T06:17:51.119 INFO:teuthology.orchestra.run.smithi039.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-14T06:17:51.119 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:51.119 INFO:teuthology.orchestra.run.smithi039.stdout:oload 1000 2024-09-14T06:17:51.119 INFO:teuthology.orchestra.run.smithi039.stdout:max_change 0.05 2024-09-14T06:17:51.119 INFO:teuthology.orchestra.run.smithi039.stdout:max_change_osds 4 2024-09-14T06:17:51.119 INFO:teuthology.orchestra.run.smithi039.stdout:average_utilization 0.0005 2024-09-14T06:17:51.120 INFO:teuthology.orchestra.run.smithi039.stdout:overload_utilization 0.0054 2024-09-14T06:17:51.120 INFO:teuthology.orchestra.run.smithi039.stderr:no change 2024-09-14T06:17:51.138 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-14T06:17:51.138 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-14T06:17:51.144 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:51.144 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:51.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:51.144 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:51.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.144 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40393 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 868352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.144 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.708703+0000", 2024-09-14T06:17:51.144 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.433491322000002, 2024-09-14T06:17:51.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12669228199999999, 2024-09-14T06:17:51.144 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.144 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.144 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.145 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.145 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.145 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.145 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40393 2024-09-14T06:17:51.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.145 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708703+0000", 2024-09-14T06:17:51.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708703+0000", 2024-09-14T06:17:51.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708881+0000", 2024-09-14T06:17:51.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00017825 2024-09-14T06:17:51.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708885+0000", 2024-09-14T06:17:51.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.019e-06 2024-09-14T06:17:51.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708894+0000", 2024-09-14T06:17:51.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5469999999999997e-06 2024-09-14T06:17:51.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.149 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.149 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708900+0000", 2024-09-14T06:17:51.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.421e-06 2024-09-14T06:17:51.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.149 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.149 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835012+0000", 2024-09-14T06:17:51.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12611203400000001 2024-09-14T06:17:51.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.150 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835095+0000", 2024-09-14T06:17:51.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.229e-05 2024-09-14T06:17:51.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.150 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835395+0000", 2024-09-14T06:17:51.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000300721 2024-09-14T06:17:51.150 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.150 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.150 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.150 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40395 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 876544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.151 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.708914+0000", 2024-09-14T06:17:51.151 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.433280689999997, 2024-09-14T06:17:51.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12688781199999999, 2024-09-14T06:17:51.151 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.151 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.151 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.151 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.151 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.151 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.151 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40395 2024-09-14T06:17:51.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.151 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708914+0000", 2024-09-14T06:17:51.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.708914+0000", 2024-09-14T06:17:51.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709267+0000", 2024-09-14T06:17:51.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00035336999999999999 2024-09-14T06:17:51.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709272+0000", 2024-09-14T06:17:51.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4430000000000002e-06 2024-09-14T06:17:51.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709283+0000", 2024-09-14T06:17:51.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1034e-05 2024-09-14T06:17:51.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709291+0000", 2024-09-14T06:17:51.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4739999999999999e-06 2024-09-14T06:17:51.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835428+0000", 2024-09-14T06:17:51.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126137052 2024-09-14T06:17:51.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835506+0000", 2024-09-14T06:17:51.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8492000000000007e-05 2024-09-14T06:17:51.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835801+0000", 2024-09-14T06:17:51.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00029494699999999999 2024-09-14T06:17:51.154 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.154 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.154 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.155 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40397 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.155 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.709307+0000", 2024-09-14T06:17:51.155 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.432887587000003, 2024-09-14T06:17:51.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126901031, 2024-09-14T06:17:51.155 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.155 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.155 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.155 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.155 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.155 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.155 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40397 2024-09-14T06:17:51.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.155 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.156 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.156 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709307+0000", 2024-09-14T06:17:51.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.156 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.156 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709307+0000", 2024-09-14T06:17:51.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.156 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.156 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709590+0000", 2024-09-14T06:17:51.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00028368499999999999 2024-09-14T06:17:51.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709596+0000", 2024-09-14T06:17:51.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3940000000000004e-06 2024-09-14T06:17:51.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709605+0000", 2024-09-14T06:17:51.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8650000000000007e-06 2024-09-14T06:17:51.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709610+0000", 2024-09-14T06:17:51.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1139999999999997e-06 2024-09-14T06:17:51.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835834+0000", 2024-09-14T06:17:51.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12622423199999999 2024-09-14T06:17:51.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.835915+0000", 2024-09-14T06:17:51.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0859999999999995e-05 2024-09-14T06:17:51.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836208+0000", 2024-09-14T06:17:51.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00029288099999999999 2024-09-14T06:17:51.159 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.159 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.159 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.159 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40399 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.159 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.709624+0000", 2024-09-14T06:17:51.159 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.432569790000002, 2024-09-14T06:17:51.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.127024368, 2024-09-14T06:17:51.159 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.159 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.159 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.159 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.159 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.160 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.160 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40399 2024-09-14T06:17:51.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.160 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709624+0000", 2024-09-14T06:17:51.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.709624+0000", 2024-09-14T06:17:51.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710144+0000", 2024-09-14T06:17:51.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00051909700000000005 2024-09-14T06:17:51.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710156+0000", 2024-09-14T06:17:51.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2846e-05 2024-09-14T06:17:51.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710166+0000", 2024-09-14T06:17:51.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:17:51.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710174+0000", 2024-09-14T06:17:51.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:17:51.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836240+0000", 2024-09-14T06:17:51.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12606634899999999 2024-09-14T06:17:51.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836320+0000", 2024-09-14T06:17:51.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0073000000000004e-05 2024-09-14T06:17:51.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836649+0000", 2024-09-14T06:17:51.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032834999999999999 2024-09-14T06:17:51.163 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.163 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.163 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.163 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40401 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.163 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.710189+0000", 2024-09-14T06:17:51.163 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.432005623000002, 2024-09-14T06:17:51.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126903455, 2024-09-14T06:17:51.163 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.164 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.164 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.164 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.164 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.164 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.164 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40401 2024-09-14T06:17:51.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.164 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710189+0000", 2024-09-14T06:17:51.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.710189+0000", 2024-09-14T06:17:51.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712809+0000", 2024-09-14T06:17:51.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026201179999999998 2024-09-14T06:17:51.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712815+0000", 2024-09-14T06:17:51.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7960000000000001e-06 2024-09-14T06:17:51.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712825+0000", 2024-09-14T06:17:51.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0044000000000001e-05 2024-09-14T06:17:51.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712832+0000", 2024-09-14T06:17:51.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.328e-06 2024-09-14T06:17:51.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836682+0000", 2024-09-14T06:17:51.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.123850141 2024-09-14T06:17:51.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.836762+0000", 2024-09-14T06:17:51.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0207e-05 2024-09-14T06:17:51.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.837092+0000", 2024-09-14T06:17:51.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000329821 2024-09-14T06:17:51.167 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.167 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.167 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.168 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40407 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.168 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:51.168 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.429312207999999, 2024-09-14T06:17:51.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.127190898, 2024-09-14T06:17:51.168 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.168 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.168 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.168 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.168 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.168 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.168 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40407 2024-09-14T06:17:51.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.169 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.169 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.169 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:51.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.169 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.169 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:51.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.169 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.170 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712882+0000", 2024-09-14T06:17:51.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0900000000000002e-07 2024-09-14T06:17:51.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.170 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712884+0000", 2024-09-14T06:17:51.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5579999999999999e-06 2024-09-14T06:17:51.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.170 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712887+0000", 2024-09-14T06:17:51.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3119999999999998e-06 2024-09-14T06:17:51.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712890+0000", 2024-09-14T06:17:51.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7e-06 2024-09-14T06:17:51.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.839114+0000", 2024-09-14T06:17:51.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.126223954 2024-09-14T06:17:51.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.839197+0000", 2024-09-14T06:17:51.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3250000000000004e-05 2024-09-14T06:17:51.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840073+0000", 2024-09-14T06:17:51.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000875715 2024-09-14T06:17:51.172 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.172 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.172 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.172 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40409 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.172 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:51.172 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.429300427000001, 2024-09-14T06:17:51.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12739108900000001, 2024-09-14T06:17:51.172 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.173 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.173 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.173 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.173 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.173 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.173 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40409 2024-09-14T06:17:51.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.173 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.173 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.173 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:51.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.174 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:51.174 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.174 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712894+0000", 2024-09-14T06:17:51.174 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.65e-07 2024-09-14T06:17:51.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.174 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712895+0000", 2024-09-14T06:17:51.174 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.325e-06 2024-09-14T06:17:51.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712899+0000", 2024-09-14T06:17:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.219e-06 2024-09-14T06:17:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712901+0000", 2024-09-14T06:17:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7310000000000002e-06 2024-09-14T06:17:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.839921+0000", 2024-09-14T06:17:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12701995199999999 2024-09-14T06:17:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840006+0000", 2024-09-14T06:17:51.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5063999999999997e-05 2024-09-14T06:17:51.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840285+0000", 2024-09-14T06:17:51.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027853300000000002 2024-09-14T06:17:51.176 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.176 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.176 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.177 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40411 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.177 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712909+0000", 2024-09-14T06:17:51.177 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.429285055999998, 2024-09-14T06:17:51.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.128251319, 2024-09-14T06:17:51.177 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.177 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.177 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.177 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.177 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.177 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.177 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40411 2024-09-14T06:17:51.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.177 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.177 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.178 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.178 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712909+0000", 2024-09-14T06:17:51.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.178 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.178 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712909+0000", 2024-09-14T06:17:51.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.178 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.178 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712910+0000", 2024-09-14T06:17:51.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.39e-07 2024-09-14T06:17:51.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.179 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712911+0000", 2024-09-14T06:17:51.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.37e-06 2024-09-14T06:17:51.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.179 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712914+0000", 2024-09-14T06:17:51.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4999999999999999e-06 2024-09-14T06:17:51.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.179 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712929+0000", 2024-09-14T06:17:51.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4406000000000001e-05 2024-09-14T06:17:51.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840856+0000", 2024-09-14T06:17:51.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.127927546 2024-09-14T06:17:51.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.840942+0000", 2024-09-14T06:17:51.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5232e-05 2024-09-14T06:17:51.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.181 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.181 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.181 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841161+0000", 2024-09-14T06:17:51.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021892600000000001 2024-09-14T06:17:51.181 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.181 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.181 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.181 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.181 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40413 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.181 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712938+0000", 2024-09-14T06:17:51.181 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.429256129999999, 2024-09-14T06:17:51.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.128612052, 2024-09-14T06:17:51.181 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.182 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.182 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.182 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.182 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.182 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.182 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40413 2024-09-14T06:17:51.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.182 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.182 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.182 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.182 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712938+0000", 2024-09-14T06:17:51.182 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.182 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.182 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.183 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712938+0000", 2024-09-14T06:17:51.183 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.183 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.183 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.183 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712939+0000", 2024-09-14T06:17:51.183 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1099999999999996e-07 2024-09-14T06:17:51.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.183 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.183 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.183 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712942+0000", 2024-09-14T06:17:51.183 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0920000000000002e-06 2024-09-14T06:17:51.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.183 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.184 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.184 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712949+0000", 2024-09-14T06:17:51.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.075e-06 2024-09-14T06:17:51.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.184 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.184 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712954+0000", 2024-09-14T06:17:51.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2229999999999998e-06 2024-09-14T06:17:51.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.184 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.184 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841192+0000", 2024-09-14T06:17:51.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12823817100000001 2024-09-14T06:17:51.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.185 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841280+0000", 2024-09-14T06:17:51.185 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8190999999999996e-05 2024-09-14T06:17:51.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.185 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841550+0000", 2024-09-14T06:17:51.185 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00026978899999999999 2024-09-14T06:17:51.185 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.185 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.185 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.185 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40415 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.186 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712962+0000", 2024-09-14T06:17:51.186 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.429231852999997, 2024-09-14T06:17:51.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12902585499999999, 2024-09-14T06:17:51.186 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.186 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.186 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.186 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.186 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.186 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.186 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40415 2024-09-14T06:17:51.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.186 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712962+0000", 2024-09-14T06:17:51.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712962+0000", 2024-09-14T06:17:51.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712963+0000", 2024-09-14T06:17:51.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:17:51.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712966+0000", 2024-09-14T06:17:51.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6649999999999999e-06 2024-09-14T06:17:51.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.188 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712972+0000", 2024-09-14T06:17:51.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4819999999999999e-06 2024-09-14T06:17:51.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.188 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712978+0000", 2024-09-14T06:17:51.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4299999999999997e-06 2024-09-14T06:17:51.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.188 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.189 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841581+0000", 2024-09-14T06:17:51.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12860382100000001 2024-09-14T06:17:51.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.189 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.189 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841665+0000", 2024-09-14T06:17:51.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3228999999999999e-05 2024-09-14T06:17:51.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.189 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.189 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.841988+0000", 2024-09-14T06:17:51.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032370599999999999 2024-09-14T06:17:51.189 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.189 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.189 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.190 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40417 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.190 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.712983+0000", 2024-09-14T06:17:51.190 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.429211119000001, 2024-09-14T06:17:51.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129525837, 2024-09-14T06:17:51.190 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.190 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.190 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.190 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.190 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.190 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.190 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40417 2024-09-14T06:17:51.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.191 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712983+0000", 2024-09-14T06:17:51.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.712983+0000", 2024-09-14T06:17:51.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713182+0000", 2024-09-14T06:17:51.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000198569 2024-09-14T06:17:51.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713186+0000", 2024-09-14T06:17:51.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6160000000000004e-06 2024-09-14T06:17:51.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713197+0000", 2024-09-14T06:17:51.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.046e-05 2024-09-14T06:17:51.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713204+0000", 2024-09-14T06:17:51.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.515e-06 2024-09-14T06:17:51.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842021+0000", 2024-09-14T06:17:51.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12881625199999999 2024-09-14T06:17:51.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842158+0000", 2024-09-14T06:17:51.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013778099999999999 2024-09-14T06:17:51.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842509+0000", 2024-09-14T06:17:51.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000350644 2024-09-14T06:17:51.194 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.194 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.194 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.194 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40419 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.194 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.713217+0000", 2024-09-14T06:17:51.194 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.428977582000002, 2024-09-14T06:17:51.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12982854599999999, 2024-09-14T06:17:51.194 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.194 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.194 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.194 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.195 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.195 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.195 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40419 2024-09-14T06:17:51.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.195 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.195 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.195 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713217+0000", 2024-09-14T06:17:51.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.195 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.195 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713217+0000", 2024-09-14T06:17:51.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713831+0000", 2024-09-14T06:17:51.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061477700000000005 2024-09-14T06:17:51.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713841+0000", 2024-09-14T06:17:51.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3789999999999995e-06 2024-09-14T06:17:51.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713853+0000", 2024-09-14T06:17:51.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1952e-05 2024-09-14T06:17:51.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713861+0000", 2024-09-14T06:17:51.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4500000000000004e-06 2024-09-14T06:17:51.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842560+0000", 2024-09-14T06:17:51.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12869836900000001 2024-09-14T06:17:51.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.842655+0000", 2024-09-14T06:17:51.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5617000000000003e-05 2024-09-14T06:17:51.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.198 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843045+0000", 2024-09-14T06:17:51.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00039000200000000002 2024-09-14T06:17:51.198 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.198 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.198 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.198 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40421 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.198 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.713882+0000", 2024-09-14T06:17:51.198 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.428312214999998, 2024-09-14T06:17:51.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12974492600000001, 2024-09-14T06:17:51.199 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.199 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.199 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.199 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.199 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.199 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.199 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40421 2024-09-14T06:17:51.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.199 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.199 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713882+0000", 2024-09-14T06:17:51.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.713882+0000", 2024-09-14T06:17:51.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714044+0000", 2024-09-14T06:17:51.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016224999999999999 2024-09-14T06:17:51.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714049+0000", 2024-09-14T06:17:51.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0150000000000003e-06 2024-09-14T06:17:51.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714068+0000", 2024-09-14T06:17:51.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8749e-05 2024-09-14T06:17:51.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714077+0000", 2024-09-14T06:17:51.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4379999999999995e-06 2024-09-14T06:17:51.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843079+0000", 2024-09-14T06:17:51.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129002958 2024-09-14T06:17:51.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843290+0000", 2024-09-14T06:17:51.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021005899999999999 2024-09-14T06:17:51.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843627+0000", 2024-09-14T06:17:51.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00033745699999999999 2024-09-14T06:17:51.203 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.203 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.203 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.203 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40423 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.203 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.714090+0000", 2024-09-14T06:17:51.203 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.428103843999999, 2024-09-14T06:17:51.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129886166, 2024-09-14T06:17:51.203 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.203 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.203 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.203 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.203 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.204 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.204 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40423 2024-09-14T06:17:51.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.204 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714090+0000", 2024-09-14T06:17:51.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714090+0000", 2024-09-14T06:17:51.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.205 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.205 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714456+0000", 2024-09-14T06:17:51.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000366058 2024-09-14T06:17:51.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.205 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.205 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714461+0000", 2024-09-14T06:17:51.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.228e-06 2024-09-14T06:17:51.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.205 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.205 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714469+0000", 2024-09-14T06:17:51.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6589999999999996e-06 2024-09-14T06:17:51.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714476+0000", 2024-09-14T06:17:51.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:17:51.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843659+0000", 2024-09-14T06:17:51.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129183137 2024-09-14T06:17:51.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843745+0000", 2024-09-14T06:17:51.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5354999999999998e-05 2024-09-14T06:17:51.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.843977+0000", 2024-09-14T06:17:51.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000231883 2024-09-14T06:17:51.207 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.207 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.207 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.207 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40425 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.207 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.714490+0000", 2024-09-14T06:17:51.207 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.427704648999999, 2024-09-14T06:17:51.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13064684600000001, 2024-09-14T06:17:51.207 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.208 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.208 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.208 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.208 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.208 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.208 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40425 2024-09-14T06:17:51.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.208 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.208 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.208 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.208 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714490+0000", 2024-09-14T06:17:51.208 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.209 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714490+0000", 2024-09-14T06:17:51.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.209 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714725+0000", 2024-09-14T06:17:51.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000235438 2024-09-14T06:17:51.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.209 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714729+0000", 2024-09-14T06:17:51.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8909999999999997e-06 2024-09-14T06:17:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714737+0000", 2024-09-14T06:17:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5329999999999994e-06 2024-09-14T06:17:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714742+0000", 2024-09-14T06:17:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9559999999999996e-06 2024-09-14T06:17:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.844184+0000", 2024-09-14T06:17:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.129441954 2024-09-14T06:17:51.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.211 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.211 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.211 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.844272+0000", 2024-09-14T06:17:51.211 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7151999999999995e-05 2024-09-14T06:17:51.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.211 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.211 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.211 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845136+0000", 2024-09-14T06:17:51.211 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00086492199999999996 2024-09-14T06:17:51.211 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.211 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.211 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.212 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.212 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40426 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.212 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.714753+0000", 2024-09-14T06:17:51.212 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.427441684999998, 2024-09-14T06:17:51.212 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.130693374, 2024-09-14T06:17:51.212 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.212 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.212 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.212 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.212 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.212 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.212 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40426 2024-09-14T06:17:51.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.213 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.213 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.213 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.213 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714753+0000", 2024-09-14T06:17:51.213 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.213 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.213 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.213 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.714753+0000", 2024-09-14T06:17:51.213 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.213 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.213 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.213 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715118+0000", 2024-09-14T06:17:51.213 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036512099999999999 2024-09-14T06:17:51.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.214 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.214 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.214 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715123+0000", 2024-09-14T06:17:51.214 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9910000000000002e-06 2024-09-14T06:17:51.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.214 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.214 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.214 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715141+0000", 2024-09-14T06:17:51.214 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8219999999999998e-05 2024-09-14T06:17:51.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.214 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.214 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.214 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715149+0000", 2024-09-14T06:17:51.215 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8979999999999996e-06 2024-09-14T06:17:51.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.215 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.215 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845170+0000", 2024-09-14T06:17:51.215 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13002106799999999 2024-09-14T06:17:51.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.215 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.215 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845249+0000", 2024-09-14T06:17:51.215 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9163000000000001e-05 2024-09-14T06:17:51.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.216 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.216 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845446+0000", 2024-09-14T06:17:51.216 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196913 2024-09-14T06:17:51.216 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.216 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.216 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.216 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.216 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40428 3.15 3:aff4d375:::rbd_data.115274ed0aa8.000000000000000a:14 [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.216 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.715166+0000", 2024-09-14T06:17:51.216 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.427028733999997, 2024-09-14T06:17:51.216 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13071775199999999, 2024-09-14T06:17:51.216 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.217 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.217 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.217 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.217 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.217 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.217 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40428 2024-09-14T06:17:51.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.217 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.217 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.217 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.217 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715166+0000", 2024-09-14T06:17:51.217 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.217 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.218 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.218 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715166+0000", 2024-09-14T06:17:51.218 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.218 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.218 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.218 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715361+0000", 2024-09-14T06:17:51.218 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00019536400000000001 2024-09-14T06:17:51.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.218 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.218 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.218 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715366+0000", 2024-09-14T06:17:51.218 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4589999999999997e-06 2024-09-14T06:17:51.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.219 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.219 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.219 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715387+0000", 2024-09-14T06:17:51.219 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0829e-05 2024-09-14T06:17:51.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.219 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.219 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.219 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715402+0000", 2024-09-14T06:17:51.219 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4429e-05 2024-09-14T06:17:51.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.219 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.219 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.219 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845476+0000", 2024-09-14T06:17:51.219 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13007447999999999 2024-09-14T06:17:51.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.220 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.220 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.220 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845557+0000", 2024-09-14T06:17:51.220 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1100000000000006e-05 2024-09-14T06:17:51.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.220 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.220 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.220 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.845883+0000", 2024-09-14T06:17:51.220 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00032609099999999998 2024-09-14T06:17:51.220 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.220 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.220 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.220 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.220 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40430 3.15 3.aecb2ff5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.221 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.715416+0000", 2024-09-14T06:17:51.221 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.426778269000003, 2024-09-14T06:17:51.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13113939699999999, 2024-09-14T06:17:51.221 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.221 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.221 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:51.221 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.221 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.221 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.221 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40430 2024-09-14T06:17:51.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.221 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.221 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.221 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.222 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715416+0000", 2024-09-14T06:17:51.222 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.222 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.222 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715416+0000", 2024-09-14T06:17:51.222 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.222 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.222 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.715995+0000", 2024-09-14T06:17:51.222 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057881100000000004 2024-09-14T06:17:51.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.222 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.223 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716000+0000", 2024-09-14T06:17:51.223 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.677e-06 2024-09-14T06:17:51.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.223 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.223 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716013+0000", 2024-09-14T06:17:51.223 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2326e-05 2024-09-14T06:17:51.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.223 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.223 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716021+0000", 2024-09-14T06:17:51.223 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9750000000000007e-06 2024-09-14T06:17:51.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.224 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.224 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846538+0000", 2024-09-14T06:17:51.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13051717400000001 2024-09-14T06:17:51.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.224 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.224 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846555+0000", 2024-09-14T06:17:51.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7433999999999999e-05 2024-09-14T06:17:51.224 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.225 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.225 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.225 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.225 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40432 3.15 3.aecb2ff5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.225 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:51.225 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.426158630000003, 2024-09-14T06:17:51.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13055824499999999, 2024-09-14T06:17:51.225 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.225 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.225 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:51.226 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.226 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.226 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.226 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40432 2024-09-14T06:17:51.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.226 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.226 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.226 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.226 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:51.226 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.226 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.226 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.226 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:51.226 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.227 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716036+0000", 2024-09-14T06:17:51.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9599999999999998e-07 2024-09-14T06:17:51.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.227 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716040+0000", 2024-09-14T06:17:51.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.399e-06 2024-09-14T06:17:51.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.227 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716048+0000", 2024-09-14T06:17:51.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2179999999999999e-06 2024-09-14T06:17:51.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.228 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.228 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716054+0000", 2024-09-14T06:17:51.228 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2110000000000003e-06 2024-09-14T06:17:51.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.228 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.228 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846586+0000", 2024-09-14T06:17:51.228 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.130531921 2024-09-14T06:17:51.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.228 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.228 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846594+0000", 2024-09-14T06:17:51.228 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7000000000000008e-06 2024-09-14T06:17:51.229 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.229 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.229 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.229 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.229 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:40434 3.15 3.aecb2ff5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.229 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.716065+0000", 2024-09-14T06:17:51.229 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.426128802999997, 2024-09-14T06:17:51.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.13055924899999999, 2024-09-14T06:17:51.229 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.229 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.229 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:51.229 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.229 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.229 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.230 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 40434 2024-09-14T06:17:51.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.230 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.230 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.230 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716065+0000", 2024-09-14T06:17:51.230 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.230 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.230 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.716065+0000", 2024-09-14T06:17:51.230 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.231 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.723984+0000", 2024-09-14T06:17:51.231 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0079189670000000007 2024-09-14T06:17:51.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.231 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.723991+0000", 2024-09-14T06:17:51.231 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1759999999999998e-06 2024-09-14T06:17:51.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.231 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.724003+0000", 2024-09-14T06:17:51.231 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1914e-05 2024-09-14T06:17:51.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.232 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.232 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.724013+0000", 2024-09-14T06:17:51.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441999999999999e-05 2024-09-14T06:17:51.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.232 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.232 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846617+0000", 2024-09-14T06:17:51.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.12260452199999999 2024-09-14T06:17:51.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.232 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.232 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.846625+0000", 2024-09-14T06:17:51.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2280000000000001e-06 2024-09-14T06:17:51.233 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.233 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.233 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.233 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.233 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.233 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:51.234 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:17:51.235 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:51.228+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:51.335 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:51.328+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:51.403 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:51.403 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:51.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.403 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:67300 3.10 3:0f2ebeaf:::rbd_object_map.1152cd930a87:head [call lock.assert_locked in=44b,call rbd.object_map_snap_remove in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T06:17:51.403 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:51.402442+0000", 2024-09-14T06:17:51.403 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.00065955500000000004, 2024-09-14T06:17:51.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00067938600000000003, 2024-09-14T06:17:51.403 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.404 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.404 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "waiting for sub ops", 2024-09-14T06:17:51.404 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.404 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.404 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.404 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 67300 2024-09-14T06:17:51.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.404 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:51.402442+0000", 2024-09-14T06:17:51.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:51.402442+0000", 2024-09-14T06:17:51.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:51.402443+0000", 2024-09-14T06:17:51.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9599999999999998e-07 2024-09-14T06:17:51.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:51.402448+0000", 2024-09-14T06:17:51.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.403e-06 2024-09-14T06:17:51.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:51.402459+0000", 2024-09-14T06:17:51.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1078e-05 2024-09-14T06:17:51.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:51.402464+0000", 2024-09-14T06:17:51.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.699e-06 2024-09-14T06:17:51.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:51.402512+0000", 2024-09-14T06:17:51.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7698000000000003e-05 2024-09-14T06:17:51.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:51.402566+0000", 2024-09-14T06:17:51.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3862e-05 2024-09-14T06:17:51.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:17:51.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:51.402708+0000", 2024-09-14T06:17:51.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000141729 2024-09-14T06:17:51.407 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.408 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.408 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.408 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.408 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:17:51.408 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 1 2024-09-14T06:17:51.408 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:51.416 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:17:51.459 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:51.457+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:51.462 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-14T06:17:51.457+0000 7f4add550640 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-14T06:17:51.463 INFO:teuthology.orchestra.run.smithi039.stdout:moved 1 / 82 (1.21951%) 2024-09-14T06:17:51.463 INFO:teuthology.orchestra.run.smithi039.stdout:avg 10.25 2024-09-14T06:17:51.463 INFO:teuthology.orchestra.run.smithi039.stdout:stddev 2.63391 -> 2.90474 (expected baseline 2.99479) 2024-09-14T06:17:51.463 INFO:teuthology.orchestra.run.smithi039.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-14T06:17:51.463 INFO:teuthology.orchestra.run.smithi039.stdout:max osd.3 with 15 -> 16 pgs (1.46341 -> 1.56098 * mean) 2024-09-14T06:17:51.463 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:17:51.463 INFO:teuthology.orchestra.run.smithi039.stdout:oload 110 2024-09-14T06:17:51.463 INFO:teuthology.orchestra.run.smithi039.stdout:max_change 0.05 2024-09-14T06:17:51.463 INFO:teuthology.orchestra.run.smithi039.stdout:max_change_osds 4 2024-09-14T06:17:51.464 INFO:teuthology.orchestra.run.smithi039.stdout:average_utilization 0.0005 2024-09-14T06:17:51.464 INFO:teuthology.orchestra.run.smithi039.stdout:overload_utilization 0.0006 2024-09-14T06:17:51.464 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 weight 1.0000 -> 0.9500 2024-09-14T06:17:51.464 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 weight 1.0000 -> 0.9500 2024-09-14T06:17:51.464 INFO:teuthology.orchestra.run.smithi039.stderr:no change 2024-09-14T06:17:51.525 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:51.525 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:51.525 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:51.525 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:51.525 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:51.537 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:17:51.538 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:51.533+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:51.637 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:51.632+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:51.704 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:51.704 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:51.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:51.704 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:51.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.704 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:17:51.704 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:17:51.704 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 107.623825306, 2024-09-14T06:17:51.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.56009052699999995, 2024-09-14T06:17:51.705 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.705 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.705 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:51.705 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.705 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:17:51.705 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:17:51.705 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 1 2024-09-14T06:17:51.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.705 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:17:51.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:17:51.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077065+0000", 2024-09-14T06:17:51.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6429999999999999e-06 2024-09-14T06:17:51.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077079+0000", 2024-09-14T06:17:51.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4447000000000001e-05 2024-09-14T06:17:51.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077103+0000", 2024-09-14T06:17:51.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3856e-05 2024-09-14T06:17:51.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077120+0000", 2024-09-14T06:17:51.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7337e-05 2024-09-14T06:17:51.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077255+0000", 2024-09-14T06:17:51.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000134652 2024-09-14T06:17:51.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:51.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077274+0000", 2024-09-14T06:17:51.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9003000000000001e-05 2024-09-14T06:17:51.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634272+0000", 2024-09-14T06:17:51.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.55699813099999995 2024-09-14T06:17:51.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634591+0000", 2024-09-14T06:17:51.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031910800000000002 2024-09-14T06:17:51.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:17:51.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634892+0000", 2024-09-14T06:17:51.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00030066400000000002 2024-09-14T06:17:51.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:51.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636345+0000", 2024-09-14T06:17:51.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001452896 2024-09-14T06:17:51.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:51.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636974+0000", 2024-09-14T06:17:51.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00062927800000000004 2024-09-14T06:17:51.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:51.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637071+0000", 2024-09-14T06:17:51.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7178999999999999e-05 2024-09-14T06:17:51.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637153+0000", 2024-09-14T06:17:51.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1333000000000002e-05 2024-09-14T06:17:51.711 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.711 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.711 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.711 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38947 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 159744~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.711 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:17:51.711 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020767018999997, 2024-09-14T06:17:51.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085496275999999996, 2024-09-14T06:17:51.711 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.712 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.712 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.712 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.712 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.712 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.712 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38947 2024-09-14T06:17:51.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.712 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:17:51.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:17:51.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680121+0000", 2024-09-14T06:17:51.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.82e-07 2024-09-14T06:17:51.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680123+0000", 2024-09-14T06:17:51.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5629999999999999e-06 2024-09-14T06:17:51.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680133+0000", 2024-09-14T06:17:51.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0890000000000006e-06 2024-09-14T06:17:51.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680138+0000", 2024-09-14T06:17:51.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.728e-06 2024-09-14T06:17:51.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765422+0000", 2024-09-14T06:17:51.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085283701000000003 2024-09-14T06:17:51.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765496+0000", 2024-09-14T06:17:51.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4276000000000005e-05 2024-09-14T06:17:51.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765617+0000", 2024-09-14T06:17:51.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000120437 2024-09-14T06:17:51.716 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.716 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.716 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.716 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38990 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 163840~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.717 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:17:51.717 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020741035999997, 2024-09-14T06:17:51.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085695970999999996, 2024-09-14T06:17:51.717 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.717 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.717 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.717 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.717 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.717 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.717 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38990 2024-09-14T06:17:51.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.717 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:17:51.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:17:51.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680147+0000", 2024-09-14T06:17:51.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7599999999999997e-07 2024-09-14T06:17:51.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680149+0000", 2024-09-14T06:17:51.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4700000000000001e-06 2024-09-14T06:17:51.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680156+0000", 2024-09-14T06:17:51.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.058e-06 2024-09-14T06:17:51.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680162+0000", 2024-09-14T06:17:51.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7080000000000002e-06 2024-09-14T06:17:51.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765657+0000", 2024-09-14T06:17:51.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085494566999999994 2024-09-14T06:17:51.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765741+0000", 2024-09-14T06:17:51.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4782000000000002e-05 2024-09-14T06:17:51.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765842+0000", 2024-09-14T06:17:51.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010090999999999999 2024-09-14T06:17:51.721 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.721 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.721 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.721 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38991 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 172032~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.721 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:17:51.721 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020717079000001, 2024-09-14T06:17:51.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085875878000000003, 2024-09-14T06:17:51.722 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.722 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.722 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.722 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.722 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.722 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.722 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38991 2024-09-14T06:17:51.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.722 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:17:51.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:17:51.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680171+0000", 2024-09-14T06:17:51.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4299999999999998e-07 2024-09-14T06:17:51.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680173+0000", 2024-09-14T06:17:51.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1639999999999999e-06 2024-09-14T06:17:51.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680181+0000", 2024-09-14T06:17:51.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0539999999999995e-06 2024-09-14T06:17:51.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680187+0000", 2024-09-14T06:17:51.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7529999999999998e-06 2024-09-14T06:17:51.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765875+0000", 2024-09-14T06:17:51.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085688722999999994 2024-09-14T06:17:51.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765948+0000", 2024-09-14T06:17:51.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2126999999999996e-05 2024-09-14T06:17:51.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766046+0000", 2024-09-14T06:17:51.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8614000000000001e-05 2024-09-14T06:17:51.725 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.725 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.726 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.726 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38992 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 180224~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.726 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:51.726 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020691802999998, 2024-09-14T06:17:51.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086050225999999994, 2024-09-14T06:17:51.726 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.726 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.726 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.726 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.726 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.726 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.727 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38992 2024-09-14T06:17:51.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.727 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:51.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:51.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:51.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1199999999999998e-07 2024-09-14T06:17:51.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680199+0000", 2024-09-14T06:17:51.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:17:51.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680227+0000", 2024-09-14T06:17:51.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8238999999999998e-05 2024-09-14T06:17:51.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680234+0000", 2024-09-14T06:17:51.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0550000000000002e-06 2024-09-14T06:17:51.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766077+0000", 2024-09-14T06:17:51.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085843002000000002 2024-09-14T06:17:51.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766155+0000", 2024-09-14T06:17:51.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8112e-05 2024-09-14T06:17:51.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766246+0000", 2024-09-14T06:17:51.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0785999999999994e-05 2024-09-14T06:17:51.730 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.730 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.730 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.730 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38993 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 188416~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.731 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:17:51.731 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020600451999996, 2024-09-14T06:17:51.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086166131000000007, 2024-09-14T06:17:51.731 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.731 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.731 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.731 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.731 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.731 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.731 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38993 2024-09-14T06:17:51.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.731 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:17:51.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:17:51.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680288+0000", 2024-09-14T06:17:51.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7199999999999998e-07 2024-09-14T06:17:51.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680291+0000", 2024-09-14T06:17:51.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7280000000000001e-06 2024-09-14T06:17:51.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680301+0000", 2024-09-14T06:17:51.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3360000000000001e-06 2024-09-14T06:17:51.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680308+0000", 2024-09-14T06:17:51.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1389999999999998e-06 2024-09-14T06:17:51.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766280+0000", 2024-09-14T06:17:51.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085972111000000004 2024-09-14T06:17:51.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766357+0000", 2024-09-14T06:17:51.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7477000000000004e-05 2024-09-14T06:17:51.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766453+0000", 2024-09-14T06:17:51.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5668000000000006e-05 2024-09-14T06:17:51.735 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.735 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.735 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.735 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38994 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 196608~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.735 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:51.735 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020510645000002, 2024-09-14T06:17:51.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086290477000000004, 2024-09-14T06:17:51.735 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.735 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.735 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.735 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.735 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.736 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.736 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38994 2024-09-14T06:17:51.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.736 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:51.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:51.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:51.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3e-07 2024-09-14T06:17:51.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680381+0000", 2024-09-14T06:17:51.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.439e-06 2024-09-14T06:17:51.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680438+0000", 2024-09-14T06:17:51.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6693000000000001e-05 2024-09-14T06:17:51.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680446+0000", 2024-09-14T06:17:51.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9160000000000001e-06 2024-09-14T06:17:51.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766490+0000", 2024-09-14T06:17:51.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086044617000000004 2024-09-14T06:17:51.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766563+0000", 2024-09-14T06:17:51.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2430999999999998e-05 2024-09-14T06:17:51.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766667+0000", 2024-09-14T06:17:51.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000104651 2024-09-14T06:17:51.739 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.739 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.739 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.739 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38995 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 204800~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.739 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:17:51.739 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020427619000003, 2024-09-14T06:17:51.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086415360999999996, 2024-09-14T06:17:51.740 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.740 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.740 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.740 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.740 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.740 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.740 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38995 2024-09-14T06:17:51.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.740 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:17:51.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:17:51.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680461+0000", 2024-09-14T06:17:51.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:17:51.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680464+0000", 2024-09-14T06:17:51.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4929999999999998e-06 2024-09-14T06:17:51.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680479+0000", 2024-09-14T06:17:51.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5322999999999999e-05 2024-09-14T06:17:51.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680496+0000", 2024-09-14T06:17:51.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6801000000000001e-05 2024-09-14T06:17:51.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766699+0000", 2024-09-14T06:17:51.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086203265000000001 2024-09-14T06:17:51.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766777+0000", 2024-09-14T06:17:51.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7284000000000001e-05 2024-09-14T06:17:51.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766875+0000", 2024-09-14T06:17:51.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8461000000000005e-05 2024-09-14T06:17:51.743 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.743 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.744 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.744 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38996 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 212992~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.744 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:17:51.744 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020380179999997, 2024-09-14T06:17:51.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086576474, 2024-09-14T06:17:51.744 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.744 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.744 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.744 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.744 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.744 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.744 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38996 2024-09-14T06:17:51.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.745 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:17:51.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:17:51.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680508+0000", 2024-09-14T06:17:51.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1799999999999995e-07 2024-09-14T06:17:51.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680511+0000", 2024-09-14T06:17:51.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9739999999999998e-06 2024-09-14T06:17:51.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680520+0000", 2024-09-14T06:17:51.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.329e-06 2024-09-14T06:17:51.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680528+0000", 2024-09-14T06:17:51.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4660000000000002e-06 2024-09-14T06:17:51.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766907+0000", 2024-09-14T06:17:51.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086379529999999996 2024-09-14T06:17:51.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766987+0000", 2024-09-14T06:17:51.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0280999999999999e-05 2024-09-14T06:17:51.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767084+0000", 2024-09-14T06:17:51.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6275999999999998e-05 2024-09-14T06:17:51.748 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.748 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.748 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.748 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38997 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 221184~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.748 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:51.748 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020350839000002, 2024-09-14T06:17:51.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086757327999999995, 2024-09-14T06:17:51.748 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.749 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.749 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.749 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.749 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.749 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.749 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38997 2024-09-14T06:17:51.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.749 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:51.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:51.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:51.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:17:51.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680540+0000", 2024-09-14T06:17:51.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6910000000000002e-06 2024-09-14T06:17:51.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680548+0000", 2024-09-14T06:17:51.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7339999999999997e-06 2024-09-14T06:17:51.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680555+0000", 2024-09-14T06:17:51.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0900000000000001e-06 2024-09-14T06:17:51.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767116+0000", 2024-09-14T06:17:51.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086561551 2024-09-14T06:17:51.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767192+0000", 2024-09-14T06:17:51.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5504000000000001e-05 2024-09-14T06:17:51.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767294+0000", 2024-09-14T06:17:51.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000102308 2024-09-14T06:17:51.752 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.752 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.752 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.753 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38998 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 229376~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.753 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:17:51.753 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020324228, 2024-09-14T06:17:51.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087010193999999999, 2024-09-14T06:17:51.753 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.753 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.753 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.753 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.753 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.753 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.753 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38998 2024-09-14T06:17:51.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.754 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:17:51.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:17:51.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680564+0000", 2024-09-14T06:17:51.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1800000000000001e-07 2024-09-14T06:17:51.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680566+0000", 2024-09-14T06:17:51.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.503e-06 2024-09-14T06:17:51.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680574+0000", 2024-09-14T06:17:51.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3979999999999999e-06 2024-09-14T06:17:51.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680580+0000", 2024-09-14T06:17:51.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4859999999999997e-06 2024-09-14T06:17:51.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767359+0000", 2024-09-14T06:17:51.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086779447999999995 2024-09-14T06:17:51.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767445+0000", 2024-09-14T06:17:51.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6007000000000005e-05 2024-09-14T06:17:51.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767573+0000", 2024-09-14T06:17:51.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012793400000000001 2024-09-14T06:17:51.757 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.757 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.757 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.757 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38999 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 237568~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.757 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:17:51.757 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020295363999999, 2024-09-14T06:17:51.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087211699000000004, 2024-09-14T06:17:51.757 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.758 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.758 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.760 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.760 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.760 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.760 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38999 2024-09-14T06:17:51.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.760 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:17:51.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:17:51.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680593+0000", 2024-09-14T06:17:51.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7700000000000005e-07 2024-09-14T06:17:51.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680600+0000", 2024-09-14T06:17:51.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7130000000000005e-06 2024-09-14T06:17:51.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680610+0000", 2024-09-14T06:17:51.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.55e-06 2024-09-14T06:17:51.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680616+0000", 2024-09-14T06:17:51.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3069999999999999e-06 2024-09-14T06:17:51.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767607+0000", 2024-09-14T06:17:51.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086990703000000003 2024-09-14T06:17:51.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767682+0000", 2024-09-14T06:17:51.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4944999999999995e-05 2024-09-14T06:17:51.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767804+0000", 2024-09-14T06:17:51.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012200400000000001 2024-09-14T06:17:51.764 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.764 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.764 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.764 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39000 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 245760~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.764 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:51.764 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020261742000002, 2024-09-14T06:17:51.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087408649000000005, 2024-09-14T06:17:51.764 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.764 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.764 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.764 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.765 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.765 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.765 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39000 2024-09-14T06:17:51.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.765 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:51.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:51.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:51.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1699999999999998e-07 2024-09-14T06:17:51.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680629+0000", 2024-09-14T06:17:51.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5469999999999999e-06 2024-09-14T06:17:51.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680636+0000", 2024-09-14T06:17:51.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.973e-06 2024-09-14T06:17:51.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680642+0000", 2024-09-14T06:17:51.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0349999999999998e-06 2024-09-14T06:17:51.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767839+0000", 2024-09-14T06:17:51.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087197067000000003 2024-09-14T06:17:51.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767913+0000", 2024-09-14T06:17:51.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3832999999999996e-05 2024-09-14T06:17:51.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768034+0000", 2024-09-14T06:17:51.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012167699999999999 2024-09-14T06:17:51.768 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.768 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.768 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.768 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39001 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 253952~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.768 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:17:51.768 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020237299000001, 2024-09-14T06:17:51.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087623312999999994, 2024-09-14T06:17:51.769 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.769 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.769 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.769 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.769 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.769 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.769 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39001 2024-09-14T06:17:51.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.769 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:17:51.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:17:51.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680651+0000", 2024-09-14T06:17:51.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7700000000000004e-07 2024-09-14T06:17:51.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680653+0000", 2024-09-14T06:17:51.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5739999999999998e-06 2024-09-14T06:17:51.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680702+0000", 2024-09-14T06:17:51.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8476000000000001e-05 2024-09-14T06:17:51.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680709+0000", 2024-09-14T06:17:51.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8170000000000001e-06 2024-09-14T06:17:51.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768066+0000", 2024-09-14T06:17:51.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087357852999999999 2024-09-14T06:17:51.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768163+0000", 2024-09-14T06:17:51.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6869999999999999e-05 2024-09-14T06:17:51.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768273+0000", 2024-09-14T06:17:51.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000110146 2024-09-14T06:17:51.773 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.773 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.773 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.773 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39002 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 262144~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.773 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:17:51.773 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020157193000003, 2024-09-14T06:17:51.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088150478000000004, 2024-09-14T06:17:51.773 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.773 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.773 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.773 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.773 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.774 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.774 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39002 2024-09-14T06:17:51.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.774 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:17:51.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:17:51.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680739+0000", 2024-09-14T06:17:51.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3979999999999999e-06 2024-09-14T06:17:51.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680742+0000", 2024-09-14T06:17:51.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8349999999999997e-06 2024-09-14T06:17:51.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680751+0000", 2024-09-14T06:17:51.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8599999999999999e-06 2024-09-14T06:17:51.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680758+0000", 2024-09-14T06:17:51.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4790000000000001e-06 2024-09-14T06:17:51.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768307+0000", 2024-09-14T06:17:51.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087549342000000002 2024-09-14T06:17:51.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768392+0000", 2024-09-14T06:17:51.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.441e-05 2024-09-14T06:17:51.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768881+0000", 2024-09-14T06:17:51.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000489154 2024-09-14T06:17:51.777 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.777 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.777 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.777 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39003 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 270336~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.777 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:51.778 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020104902999996, 2024-09-14T06:17:51.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088375555999999994, 2024-09-14T06:17:51.778 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.778 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.778 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.778 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.778 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.778 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.778 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39003 2024-09-14T06:17:51.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.778 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:51.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:51.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:51.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7000000000000004e-07 2024-09-14T06:17:51.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680786+0000", 2024-09-14T06:17:51.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:17:51.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680794+0000", 2024-09-14T06:17:51.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9840000000000001e-06 2024-09-14T06:17:51.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680800+0000", 2024-09-14T06:17:51.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1550000000000004e-06 2024-09-14T06:17:51.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768965+0000", 2024-09-14T06:17:51.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088165091000000001 2024-09-14T06:17:51.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769059+0000", 2024-09-14T06:17:51.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3923000000000002e-05 2024-09-14T06:17:51.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769158+0000", 2024-09-14T06:17:51.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8820000000000006e-05 2024-09-14T06:17:51.782 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.782 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.782 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.782 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39004 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 278528~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.782 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:17:51.782 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020077215000001, 2024-09-14T06:17:51.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088499624999999998, 2024-09-14T06:17:51.782 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.782 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.783 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.783 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.783 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.783 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.783 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39004 2024-09-14T06:17:51.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.783 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:17:51.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:17:51.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680811+0000", 2024-09-14T06:17:51.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:17:51.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680814+0000", 2024-09-14T06:17:51.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:17:51.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680821+0000", 2024-09-14T06:17:51.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9009999999999997e-06 2024-09-14T06:17:51.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680826+0000", 2024-09-14T06:17:51.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5350000000000004e-06 2024-09-14T06:17:51.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769189+0000", 2024-09-14T06:17:51.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088363024999999998 2024-09-14T06:17:51.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769246+0000", 2024-09-14T06:17:51.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7017e-05 2024-09-14T06:17:51.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769310+0000", 2024-09-14T06:17:51.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3541999999999996e-05 2024-09-14T06:17:51.786 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.786 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39005 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 286720~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020053130999997, 2024-09-14T06:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088681092000000003, 2024-09-14T06:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.788 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39005 2024-09-14T06:17:51.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.788 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:17:51.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:17:51.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680835+0000", 2024-09-14T06:17:51.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2699999999999999e-07 2024-09-14T06:17:51.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680837+0000", 2024-09-14T06:17:51.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.413e-06 2024-09-14T06:17:51.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680844+0000", 2024-09-14T06:17:51.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8820000000000003e-06 2024-09-14T06:17:51.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680850+0000", 2024-09-14T06:17:51.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5169999999999999e-06 2024-09-14T06:17:51.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769337+0000", 2024-09-14T06:17:51.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088487471999999998 2024-09-14T06:17:51.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769404+0000", 2024-09-14T06:17:51.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7111000000000002e-05 2024-09-14T06:17:51.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.791 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.791 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769515+0000", 2024-09-14T06:17:51.791 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011116999999999999 2024-09-14T06:17:51.791 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.791 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.791 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.791 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39006 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 294912~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.791 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:17:51.791 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020029968000003, 2024-09-14T06:17:51.791 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088886572999999997, 2024-09-14T06:17:51.791 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.792 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.792 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.792 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.792 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.792 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.792 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39006 2024-09-14T06:17:51.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.792 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:17:51.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:17:51.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680858+0000", 2024-09-14T06:17:51.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7e-07 2024-09-14T06:17:51.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680860+0000", 2024-09-14T06:17:51.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:17:51.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680867+0000", 2024-09-14T06:17:51.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0430000000000002e-06 2024-09-14T06:17:51.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680873+0000", 2024-09-14T06:17:51.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7389999999999996e-06 2024-09-14T06:17:51.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769547+0000", 2024-09-14T06:17:51.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088673830999999995 2024-09-14T06:17:51.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769631+0000", 2024-09-14T06:17:51.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3660999999999997e-05 2024-09-14T06:17:51.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769744+0000", 2024-09-14T06:17:51.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000113482 2024-09-14T06:17:51.795 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.795 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.795 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.796 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39007 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 303104~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:51.796 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:17:51.796 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.020006555999998, 2024-09-14T06:17:51.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.089077397000000003, 2024-09-14T06:17:51.796 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:51.796 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:51.796 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:51.796 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:51.796 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:51.796 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:51.796 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39007 2024-09-14T06:17:51.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.796 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:51.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:51.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:17:51.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:51.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:17:51.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:51.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:51.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687953+0000", 2024-09-14T06:17:51.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00707203 2024-09-14T06:17:51.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:51.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687960+0000", 2024-09-14T06:17:51.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9890000000000003e-06 2024-09-14T06:17:51.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:51.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687974+0000", 2024-09-14T06:17:51.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4226999999999999e-05 2024-09-14T06:17:51.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:51.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687983+0000", 2024-09-14T06:17:51.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9800000000000004e-06 2024-09-14T06:17:51.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:51.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769775+0000", 2024-09-14T06:17:51.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.081792089999999998 2024-09-14T06:17:51.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:51.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769849+0000", 2024-09-14T06:17:51.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3925999999999997e-05 2024-09-14T06:17:51.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:51.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:51.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:51.800 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769958+0000", 2024-09-14T06:17:51.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010915499999999999 2024-09-14T06:17:51.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.800 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:51.800 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:51.800 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:51.802 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:17:51.802 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:51.797+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:51.902 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:51.897+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:51.973 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:51.974 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:51.974 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:51.974 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:51.988 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:17:52.032 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:52.029+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:52.104 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:52.101+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:52.156 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:52.157 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:52.157 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:52.157 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:52.157 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:52.170 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:17:52.212 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:52.209+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:52.216 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:17:52.306 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:52.300+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:52.362 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:52.362 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:52.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:52.362 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:52.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.363 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3916 3.16 3:6ab8b1d8:::rbd_data.1152bd92e04f.0000000000000002:head [stat,truncate 1590169] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:17:52.363 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:17:52.363 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 84.781931643999997, 2024-09-14T06:17:52.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022958078999999999, 2024-09-14T06:17:52.363 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.363 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.363 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.363 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:52.363 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:52.363 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:52.363 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3916 2024-09-14T06:17:52.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.363 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:17:52.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:17:52.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579203+0000", 2024-09-14T06:17:52.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.748e-06 2024-09-14T06:17:52.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579208+0000", 2024-09-14T06:17:52.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:17:52.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579223+0000", 2024-09-14T06:17:52.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.508e-05 2024-09-14T06:17:52.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579232+0000", 2024-09-14T06:17:52.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9719999999999998e-06 2024-09-14T06:17:52.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579307+0000", 2024-09-14T06:17:52.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5028000000000001e-05 2024-09-14T06:17:52.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579412+0000", 2024-09-14T06:17:52.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010525400000000001 2024-09-14T06:17:52.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:17:52.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579535+0000", 2024-09-14T06:17:52.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122726 2024-09-14T06:17:52.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:52.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.581415+0000", 2024-09-14T06:17:52.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018800360000000001 2024-09-14T06:17:52.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:52.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602017+0000", 2024-09-14T06:17:52.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.020602425000000001 2024-09-14T06:17:52.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602096+0000", 2024-09-14T06:17:52.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8968999999999996e-05 2024-09-14T06:17:52.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602159+0000", 2024-09-14T06:17:52.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2824999999999996e-05 2024-09-14T06:17:52.368 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.369 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.369 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.369 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:7151 3.e 3:7793b17d:::rbd_data.1152f87cbac6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:17:52.369 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:17:52.369 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.031535777000002, 2024-09-14T06:17:52.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.029556783, 2024-09-14T06:17:52.369 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.369 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.369 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.369 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:52.372 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:52.372 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:52.372 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 7151 2024-09-14T06:17:52.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.372 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:17:52.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:17:52.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329599+0000", 2024-09-14T06:17:52.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5270000000000001e-06 2024-09-14T06:17:52.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329610+0000", 2024-09-14T06:17:52.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0431000000000001e-05 2024-09-14T06:17:52.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329634+0000", 2024-09-14T06:17:52.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3617000000000001e-05 2024-09-14T06:17:52.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329644+0000", 2024-09-14T06:17:52.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.095e-05 2024-09-14T06:17:52.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329712+0000", 2024-09-14T06:17:52.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7067e-05 2024-09-14T06:17:52.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329827+0000", 2024-09-14T06:17:52.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011498 2024-09-14T06:17:52.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:17:52.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.334271+0000", 2024-09-14T06:17:52.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044445839999999997 2024-09-14T06:17:52.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:52.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.358967+0000", 2024-09-14T06:17:52.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024696356999999999 2024-09-14T06:17:52.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:52.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359027+0000", 2024-09-14T06:17:52.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0006000000000001e-05 2024-09-14T06:17:52.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359099+0000", 2024-09-14T06:17:52.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1052999999999999e-05 2024-09-14T06:17:52.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359154+0000", 2024-09-14T06:17:52.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5210999999999997e-05 2024-09-14T06:17:52.378 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.378 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.378 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.378 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:7151 3.e e32/29 ondisk, result = 0)", 2024-09-14T06:17:52.378 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:17:52.378 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.011591343000006, 2024-09-14T06:17:52.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0096251340000000005, 2024-09-14T06:17:52.378 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.378 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.378 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:52.378 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:17:52.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:17:52.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349543+0000", 2024-09-14T06:17:52.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:17:52.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349552+0000", 2024-09-14T06:17:52.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1039999999999996e-06 2024-09-14T06:17:52.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349579+0000", 2024-09-14T06:17:52.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6449999999999999e-05 2024-09-14T06:17:52.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349593+0000", 2024-09-14T06:17:52.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4099e-05 2024-09-14T06:17:52.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359003+0000", 2024-09-14T06:17:52.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0094096119999999995 2024-09-14T06:17:52.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359014+0000", 2024-09-14T06:17:52.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0871e-05 2024-09-14T06:17:52.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.382 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359167+0000", 2024-09-14T06:17:52.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015300700000000001 2024-09-14T06:17:52.382 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.382 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.382 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.382 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15619 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'124, mlcod=36'119)", 2024-09-14T06:17:52.382 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:52.382 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 76.063278181000001, 2024-09-14T06:17:52.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061545389999999997, 2024-09-14T06:17:52.382 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.382 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.383 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.383 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:52.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:52.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:52.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3399999999999998e-07 2024-09-14T06:17:52.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297861+0000", 2024-09-14T06:17:52.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7640000000000002e-06 2024-09-14T06:17:52.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297876+0000", 2024-09-14T06:17:52.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4881e-05 2024-09-14T06:17:52.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297884+0000", 2024-09-14T06:17:52.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.588e-06 2024-09-14T06:17:52.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298294+0000", 2024-09-14T06:17:52.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00041011200000000002 2024-09-14T06:17:52.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298317+0000", 2024-09-14T06:17:52.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3417999999999998e-05 2024-09-14T06:17:52.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.386 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.386 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303972+0000", 2024-09-14T06:17:52.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056548680000000004 2024-09-14T06:17:52.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.386 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.386 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304009+0000", 2024-09-14T06:17:52.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7073999999999998e-05 2024-09-14T06:17:52.386 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.386 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.386 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.387 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15621 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'125, mlcod=36'119)", 2024-09-14T06:17:52.387 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:52.387 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 76.061767164000003, 2024-09-14T06:17:52.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0069496169999999999, 2024-09-14T06:17:52.387 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.387 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.387 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.387 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.387 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:52.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:52.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:52.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2200000000000003e-07 2024-09-14T06:17:52.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299372+0000", 2024-09-14T06:17:52.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3850000000000001e-06 2024-09-14T06:17:52.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299387+0000", 2024-09-14T06:17:52.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4905e-05 2024-09-14T06:17:52.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299393+0000", 2024-09-14T06:17:52.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9090000000000003e-06 2024-09-14T06:17:52.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303591+0000", 2024-09-14T06:17:52.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041976640000000003 2024-09-14T06:17:52.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303619+0000", 2024-09-14T06:17:52.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7651e-05 2024-09-14T06:17:52.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306287+0000", 2024-09-14T06:17:52.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026686689999999998 2024-09-14T06:17:52.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.391 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.391 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306315+0000", 2024-09-14T06:17:52.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7712000000000002e-05 2024-09-14T06:17:52.391 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.391 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.391 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.391 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:18539 3.1e e40/29 3:7e28f783:::rbd_data.1152227be94f.0000000000000000:head v 40'59, mlcod=40'57)", 2024-09-14T06:17:52.391 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:17:52.391 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 70.078091813, 2024-09-14T06:17:52.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0059097430000000003, 2024-09-14T06:17:52.392 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.392 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.392 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.392 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:17:52.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283041+0000", 2024-09-14T06:17:52.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.283043+0000", 2024-09-14T06:17:52.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4499999999999998e-06 2024-09-14T06:17:52.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285878+0000", 2024-09-14T06:17:52.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028348039999999998 2024-09-14T06:17:52.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285908+0000", 2024-09-14T06:17:52.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9756000000000001e-05 2024-09-14T06:17:52.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.285922+0000", 2024-09-14T06:17:52.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.43e-05 2024-09-14T06:17:52.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.286030+0000", 2024-09-14T06:17:52.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000107954 2024-09-14T06:17:52.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.286058+0000", 2024-09-14T06:17:52.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7796e-05 2024-09-14T06:17:52.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.288888+0000", 2024-09-14T06:17:52.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028295799999999999 2024-09-14T06:17:52.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:42.288951+0000", 2024-09-14T06:17:52.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3102999999999995e-05 2024-09-14T06:17:52.395 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.395 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.396 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.396 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:24229 3.5 e47/34 3:a30fd30b:::rbd_data.1152db28e106.0000000000000001:head v 47'55, mlcod=45'53)", 2024-09-14T06:17:52.396 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:17:52.396 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 60.451121565999998, 2024-09-14T06:17:52.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0092924199999999992, 2024-09-14T06:17:52.396 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.396 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.396 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.396 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.396 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.396 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:17:52.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:17:52.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910013+0000", 2024-09-14T06:17:52.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.812e-06 2024-09-14T06:17:52.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915374+0000", 2024-09-14T06:17:52.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053609240000000004 2024-09-14T06:17:52.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915395+0000", 2024-09-14T06:17:52.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1118e-05 2024-09-14T06:17:52.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915406+0000", 2024-09-14T06:17:52.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1303e-05 2024-09-14T06:17:52.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915490+0000", 2024-09-14T06:17:52.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4017000000000006e-05 2024-09-14T06:17:52.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915515+0000", 2024-09-14T06:17:52.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4712e-05 2024-09-14T06:17:52.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919240+0000", 2024-09-14T06:17:52.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037247690000000002 2024-09-14T06:17:52.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.400 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.400 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919304+0000", 2024-09-14T06:17:52.400 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3764999999999997e-05 2024-09-14T06:17:52.400 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.400 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.400 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.400 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:head, version: 31'47, data_included: [0~98304,360448~278528], data_size: 376832, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:head@31'47, size: 638976, copy_subset: [(0, 638976)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 638976, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:17:52.400 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:17:52.401 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.150858620999998, 2024-09-14T06:17:52.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062487460000000003, 2024-09-14T06:17:52.401 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.401 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.401 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:52.401 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.401 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.401 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:17:52.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.401 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.401 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:17:52.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210275+0000", 2024-09-14T06:17:52.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:17:52.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210752+0000", 2024-09-14T06:17:52.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047772100000000002 2024-09-14T06:17:52.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210781+0000", 2024-09-14T06:17:52.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8391999999999999e-05 2024-09-14T06:17:52.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210805+0000", 2024-09-14T06:17:52.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4142999999999999e-05 2024-09-14T06:17:52.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.213641+0000", 2024-09-14T06:17:52.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002835827 2024-09-14T06:17:52.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.213663+0000", 2024-09-14T06:17:52.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.243e-05 2024-09-14T06:17:52.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.216523+0000", 2024-09-14T06:17:52.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028597729999999999 2024-09-14T06:17:52.404 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.404 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.404 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.405 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:50564 3.16 3:689e8220:::rbd_header.1152757de986:head [call rbd.snapshot_add in=15b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-14T06:17:52.405 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:13.319284+0000", 2024-09-14T06:17:52.405 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.041848240999997, 2024-09-14T06:17:52.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.030881613, 2024-09-14T06:17:52.405 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.405 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.405 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.405 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:52.406 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:52.406 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:52.406 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50564 2024-09-14T06:17:52.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.406 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319284+0000", 2024-09-14T06:17:52.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319284+0000", 2024-09-14T06:17:52.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319286+0000", 2024-09-14T06:17:52.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.849e-06 2024-09-14T06:17:52.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319293+0000", 2024-09-14T06:17:52.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.72e-06 2024-09-14T06:17:52.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319313+0000", 2024-09-14T06:17:52.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9833000000000001e-05 2024-09-14T06:17:52.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319325+0000", 2024-09-14T06:17:52.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1924e-05 2024-09-14T06:17:52.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319441+0000", 2024-09-14T06:17:52.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000115952 2024-09-14T06:17:52.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op must wait for map", 2024-09-14T06:17:52.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319453+0000", 2024-09-14T06:17:52.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1810999999999999e-05 2024-09-14T06:17:52.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322905+0000", 2024-09-14T06:17:52.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034519680000000001 2024-09-14T06:17:52.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:52.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322917+0000", 2024-09-14T06:17:52.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1946999999999999e-05 2024-09-14T06:17:52.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.342215+0000", 2024-09-14T06:17:52.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0192988300000001 2024-09-14T06:17:52.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.342451+0000", 2024-09-14T06:17:52.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00023546 2024-09-14T06:17:52.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:52.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.342959+0000", 2024-09-14T06:17:52.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050845999999999997 2024-09-14T06:17:52.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:52.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.347472+0000", 2024-09-14T06:17:52.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045125310000000002 2024-09-14T06:17:52.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:52.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.349544+0000", 2024-09-14T06:17:52.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020721730000000001 2024-09-14T06:17:52.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.349727+0000", 2024-09-14T06:17:52.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000182776 2024-09-14T06:17:52.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.350166+0000", 2024-09-14T06:17:52.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000439379 2024-09-14T06:17:52.412 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.412 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.412 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.412 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:50565 3.16 3:689e8220:::rbd_header.1152757de986:head [watch reconnect cookie 140175519011472 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-14T06:17:52.412 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:13.319351+0000", 2024-09-14T06:17:52.412 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.041781663000002, 2024-09-14T06:17:52.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0352714430000001, 2024-09-14T06:17:52.413 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.413 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.413 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:52.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:52.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:52.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:52.413 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50565 2024-09-14T06:17:52.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.413 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.413 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.413 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319351+0000", 2024-09-14T06:17:52.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319351+0000", 2024-09-14T06:17:52.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319352+0000", 2024-09-14T06:17:52.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0500000000000003e-07 2024-09-14T06:17:52.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319355+0000", 2024-09-14T06:17:52.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3960000000000002e-06 2024-09-14T06:17:52.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319367+0000", 2024-09-14T06:17:52.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1518999999999999e-05 2024-09-14T06:17:52.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319373+0000", 2024-09-14T06:17:52.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7800000000000003e-06 2024-09-14T06:17:52.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319517+0000", 2024-09-14T06:17:52.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000143719 2024-09-14T06:17:52.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting_for_map not empty", 2024-09-14T06:17:52.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319520+0000", 2024-09-14T06:17:52.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1990000000000002e-06 2024-09-14T06:17:52.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322963+0000", 2024-09-14T06:17:52.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034424920000000001 2024-09-14T06:17:52.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:52.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322971+0000", 2024-09-14T06:17:52.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0860000000000002e-06 2024-09-14T06:17:52.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343565+0000", 2024-09-14T06:17:52.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0205935530000001 2024-09-14T06:17:52.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for rw locks", 2024-09-14T06:17:52.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343675+0000", 2024-09-14T06:17:52.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011048900000000001 2024-09-14T06:17:52.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.350295+0000", 2024-09-14T06:17:52.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006620094 2024-09-14T06:17:52.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.350584+0000", 2024-09-14T06:17:52.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00028850000000000002 2024-09-14T06:17:52.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.354623+0000", 2024-09-14T06:17:52.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040389110000000001 2024-09-14T06:17:52.419 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.419 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.419 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.419 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:50566 3.16 3:689e8220:::rbd_header.1152757de986:head [watch ping cookie 140175519011472 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-14T06:17:52.419 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:13.960177+0000", 2024-09-14T06:17:52.419 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.400955516000003, 2024-09-14T06:17:52.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.39354602500000002, 2024-09-14T06:17:52.419 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.419 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.419 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:52.419 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:52.419 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:52.420 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:52.420 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50566 2024-09-14T06:17:52.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.420 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960177+0000", 2024-09-14T06:17:52.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960177+0000", 2024-09-14T06:17:52.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960178+0000", 2024-09-14T06:17:52.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.136e-06 2024-09-14T06:17:52.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960186+0000", 2024-09-14T06:17:52.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0299999999999994e-06 2024-09-14T06:17:52.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960202+0000", 2024-09-14T06:17:52.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5238999999999999e-05 2024-09-14T06:17:52.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960218+0000", 2024-09-14T06:17:52.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.668e-05 2024-09-14T06:17:52.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960350+0000", 2024-09-14T06:17:52.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013201 2024-09-14T06:17:52.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:52.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960368+0000", 2024-09-14T06:17:52.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7526000000000001e-05 2024-09-14T06:17:52.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343805+0000", 2024-09-14T06:17:52.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.38343735400000001 2024-09-14T06:17:52.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for rw locks", 2024-09-14T06:17:52.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343907+0000", 2024-09-14T06:17:52.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010133599999999999 2024-09-14T06:17:52.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.352997+0000", 2024-09-14T06:17:52.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0090907409999999994 2024-09-14T06:17:52.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.353352+0000", 2024-09-14T06:17:52.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00035436000000000002 2024-09-14T06:17:52.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.353723+0000", 2024-09-14T06:17:52.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00037161300000000002 2024-09-14T06:17:52.424 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.424 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.425 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.425 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:51037 3.11 3:89b51a5c:::rbd_data.11522fd7a0c5.0000000000000059:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-14T06:17:52.425 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:15.038049+0000", 2024-09-14T06:17:52.425 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.323084174000002, 2024-09-14T06:17:52.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.075311116999999997, 2024-09-14T06:17:52.425 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.425 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.425 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.425 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:52.425 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:52.425 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:52.425 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 51037 2024-09-14T06:17:52.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.426 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.426 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038049+0000", 2024-09-14T06:17:52.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.426 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038049+0000", 2024-09-14T06:17:52.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038051+0000", 2024-09-14T06:17:52.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:17:52.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038065+0000", 2024-09-14T06:17:52.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3576e-05 2024-09-14T06:17:52.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038097+0000", 2024-09-14T06:17:52.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2566e-05 2024-09-14T06:17:52.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038121+0000", 2024-09-14T06:17:52.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3207e-05 2024-09-14T06:17:52.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038291+0000", 2024-09-14T06:17:52.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016996900000000001 2024-09-14T06:17:52.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038543+0000", 2024-09-14T06:17:52.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00025233999999999999 2024-09-14T06:17:52.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:52.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.053218+0000", 2024-09-14T06:17:52.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014674957000000001 2024-09-14T06:17:52.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:52.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113195+0000", 2024-09-14T06:17:52.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.059976697000000002 2024-09-14T06:17:52.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:52.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113212+0000", 2024-09-14T06:17:52.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7299000000000002e-05 2024-09-14T06:17:52.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.430 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.430 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.430 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113267+0000", 2024-09-14T06:17:52.430 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4632999999999997e-05 2024-09-14T06:17:52.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.430 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.430 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.430 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113360+0000", 2024-09-14T06:17:52.430 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3129999999999998e-05 2024-09-14T06:17:52.430 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.430 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.430 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.430 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.431 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:51037 3.11 e67/66 ondisk, result = 0)", 2024-09-14T06:17:52.431 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:15.093818+0000", 2024-09-14T06:17:52.431 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.267314382999999, 2024-09-14T06:17:52.431 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.019384384000000001, 2024-09-14T06:17:52.431 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.431 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.431 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:52.431 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.431 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.431 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093818+0000", 2024-09-14T06:17:52.431 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093818+0000", 2024-09-14T06:17:52.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093820+0000", 2024-09-14T06:17:52.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0169999999999999e-06 2024-09-14T06:17:52.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093829+0000", 2024-09-14T06:17:52.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6500000000000002e-06 2024-09-14T06:17:52.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093860+0000", 2024-09-14T06:17:52.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1275999999999997e-05 2024-09-14T06:17:52.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093883+0000", 2024-09-14T06:17:52.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2464999999999999e-05 2024-09-14T06:17:52.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113178+0000", 2024-09-14T06:17:52.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.019295643000000001 2024-09-14T06:17:52.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.434 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.434 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113189+0000", 2024-09-14T06:17:52.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0285999999999999e-05 2024-09-14T06:17:52.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.434 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.434 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113203+0000", 2024-09-14T06:17:52.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4047e-05 2024-09-14T06:17:52.434 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.434 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.434 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.435 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:51156 3.16 3:6daa8e8a:::rbd_data.11522fd7a0c5.00000000000000bc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-14T06:17:52.435 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:15.298652+0000", 2024-09-14T06:17:52.435 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.062480745999999, 2024-09-14T06:17:52.435 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0057204090000000001, 2024-09-14T06:17:52.435 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.435 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.435 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.435 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:52.435 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:52.435 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:52.435 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 51156 2024-09-14T06:17:52.435 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.435 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.435 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.436 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.436 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.298652+0000", 2024-09-14T06:17:52.436 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.436 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.436 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.298652+0000", 2024-09-14T06:17:52.436 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.436 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.436 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.298655+0000", 2024-09-14T06:17:52.436 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3069999999999998e-06 2024-09-14T06:17:52.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.437 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.437 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.437 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.298671+0000", 2024-09-14T06:17:52.437 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5415999999999999e-05 2024-09-14T06:17:52.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.437 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.437 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.437 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.298714+0000", 2024-09-14T06:17:52.437 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3161999999999998e-05 2024-09-14T06:17:52.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.437 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.437 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.437 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.298740+0000", 2024-09-14T06:17:52.437 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6582999999999999e-05 2024-09-14T06:17:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.298936+0000", 2024-09-14T06:17:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000195212 2024-09-14T06:17:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.299200+0000", 2024-09-14T06:17:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00026452600000000001 2024-09-14T06:17:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.299532+0000", 2024-09-14T06:17:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00033223200000000001 2024-09-14T06:17:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.301497+0000", 2024-09-14T06:17:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001964772 2024-09-14T06:17:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.303904+0000", 2024-09-14T06:17:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024067469999999999 2024-09-14T06:17:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.304072+0000", 2024-09-14T06:17:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016849899999999999 2024-09-14T06:17:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.304372+0000", 2024-09-14T06:17:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00029995299999999999 2024-09-14T06:17:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.441 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:52268 3.e e70/34 3:7013111b:::rbd_data.1152cd930a87.0000000000000001:head v 70'315, mlcod=70'313)", 2024-09-14T06:17:52.441 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:17.975434+0000", 2024-09-14T06:17:52.441 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 34.385699185, 2024-09-14T06:17:52.441 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014816765000000001, 2024-09-14T06:17:52.441 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.441 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.441 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.441 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.441 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.441 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.441 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.975434+0000", 2024-09-14T06:17:52.441 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.441 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.442 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.442 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.442 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.975434+0000", 2024-09-14T06:17:52.442 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.442 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.442 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.442 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.975436+0000", 2024-09-14T06:17:52.442 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3010000000000001e-06 2024-09-14T06:17:52.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.442 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.442 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.442 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982453+0000", 2024-09-14T06:17:52.442 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0070167909999999997 2024-09-14T06:17:52.443 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.443 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.443 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.443 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982489+0000", 2024-09-14T06:17:52.443 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6603000000000002e-05 2024-09-14T06:17:52.443 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.443 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.443 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.443 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982507+0000", 2024-09-14T06:17:52.443 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8074e-05 2024-09-14T06:17:52.443 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.443 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.443 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.444 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982610+0000", 2024-09-14T06:17:52.444 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010224 2024-09-14T06:17:52.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.444 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.444 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.444 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982639+0000", 2024-09-14T06:17:52.444 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9009999999999998e-05 2024-09-14T06:17:52.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.444 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.444 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.444 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.990188+0000", 2024-09-14T06:17:52.444 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0075497150000000002 2024-09-14T06:17:52.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.444 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.445 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.445 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.990250+0000", 2024-09-14T06:17:52.445 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2031000000000003e-05 2024-09-14T06:17:52.445 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.445 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.445 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.445 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.445 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.445 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:52269 3.1f e70/56 3:fd802c48:::rbd_data.1152cd930a87.0000000000000002:head v 70'321, mlcod=70'318)", 2024-09-14T06:17:52.445 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:17.982671+0000", 2024-09-14T06:17:52.445 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 34.378462202999998, 2024-09-14T06:17:52.445 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016453552, 2024-09-14T06:17:52.445 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.446 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.446 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.446 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.446 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.446 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.446 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982671+0000", 2024-09-14T06:17:52.446 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.446 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.446 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.446 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.446 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982671+0000", 2024-09-14T06:17:52.446 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.446 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.446 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.446 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.447 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982672+0000", 2024-09-14T06:17:52.447 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.246e-06 2024-09-14T06:17:52.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.447 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.447 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.447 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991501+0000", 2024-09-14T06:17:52.447 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0088292590000000008 2024-09-14T06:17:52.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.447 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.447 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.447 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991521+0000", 2024-09-14T06:17:52.447 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0211e-05 2024-09-14T06:17:52.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.447 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.448 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.448 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991531+0000", 2024-09-14T06:17:52.448 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0207000000000001e-05 2024-09-14T06:17:52.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.448 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.448 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.448 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991605+0000", 2024-09-14T06:17:52.448 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3066999999999997e-05 2024-09-14T06:17:52.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.448 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.448 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.448 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991633+0000", 2024-09-14T06:17:52.448 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8136e-05 2024-09-14T06:17:52.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.448 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.449 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.449 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.999052+0000", 2024-09-14T06:17:52.449 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0074192320000000004 2024-09-14T06:17:52.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.449 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.449 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.449 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.999124+0000", 2024-09-14T06:17:52.449 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2193999999999994e-05 2024-09-14T06:17:52.449 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.449 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.449 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.449 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.449 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:59899 3.9 e75/68 3:96b79e96:::rbd_data.115214b4562.0000000000000004:head v 75'319, mlcod=75'317)", 2024-09-14T06:17:52.450 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:24.822693+0000", 2024-09-14T06:17:52.450 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.538439352000001, 2024-09-14T06:17:52.450 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014246999, 2024-09-14T06:17:52.450 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.450 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.450 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.450 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.450 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.450 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.450 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.822693+0000", 2024-09-14T06:17:52.450 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.450 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.450 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.450 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.450 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.822693+0000", 2024-09-14T06:17:52.451 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.451 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.451 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.451 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.451 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.822695+0000", 2024-09-14T06:17:52.451 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.762e-06 2024-09-14T06:17:52.451 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.451 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.451 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.451 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831397+0000", 2024-09-14T06:17:52.451 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0087016230000000003 2024-09-14T06:17:52.451 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.451 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.451 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.452 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831424+0000", 2024-09-14T06:17:52.452 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7480000000000001e-05 2024-09-14T06:17:52.452 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.452 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.452 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.452 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831435+0000", 2024-09-14T06:17:52.452 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0943e-05 2024-09-14T06:17:52.452 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.452 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.452 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.452 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831509+0000", 2024-09-14T06:17:52.452 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3596000000000005e-05 2024-09-14T06:17:52.452 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.452 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.453 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.453 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831531+0000", 2024-09-14T06:17:52.453 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2628000000000001e-05 2024-09-14T06:17:52.453 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.453 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.453 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.453 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.836873+0000", 2024-09-14T06:17:52.453 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053417849999999999 2024-09-14T06:17:52.453 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.453 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.453 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.453 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.836940+0000", 2024-09-14T06:17:52.453 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7181999999999995e-05 2024-09-14T06:17:52.453 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.454 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.454 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.454 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.454 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.454 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:64148 3.2 e86/29 3:41d40d1e:::rbd_data.115267b2c56c.0000000000000000:head v 86'276, mlcod=80'274)", 2024-09-14T06:17:52.454 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:38.879703+0000", 2024-09-14T06:17:52.454 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.481429724, 2024-09-14T06:17:52.454 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014921665000000001, 2024-09-14T06:17:52.454 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.454 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.454 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.454 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.454 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.454 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.454 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.879703+0000", 2024-09-14T06:17:52.455 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.455 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.455 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.455 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.455 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.879703+0000", 2024-09-14T06:17:52.455 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.455 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.455 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.455 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.455 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.879704+0000", 2024-09-14T06:17:52.455 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0219999999999999e-06 2024-09-14T06:17:52.455 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.455 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.455 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.456 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885314+0000", 2024-09-14T06:17:52.456 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056102119999999998 2024-09-14T06:17:52.456 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.456 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.456 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.456 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885331+0000", 2024-09-14T06:17:52.456 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6696000000000001e-05 2024-09-14T06:17:52.456 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.456 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.456 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.456 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885342+0000", 2024-09-14T06:17:52.456 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0886999999999999e-05 2024-09-14T06:17:52.456 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.456 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.457 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.457 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885468+0000", 2024-09-14T06:17:52.457 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012655399999999999 2024-09-14T06:17:52.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.457 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.457 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.457 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885517+0000", 2024-09-14T06:17:52.457 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9057999999999997e-05 2024-09-14T06:17:52.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.457 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.457 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.457 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.894572+0000", 2024-09-14T06:17:52.457 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0090541130000000008 2024-09-14T06:17:52.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.458 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.458 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.458 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.894625+0000", 2024-09-14T06:17:52.458 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3122999999999999e-05 2024-09-14T06:17:52.458 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.458 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.458 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.458 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.458 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.458 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:64152 3.3 e86/29 3:c1c8d963:::rbd_data.115267b2c56c.0000000000000002:head v 86'180, mlcod=79'178)", 2024-09-14T06:17:52.458 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:38.884954+0000", 2024-09-14T06:17:52.458 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.476178261999999, 2024-09-14T06:17:52.458 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011982222000000001, 2024-09-14T06:17:52.458 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.459 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.459 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.459 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.459 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.459 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.459 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.884954+0000", 2024-09-14T06:17:52.459 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.459 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.459 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.459 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.459 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.884954+0000", 2024-09-14T06:17:52.459 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.459 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.459 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.460 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.460 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.884956+0000", 2024-09-14T06:17:52.460 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1179999999999999e-06 2024-09-14T06:17:52.460 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.460 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.460 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.460 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886783+0000", 2024-09-14T06:17:52.460 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018269289999999999 2024-09-14T06:17:52.460 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.460 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.460 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.460 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886795+0000", 2024-09-14T06:17:52.460 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1975e-05 2024-09-14T06:17:52.460 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.461 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.461 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.461 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886802+0000", 2024-09-14T06:17:52.461 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.824e-06 2024-09-14T06:17:52.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.461 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.461 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.461 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886881+0000", 2024-09-14T06:17:52.461 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8702000000000006e-05 2024-09-14T06:17:52.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.461 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.461 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.461 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886902+0000", 2024-09-14T06:17:52.461 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0854000000000001e-05 2024-09-14T06:17:52.462 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.462 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.462 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.462 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.896887+0000", 2024-09-14T06:17:52.462 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0099853700000000004 2024-09-14T06:17:52.462 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.462 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.462 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.462 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.896937+0000", 2024-09-14T06:17:52.462 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9450000000000003e-05 2024-09-14T06:17:52.462 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.462 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.462 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.462 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.463 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.463 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:64155 3.1 e86/29 3:86008c25:::rbd_data.115267b2c56c.0000000000000003:head v 86'451, mlcod=78'449)", 2024-09-14T06:17:52.463 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:17:52.463 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.458077176, 2024-09-14T06:17:52.463 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006333038, 2024-09-14T06:17:52.463 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.463 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.463 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.463 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.463 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.463 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.463 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:17:52.463 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.463 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.464 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.464 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.464 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:17:52.464 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.464 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.464 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.464 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.464 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:17:52.464 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:17:52.464 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.464 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.464 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.464 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906826+0000", 2024-09-14T06:17:52.464 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037703260000000001 2024-09-14T06:17:52.464 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.465 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.465 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.465 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906845+0000", 2024-09-14T06:17:52.465 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8488999999999999e-05 2024-09-14T06:17:52.465 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.465 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.465 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.465 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906857+0000", 2024-09-14T06:17:52.465 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2571e-05 2024-09-14T06:17:52.465 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.465 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.465 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.465 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906966+0000", 2024-09-14T06:17:52.465 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010832000000000001 2024-09-14T06:17:52.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.466 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.466 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.466 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906984+0000", 2024-09-14T06:17:52.466 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8068999999999999e-05 2024-09-14T06:17:52.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.466 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.466 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.466 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.909306+0000", 2024-09-14T06:17:52.466 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023221050000000001 2024-09-14T06:17:52.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.466 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.466 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.466 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.909389+0000", 2024-09-14T06:17:52.466 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2865e-05 2024-09-14T06:17:52.467 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.467 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.467 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.467 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.467 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.467 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:52.468 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:17:52.469 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:52.465+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:52.569 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:52.565+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:52.633 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:52.633 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:52.647 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:17:52.692 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:52.689+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:52.772 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:52.768+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:52.813 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:52.813 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:52.813 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:52.813 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:52.813 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:52.826 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:17:52.872 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:52.869+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:52.954 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:52.954 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:52.954 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:52.954 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:52.954 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.955 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:52.955 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:52.955 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 76.959349622000005, 2024-09-14T06:17:52.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:17:52.955 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.955 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.955 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:52.955 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:52.955 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:52.955 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:52.955 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:17:52.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.955 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.956 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.956 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:52.956 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.956 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.956 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.956 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:52.956 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.956 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.956 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.956 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:52.956 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:17:52.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.957 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.957 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:17:52.957 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:17:52.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.957 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.957 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:17:52.957 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:17:52.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.957 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.957 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:17:52.957 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:17:52.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.958 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.958 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:17:52.958 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:17:52.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.958 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.958 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:17:52.958 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:17:52.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.958 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.958 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:17:52.959 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:17:52.959 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.959 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.959 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.959 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.959 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:52.959 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:52.959 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 76.959336550000003, 2024-09-14T06:17:52.959 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:17:52.959 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.959 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.959 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:52.959 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:52.959 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:52.960 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:52.960 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:17:52.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.960 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.960 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.960 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:52.960 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.960 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.960 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:52.960 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.961 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.961 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.961 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:17:52.961 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:17:52.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.961 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.961 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.961 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:17:52.961 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:17:52.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.961 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.961 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.961 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:17:52.961 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:17:52.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.962 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.962 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:17:52.962 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:17:52.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.962 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.962 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:17:52.962 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:17:52.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.963 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.963 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:17:52.963 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:17:52.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.963 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.963 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:17:52.963 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:17:52.963 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.963 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.963 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.963 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:52.963 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:52.964 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 76.959319117999996, 2024-09-14T06:17:52.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:17:52.964 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.964 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.964 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:52.964 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:52.964 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:52.964 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:52.964 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:17:52.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.964 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.964 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.964 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:52.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.965 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.965 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:52.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.965 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.965 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:52.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:17:52.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.965 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.966 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:17:52.966 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:17:52.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.966 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.966 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.966 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:17:52.966 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:17:52.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.966 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.966 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.966 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:17:52.966 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:17:52.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.966 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.967 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.967 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:17:52.967 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:17:52.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.967 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.967 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:17:52.967 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:17:52.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.967 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.967 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:17:52.967 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:17:52.967 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.967 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.968 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.968 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:52.968 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:52.968 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 76.959298012999994, 2024-09-14T06:17:52.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:17:52.968 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.968 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.968 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:52.968 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:52.968 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:52.968 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:52.969 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:17:52.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.969 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:52.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:52.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.970 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.970 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:52.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:17:52.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.970 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.970 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:17:52.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:17:52.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.970 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.970 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:17:52.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:17:52.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.971 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.971 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:17:52.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:17:52.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.971 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.971 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:17:52.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:17:52.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.971 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.971 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:17:52.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:17:52.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:17:52.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:17:52.972 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.972 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.972 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.972 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:52.972 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:52.972 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 76.959276904999996, 2024-09-14T06:17:52.973 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:17:52.973 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.973 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.973 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:52.973 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:52.973 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:52.973 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:52.973 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:17:52.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.973 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.973 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.973 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:52.973 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:52.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:52.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:17:52.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:17:52.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:17:52.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:17:52.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:17:52.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:17:52.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:17:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:17:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:17:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:17:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:17:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:17:52.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:17:52.977 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.977 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.977 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.977 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.977 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48234 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:52.977 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:52.977 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.918005612999998, 2024-09-14T06:17:52.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051730353999999999, 2024-09-14T06:17:52.977 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.977 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.977 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.977 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:52.978 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:52.978 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:52.978 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48234 2024-09-14T06:17:52.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.978 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:52.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.979 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:52.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.979 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.979 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032937+0000", 2024-09-14T06:17:52.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:17:52.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.979 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.979 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032940+0000", 2024-09-14T06:17:52.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0580000000000002e-06 2024-09-14T06:17:52.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032946+0000", 2024-09-14T06:17:52.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2960000000000004e-06 2024-09-14T06:17:52.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032951+0000", 2024-09-14T06:17:52.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.938e-06 2024-09-14T06:17:52.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074419+0000", 2024-09-14T06:17:52.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041467979000000002 2024-09-14T06:17:52.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074490+0000", 2024-09-14T06:17:52.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1324999999999999e-05 2024-09-14T06:17:52.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:52.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074859+0000", 2024-09-14T06:17:52.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036914899999999999 2024-09-14T06:17:52.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:52.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078464+0000", 2024-09-14T06:17:52.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036043170000000001 2024-09-14T06:17:52.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:52.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084547+0000", 2024-09-14T06:17:52.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006083097 2024-09-14T06:17:52.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084614+0000", 2024-09-14T06:17:52.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7509999999999996e-05 2024-09-14T06:17:52.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084667+0000", 2024-09-14T06:17:52.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:17:52.983 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.983 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.983 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.983 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48237 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:52.983 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:52.983 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.917925869000001, 2024-09-14T06:17:52.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051772830999999998, 2024-09-14T06:17:52.983 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.983 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.983 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.983 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:52.984 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:52.984 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:52.984 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48237 2024-09-14T06:17:52.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.984 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.984 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:52.984 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.984 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:52.984 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:52.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0499999999999999e-07 2024-09-14T06:17:52.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033018+0000", 2024-09-14T06:17:52.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0949999999999998e-06 2024-09-14T06:17:52.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033023+0000", 2024-09-14T06:17:52.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1620000000000001e-06 2024-09-14T06:17:52.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033026+0000", 2024-09-14T06:17:52.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0120000000000001e-06 2024-09-14T06:17:52.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075349+0000", 2024-09-14T06:17:52.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042323373999999997 2024-09-14T06:17:52.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075394+0000", 2024-09-14T06:17:52.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4861000000000003e-05 2024-09-14T06:17:52.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:52.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075578+0000", 2024-09-14T06:17:52.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00018414899999999999 2024-09-14T06:17:52.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:52.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078894+0000", 2024-09-14T06:17:52.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033158480000000001 2024-09-14T06:17:52.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:52.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084677+0000", 2024-09-14T06:17:52.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0057832120000000002 2024-09-14T06:17:52.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084752+0000", 2024-09-14T06:17:52.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5123000000000006e-05 2024-09-14T06:17:52.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084789+0000", 2024-09-14T06:17:52.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6690000000000003e-05 2024-09-14T06:17:52.988 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.988 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.989 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.989 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48243 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:52.989 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:52.989 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.917844744, 2024-09-14T06:17:52.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051780221000000001, 2024-09-14T06:17:52.989 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.989 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.989 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.989 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:52.989 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:52.989 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:52.989 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48243 2024-09-14T06:17:52.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.990 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:52.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:52.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:52.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:17:52.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033100+0000", 2024-09-14T06:17:52.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:17:52.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033108+0000", 2024-09-14T06:17:52.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1760000000000004e-06 2024-09-14T06:17:52.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033111+0000", 2024-09-14T06:17:52.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0790000000000001e-06 2024-09-14T06:17:52.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075903+0000", 2024-09-14T06:17:52.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042791942999999999 2024-09-14T06:17:52.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075987+0000", 2024-09-14T06:17:52.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4467000000000003e-05 2024-09-14T06:17:52.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:52.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.076259+0000", 2024-09-14T06:17:52.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027201900000000003 2024-09-14T06:17:52.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:52.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078947+0000", 2024-09-14T06:17:52.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026875879999999999 2024-09-14T06:17:52.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:52.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084797+0000", 2024-09-14T06:17:52.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0058500380000000001 2024-09-14T06:17:52.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:52.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084843+0000", 2024-09-14T06:17:52.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6298999999999998e-05 2024-09-14T06:17:52.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:52.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084877+0000", 2024-09-14T06:17:52.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4378999999999997e-05 2024-09-14T06:17:52.994 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.994 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:52.994 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:52.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.995 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48287 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:52.995 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:52.995 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.917804943999997, 2024-09-14T06:17:52.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051924706000000001, 2024-09-14T06:17:52.995 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:52.995 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:52.995 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:52.995 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:52.995 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:52.995 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:52.995 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48287 2024-09-14T06:17:52.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.995 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:52.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:52.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:52.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:52.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:52.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:52.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:52.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033138+0000", 2024-09-14T06:17:52.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6599999999999996e-07 2024-09-14T06:17:52.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.997 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:52.997 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033141+0000", 2024-09-14T06:17:52.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0879999999999999e-06 2024-09-14T06:17:52.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.997 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:52.997 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033147+0000", 2024-09-14T06:17:52.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2249999999999997e-06 2024-09-14T06:17:52.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.997 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:52.997 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033153+0000", 2024-09-14T06:17:52.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.066e-06 2024-09-14T06:17:52.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:52.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077790+0000", 2024-09-14T06:17:52.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.044637255000000001 2024-09-14T06:17:52.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:52.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077845+0000", 2024-09-14T06:17:52.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4410999999999998e-05 2024-09-14T06:17:52.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:52.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078062+0000", 2024-09-14T06:17:52.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021755599999999999 2024-09-14T06:17:52.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:52.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.080991+0000", 2024-09-14T06:17:52.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029282969999999998 2024-09-14T06:17:52.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:52.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:52.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:52.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084885+0000", 2024-09-14T06:17:52.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003894495 2024-09-14T06:17:53.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:53.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085020+0000", 2024-09-14T06:17:53.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013500799999999999 2024-09-14T06:17:53.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:53.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085062+0000", 2024-09-14T06:17:53.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1638999999999999e-05 2024-09-14T06:17:53.000 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.000 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:53.000 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.001 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48288 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:53.001 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:53.001 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.917781245, 2024-09-14T06:17:53.001 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051994018000000003, 2024-09-14T06:17:53.001 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:53.001 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:53.001 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:53.001 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:53.001 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:53.001 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:53.001 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48288 2024-09-14T06:17:53.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.001 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:53.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:53.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:53.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:53.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:53.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:53.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:53.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.89e-07 2024-09-14T06:17:53.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:53.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033164+0000", 2024-09-14T06:17:53.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.779e-06 2024-09-14T06:17:53.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:53.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033171+0000", 2024-09-14T06:17:53.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6279999999999999e-06 2024-09-14T06:17:53.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:53.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033176+0000", 2024-09-14T06:17:53.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1610000000000002e-06 2024-09-14T06:17:53.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:53.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078991+0000", 2024-09-14T06:17:53.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045815348999999998 2024-09-14T06:17:53.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:53.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079086+0000", 2024-09-14T06:17:53.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5282e-05 2024-09-14T06:17:53.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:53.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079397+0000", 2024-09-14T06:17:53.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031055999999999998 2024-09-14T06:17:53.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.005 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:53.005 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084432+0000", 2024-09-14T06:17:53.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050355599999999997 2024-09-14T06:17:53.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.005 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:53.005 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085073+0000", 2024-09-14T06:17:53.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064015199999999995 2024-09-14T06:17:53.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.005 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:53.006 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085120+0000", 2024-09-14T06:17:53.006 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7107000000000001e-05 2024-09-14T06:17:53.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.006 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.006 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:53.006 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085155+0000", 2024-09-14T06:17:53.006 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4950999999999998e-05 2024-09-14T06:17:53.006 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.006 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:53.006 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.006 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.006 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48416 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:53.006 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:53.006 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.915540833000001, 2024-09-14T06:17:53.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053690144000000002, 2024-09-14T06:17:53.007 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:53.007 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:53.007 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:53.007 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:53.007 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:53.007 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:53.007 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48416 2024-09-14T06:17:53.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.007 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:53.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:53.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:53.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:53.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:53.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:53.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035402+0000", 2024-09-14T06:17:53.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7999999999999995e-07 2024-09-14T06:17:53.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:53.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035405+0000", 2024-09-14T06:17:53.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3960000000000002e-06 2024-09-14T06:17:53.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.009 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:53.009 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035413+0000", 2024-09-14T06:17:53.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2209999999999997e-06 2024-09-14T06:17:53.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.009 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:53.009 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035420+0000", 2024-09-14T06:17:53.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1539999999999996e-06 2024-09-14T06:17:53.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.010 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:53.010 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081021+0000", 2024-09-14T06:17:53.010 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045600389999999998 2024-09-14T06:17:53.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.010 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:53.010 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081091+0000", 2024-09-14T06:17:53.010 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0325000000000002e-05 2024-09-14T06:17:53.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.010 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:53.010 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081434+0000", 2024-09-14T06:17:53.010 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000342519 2024-09-14T06:17:53.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.011 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:53.011 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085254+0000", 2024-09-14T06:17:53.011 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038201609999999999 2024-09-14T06:17:53.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.011 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:53.011 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088977+0000", 2024-09-14T06:17:53.011 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003722805 2024-09-14T06:17:53.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.011 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:53.011 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089048+0000", 2024-09-14T06:17:53.011 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1266999999999994e-05 2024-09-14T06:17:53.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.012 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:53.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089091+0000", 2024-09-14T06:17:53.012 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3325999999999998e-05 2024-09-14T06:17:53.012 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.012 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:53.012 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.012 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48417 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:53.012 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:53.012 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.915512163999999, 2024-09-14T06:17:53.012 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053805032000000003, 2024-09-14T06:17:53.013 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:53.013 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:53.013 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:53.013 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:53.013 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:53.013 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:53.013 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48417 2024-09-14T06:17:53.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.013 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:53.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.013 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:53.013 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:53.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.014 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:53.014 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:53.014 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.014 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.014 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:53.014 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:53.014 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9500000000000003e-07 2024-09-14T06:17:53.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.014 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.014 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:53.014 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035433+0000", 2024-09-14T06:17:53.014 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.796e-06 2024-09-14T06:17:53.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.015 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.015 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:53.015 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035442+0000", 2024-09-14T06:17:53.015 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7120000000000006e-06 2024-09-14T06:17:53.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.015 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.015 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:53.015 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035448+0000", 2024-09-14T06:17:53.015 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.015e-06 2024-09-14T06:17:53.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.015 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.015 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:53.015 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081855+0000", 2024-09-14T06:17:53.015 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.046407634000000003 2024-09-14T06:17:53.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.016 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.016 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:53.016 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081938+0000", 2024-09-14T06:17:53.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2687000000000003e-05 2024-09-14T06:17:53.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.016 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.016 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:53.016 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082313+0000", 2024-09-14T06:17:53.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000374872 2024-09-14T06:17:53.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.016 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.016 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:53.016 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087616+0000", 2024-09-14T06:17:53.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053032670000000004 2024-09-14T06:17:53.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:53.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089140+0000", 2024-09-14T06:17:53.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0015234350000000001 2024-09-14T06:17:53.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:53.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089184+0000", 2024-09-14T06:17:53.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4804e-05 2024-09-14T06:17:53.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:53.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089235+0000", 2024-09-14T06:17:53.018 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0315000000000002e-05 2024-09-14T06:17:53.018 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.018 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:53.018 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.018 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.018 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48418 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:53.018 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:53.018 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.915468025000003, 2024-09-14T06:17:53.018 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053936425000000003, 2024-09-14T06:17:53.018 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:53.018 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:53.018 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:53.018 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:53.019 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:53.019 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:53.019 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48418 2024-09-14T06:17:53.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.019 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:53.019 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.019 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:53.019 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:53.019 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.019 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.019 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:53.019 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:53.019 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:53.020 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035475+0000", 2024-09-14T06:17:53.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7000000000000003e-07 2024-09-14T06:17:53.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:53.020 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035479+0000", 2024-09-14T06:17:53.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2350000000000001e-06 2024-09-14T06:17:53.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:53.020 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035488+0000", 2024-09-14T06:17:53.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8820000000000007e-06 2024-09-14T06:17:53.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.021 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.021 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:53.021 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035494+0000", 2024-09-14T06:17:53.021 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3060000000000003e-06 2024-09-14T06:17:53.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.021 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.021 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:53.021 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082826+0000", 2024-09-14T06:17:53.021 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.047332008000000002 2024-09-14T06:17:53.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.021 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.021 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:53.021 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082883+0000", 2024-09-14T06:17:53.021 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7124999999999999e-05 2024-09-14T06:17:53.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.022 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:53.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.083385+0000", 2024-09-14T06:17:53.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050134099999999998 2024-09-14T06:17:53.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.022 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:53.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088888+0000", 2024-09-14T06:17:53.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005503127 2024-09-14T06:17:53.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.022 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:53.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089294+0000", 2024-09-14T06:17:53.023 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040669700000000003 2024-09-14T06:17:53.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.023 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:53.023 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089348+0000", 2024-09-14T06:17:53.023 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3729e-05 2024-09-14T06:17:53.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.023 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:53.023 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089410+0000", 2024-09-14T06:17:53.023 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2105000000000003e-05 2024-09-14T06:17:53.023 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.023 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:53.023 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.024 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.024 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48419 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:53.024 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:53.024 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.915390395000003, 2024-09-14T06:17:53.024 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054007545999999997, 2024-09-14T06:17:53.024 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:53.024 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:53.024 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:53.024 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:53.024 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:53.024 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:53.024 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48419 2024-09-14T06:17:53.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.025 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:53.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:53.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:53.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:53.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:53.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:53.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035552+0000", 2024-09-14T06:17:53.026 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4099999999999998e-07 2024-09-14T06:17:53.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.026 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.026 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:53.026 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035557+0000", 2024-09-14T06:17:53.026 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1819999999999997e-06 2024-09-14T06:17:53.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.026 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.026 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:53.026 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035566+0000", 2024-09-14T06:17:53.026 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1120000000000002e-06 2024-09-14T06:17:53.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.026 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.026 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:53.028 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035572+0000", 2024-09-14T06:17:53.028 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0430000000000004e-06 2024-09-14T06:17:53.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.028 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.028 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:53.028 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085304+0000", 2024-09-14T06:17:53.029 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049731707 2024-09-14T06:17:53.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.029 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:53.029 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085371+0000", 2024-09-14T06:17:53.029 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6771000000000002e-05 2024-09-14T06:17:53.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.029 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:53.029 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085601+0000", 2024-09-14T06:17:53.029 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022968000000000001 2024-09-14T06:17:53.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.029 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.030 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:53.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088893+0000", 2024-09-14T06:17:53.030 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032927780000000001 2024-09-14T06:17:53.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.030 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.030 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:53.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089467+0000", 2024-09-14T06:17:53.030 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057330399999999996 2024-09-14T06:17:53.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.030 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.030 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:53.030 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089520+0000", 2024-09-14T06:17:53.030 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3133000000000001e-05 2024-09-14T06:17:53.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.031 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.031 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:53.031 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089559+0000", 2024-09-14T06:17:53.031 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9195e-05 2024-09-14T06:17:53.031 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.031 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:53.031 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.031 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.031 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48420 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:53.031 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:53.032 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.915301165000002, 2024-09-14T06:17:53.032 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056401932000000002, 2024-09-14T06:17:53.032 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:53.032 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:53.032 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:53.032 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:53.032 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:53.032 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:53.032 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48420 2024-09-14T06:17:53.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.032 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:53.032 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.033 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:53.033 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:53.033 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.033 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.033 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:53.033 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:53.033 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.033 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.033 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:53.033 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035642+0000", 2024-09-14T06:17:53.033 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1519999999999999e-06 2024-09-14T06:17:53.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.034 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:53.034 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035647+0000", 2024-09-14T06:17:53.034 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2299999999999999e-06 2024-09-14T06:17:53.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.034 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:53.034 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035660+0000", 2024-09-14T06:17:53.034 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3166999999999999e-05 2024-09-14T06:17:53.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.034 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:53.034 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035666+0000", 2024-09-14T06:17:53.034 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1720000000000004e-06 2024-09-14T06:17:53.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.035 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.035 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:53.035 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086104+0000", 2024-09-14T06:17:53.035 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050437956999999999 2024-09-14T06:17:53.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.035 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.035 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:53.035 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086164+0000", 2024-09-14T06:17:53.035 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9592999999999997e-05 2024-09-14T06:17:53.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.035 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.035 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:53.035 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086402+0000", 2024-09-14T06:17:53.036 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000238314 2024-09-14T06:17:53.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.036 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.036 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:53.036 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089103+0000", 2024-09-14T06:17:53.036 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027011600000000002 2024-09-14T06:17:53.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.036 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.036 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:53.036 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091962+0000", 2024-09-14T06:17:53.036 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028581259999999999 2024-09-14T06:17:53.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.036 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.037 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:53.037 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092005+0000", 2024-09-14T06:17:53.037 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3111000000000001e-05 2024-09-14T06:17:53.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.037 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.037 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:53.037 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092043+0000", 2024-09-14T06:17:53.037 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7950000000000001e-05 2024-09-14T06:17:53.037 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.037 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:53.037 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.037 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.038 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48424 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:53.038 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:53.038 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.915037220000002, 2024-09-14T06:17:53.038 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056273173000000003, 2024-09-14T06:17:53.038 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:53.038 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:53.038 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:53.038 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:53.038 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:53.038 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:53.038 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48424 2024-09-14T06:17:53.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.038 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:53.039 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.039 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:53.039 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:53.039 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.039 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.039 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:53.039 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:53.039 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.039 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.039 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:53.039 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:53.040 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2900000000000003e-07 2024-09-14T06:17:53.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.040 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.040 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:53.040 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035909+0000", 2024-09-14T06:17:53.040 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6490000000000001e-06 2024-09-14T06:17:53.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.040 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.040 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:53.040 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035918+0000", 2024-09-14T06:17:53.040 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5089999999999999e-06 2024-09-14T06:17:53.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.040 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.040 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:53.041 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035924+0000", 2024-09-14T06:17:53.041 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7089999999999999e-06 2024-09-14T06:17:53.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.041 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.041 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:53.041 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087662+0000", 2024-09-14T06:17:53.041 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737621999999997 2024-09-14T06:17:53.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.041 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.041 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:53.041 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087744+0000", 2024-09-14T06:17:53.041 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2174000000000003e-05 2024-09-14T06:17:53.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.041 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.042 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:53.042 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088106+0000", 2024-09-14T06:17:53.042 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000361947 2024-09-14T06:17:53.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.042 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.042 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:53.042 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092053+0000", 2024-09-14T06:17:53.042 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0039473080000000001 2024-09-14T06:17:53.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.042 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.042 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:53.042 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092087+0000", 2024-09-14T06:17:53.042 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4153000000000003e-05 2024-09-14T06:17:53.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.043 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:53.043 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092138+0000", 2024-09-14T06:17:53.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0804999999999999e-05 2024-09-14T06:17:53.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.043 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:53.043 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092178+0000", 2024-09-14T06:17:53.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9668e-05 2024-09-14T06:17:53.043 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.043 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:53.043 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.043 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48508 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:53.044 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:53.044 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.909962139000001, 2024-09-14T06:17:53.044 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054906017000000001, 2024-09-14T06:17:53.044 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:53.044 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:53.044 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:53.044 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:53.044 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:53.044 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:53.044 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48508 2024-09-14T06:17:53.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.044 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:53.044 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.044 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:53.045 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:53.045 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.045 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:53.045 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:53.045 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.045 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:53.045 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040981+0000", 2024-09-14T06:17:53.046 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7700000000000004e-07 2024-09-14T06:17:53.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.046 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:53.046 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040984+0000", 2024-09-14T06:17:53.046 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1480000000000002e-06 2024-09-14T06:17:53.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.046 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:53.046 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040991+0000", 2024-09-14T06:17:53.046 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4580000000000004e-06 2024-09-14T06:17:53.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.047 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:53.047 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040997+0000", 2024-09-14T06:17:53.047 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7799999999999997e-06 2024-09-14T06:17:53.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.047 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.047 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:53.047 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089601+0000", 2024-09-14T06:17:53.047 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048604264000000001 2024-09-14T06:17:53.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.047 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.047 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:53.047 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089708+0000", 2024-09-14T06:17:53.048 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010681200000000001 2024-09-14T06:17:53.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.048 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.048 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:53.048 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090384+0000", 2024-09-14T06:17:53.048 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00067557499999999998 2024-09-14T06:17:53.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.048 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.048 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:53.048 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093879+0000", 2024-09-14T06:17:53.048 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034954679999999998 2024-09-14T06:17:53.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.049 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:53.049 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095764+0000", 2024-09-14T06:17:53.049 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018848070000000001 2024-09-14T06:17:53.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.049 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:53.049 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095837+0000", 2024-09-14T06:17:53.049 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3433e-05 2024-09-14T06:17:53.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.050 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:53.050 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095886+0000", 2024-09-14T06:17:53.050 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8495000000000002e-05 2024-09-14T06:17:53.050 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.050 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:53.050 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.050 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.050 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48537 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:53.050 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:53.050 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.908598369000003, 2024-09-14T06:17:53.051 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053681833999999998, 2024-09-14T06:17:53.051 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:53.051 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:53.051 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:53.051 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:53.051 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:53.051 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:53.051 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48537 2024-09-14T06:17:53.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.051 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:53.051 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.051 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:53.052 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:53.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.052 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.052 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:53.052 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:53.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.052 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.052 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:53.052 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:53.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2299999999999999e-07 2024-09-14T06:17:53.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.052 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.053 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:53.053 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042350+0000", 2024-09-14T06:17:53.053 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5690000000000004e-06 2024-09-14T06:17:53.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.053 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.053 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:53.053 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042361+0000", 2024-09-14T06:17:53.053 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1337e-05 2024-09-14T06:17:53.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.053 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.053 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:53.053 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042369+0000", 2024-09-14T06:17:53.054 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8359999999999992e-06 2024-09-14T06:17:53.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.054 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:53.054 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090914+0000", 2024-09-14T06:17:53.054 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048544945999999999 2024-09-14T06:17:53.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.054 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:53.054 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090995+0000", 2024-09-14T06:17:53.054 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0548000000000002e-05 2024-09-14T06:17:53.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.055 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:53.055 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091508+0000", 2024-09-14T06:17:53.055 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000513723 2024-09-14T06:17:53.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.055 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:53.055 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093900+0000", 2024-09-14T06:17:53.055 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023920429999999999 2024-09-14T06:17:53.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.055 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:53.055 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095944+0000", 2024-09-14T06:17:53.056 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020437379999999998 2024-09-14T06:17:53.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.056 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.056 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:53.056 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095987+0000", 2024-09-14T06:17:53.056 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2521999999999997e-05 2024-09-14T06:17:53.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.056 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.056 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:53.056 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.096025+0000", 2024-09-14T06:17:53.056 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8649000000000002e-05 2024-09-14T06:17:53.056 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.056 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:53.056 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.057 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.057 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:63810 3.1d 3.c887d99d (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-14T06:17:53.057 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:17:53.057 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 19.370840482999998, 2024-09-14T06:17:53.057 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.61346978900000004, 2024-09-14T06:17:53.057 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:53.057 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:53.057 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:53.057 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:53.057 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:53.057 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:53.057 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 63810 2024-09-14T06:17:53.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.058 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:53.058 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.058 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:53.058 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:17:53.058 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.058 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.058 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:53.058 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:17:53.058 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.058 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.058 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:53.058 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580104+0000", 2024-09-14T06:17:53.058 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4650000000000001e-06 2024-09-14T06:17:53.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.059 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:53.059 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580112+0000", 2024-09-14T06:17:53.059 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9219999999999997e-06 2024-09-14T06:17:53.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.059 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:53.059 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580130+0000", 2024-09-14T06:17:53.059 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8128999999999999e-05 2024-09-14T06:17:53.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.059 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:53.059 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580140+0000", 2024-09-14T06:17:53.060 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0285000000000001e-05 2024-09-14T06:17:53.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.060 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.060 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:53.060 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580201+0000", 2024-09-14T06:17:53.060 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0517999999999999e-05 2024-09-14T06:17:53.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.060 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.060 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:53.060 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580209+0000", 2024-09-14T06:17:53.060 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6430000000000001e-06 2024-09-14T06:17:53.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.060 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.060 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:53.060 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:34.193554+0000", 2024-09-14T06:17:53.061 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.61334482899999998 2024-09-14T06:17:53.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.061 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.061 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:53.061 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:34.193571+0000", 2024-09-14T06:17:53.061 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6997999999999999e-05 2024-09-14T06:17:53.061 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.061 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:53.061 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.061 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.061 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:63832 3.1d 3:b99be113:::rbd_object_map.11525ae33f05:head [read 0~52] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T06:17:53.061 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:17:53.061 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 15.712844097, 2024-09-14T06:17:53.062 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.99066972200000003, 2024-09-14T06:17:53.062 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:53.062 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:53.062 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:53.062 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:53.062 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:53.062 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:53.062 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 63832 2024-09-14T06:17:53.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.062 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:53.062 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.062 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:53.062 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:17:53.063 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.063 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.063 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:53.063 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:17:53.063 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:53.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.063 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.063 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:53.063 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238100+0000", 2024-09-14T06:17:53.063 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0849999999999999e-06 2024-09-14T06:17:53.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.063 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.063 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:53.063 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238108+0000", 2024-09-14T06:17:53.064 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.089e-06 2024-09-14T06:17:53.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.064 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.064 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:53.064 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238123+0000", 2024-09-14T06:17:53.064 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5418e-05 2024-09-14T06:17:53.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.064 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.064 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:53.064 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238136+0000", 2024-09-14T06:17:53.064 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2204e-05 2024-09-14T06:17:53.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.064 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.064 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:53.065 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238913+0000", 2024-09-14T06:17:53.065 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00077720399999999998 2024-09-14T06:17:53.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.065 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.065 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:53.065 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238934+0000", 2024-09-14T06:17:53.065 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1311999999999999e-05 2024-09-14T06:17:53.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.065 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.065 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:53.065 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228134+0000", 2024-09-14T06:17:53.065 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.98919991200000001 2024-09-14T06:17:53.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.065 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.065 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:53.066 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228261+0000", 2024-09-14T06:17:53.066 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000126742 2024-09-14T06:17:53.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:53.066 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:53.066 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:53.066 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228768+0000", 2024-09-14T06:17:53.066 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050675600000000003 2024-09-14T06:17:53.066 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.066 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:53.066 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.066 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:53.066 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:53.066 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:53.067 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:17:53.069 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:17:53.069 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:53.065+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:53.169 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:53.164+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:53.199 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:53.200 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:53.200 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:53.200 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:53.214 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:17:53.229 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-09-14T06:17:53.269 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:53.265+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:53.271 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:53.271 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:53.289 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:53.290 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:53.307 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:53.307 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:53.326 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:53.351 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:53.352 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:53.370 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:53.365+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:53.372 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:53.372 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:53.391 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:53.391 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:53.396 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:53.396 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:53.405 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:17:53.413 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:53.413 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:53.471 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:53.464+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:53.519 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:53.520 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:53.534 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:17:53.576 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:53.572+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:53.673 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:53.669+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:53.696 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:53.696 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:53.710 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:17:53.774 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:53.773+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:53.830 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:53.831 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:53.845 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:17:53.875 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:53.869+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:53.968 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:53.968 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:53.975 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:53.972+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:53.982 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:17:54.076 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:54.073+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:54.142 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:54.142 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:54.156 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:17:54.177 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:54.173+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:54.278 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:54.273+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:54.280 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:54.280 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:54.294 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:17:54.379 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:54.377+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:54.423 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:54.423 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:54.438 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:17:54.484 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:54.484+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:54.561 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:54.561 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:54.575 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:17:54.620 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:54.620+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:54.682 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:54.680+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:54.737 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:54.737 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:54.751 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:17:54.783 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:54.777+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:54.884 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:54.884+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:54.923 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:54.923 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:54.937 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:17:54.984 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:54.981+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:55.073 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:55.073 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:55.085 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:55.084+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:55.088 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:17:55.186 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:55.181+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:55.257 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:17:55.262 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:55.262 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:55.287 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:17:55.287 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:55.281+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:55.388 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:55.385+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:55.429 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:17:55.482 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:55.482 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:55.488 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:55.485+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:55.499 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:17:55.574 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:55.574 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:55.590 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:55.588+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:55.591 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:55.591 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:55.608 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:55.625 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:55.625 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:55.648 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:55.649 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:55.664 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:55.664 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:55.664 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:55.664 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:55.666 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:55.667 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:55.679 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:17:55.688 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:55.688 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:55.690 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:55.688+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:55.710 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:55.710 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:55.791 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:55.785+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:55.894 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:55.894 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:55.894 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:55.894 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:55.894 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:55.895 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:17:55.895 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:55.889+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:55.994 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:55.989+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:56.069 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:17:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:17:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 111.989064509, 2024-09-14T06:17:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.56009052699999995, 2024-09-14T06:17:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:17:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:17:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 1 2024-09-14T06:17:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:17:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.072 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.072 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:17:56.072 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.072 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.072 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.072 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077065+0000", 2024-09-14T06:17:56.072 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6429999999999999e-06 2024-09-14T06:17:56.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077079+0000", 2024-09-14T06:17:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4447000000000001e-05 2024-09-14T06:17:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077103+0000", 2024-09-14T06:17:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3856e-05 2024-09-14T06:17:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077120+0000", 2024-09-14T06:17:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7337e-05 2024-09-14T06:17:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077255+0000", 2024-09-14T06:17:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000134652 2024-09-14T06:17:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077274+0000", 2024-09-14T06:17:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9003000000000001e-05 2024-09-14T06:17:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634272+0000", 2024-09-14T06:17:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.55699813099999995 2024-09-14T06:17:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634591+0000", 2024-09-14T06:17:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031910800000000002 2024-09-14T06:17:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:17:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634892+0000", 2024-09-14T06:17:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00030066400000000002 2024-09-14T06:17:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636345+0000", 2024-09-14T06:17:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001452896 2024-09-14T06:17:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636974+0000", 2024-09-14T06:17:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00062927800000000004 2024-09-14T06:17:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637071+0000", 2024-09-14T06:17:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7178999999999999e-05 2024-09-14T06:17:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637153+0000", 2024-09-14T06:17:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1333000000000002e-05 2024-09-14T06:17:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38947 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 159744~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:17:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.386006221999999, 2024-09-14T06:17:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085496275999999996, 2024-09-14T06:17:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38947 2024-09-14T06:17:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:17:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:17:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680121+0000", 2024-09-14T06:17:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.82e-07 2024-09-14T06:17:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680123+0000", 2024-09-14T06:17:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5629999999999999e-06 2024-09-14T06:17:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680133+0000", 2024-09-14T06:17:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0890000000000006e-06 2024-09-14T06:17:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680138+0000", 2024-09-14T06:17:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.728e-06 2024-09-14T06:17:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765422+0000", 2024-09-14T06:17:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085283701000000003 2024-09-14T06:17:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765496+0000", 2024-09-14T06:17:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4276000000000005e-05 2024-09-14T06:17:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765617+0000", 2024-09-14T06:17:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000120437 2024-09-14T06:17:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38990 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 163840~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:17:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385980238999998, 2024-09-14T06:17:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085695970999999996, 2024-09-14T06:17:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38990 2024-09-14T06:17:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:17:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:17:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680147+0000", 2024-09-14T06:17:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7599999999999997e-07 2024-09-14T06:17:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680149+0000", 2024-09-14T06:17:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4700000000000001e-06 2024-09-14T06:17:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680156+0000", 2024-09-14T06:17:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.058e-06 2024-09-14T06:17:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680162+0000", 2024-09-14T06:17:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7080000000000002e-06 2024-09-14T06:17:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765657+0000", 2024-09-14T06:17:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085494566999999994 2024-09-14T06:17:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765741+0000", 2024-09-14T06:17:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4782000000000002e-05 2024-09-14T06:17:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765842+0000", 2024-09-14T06:17:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010090999999999999 2024-09-14T06:17:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38991 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 172032~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:17:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385956282000002, 2024-09-14T06:17:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085875878000000003, 2024-09-14T06:17:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38991 2024-09-14T06:17:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:17:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:17:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680171+0000", 2024-09-14T06:17:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4299999999999998e-07 2024-09-14T06:17:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680173+0000", 2024-09-14T06:17:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1639999999999999e-06 2024-09-14T06:17:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680181+0000", 2024-09-14T06:17:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0539999999999995e-06 2024-09-14T06:17:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680187+0000", 2024-09-14T06:17:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7529999999999998e-06 2024-09-14T06:17:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765875+0000", 2024-09-14T06:17:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085688722999999994 2024-09-14T06:17:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765948+0000", 2024-09-14T06:17:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2126999999999996e-05 2024-09-14T06:17:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766046+0000", 2024-09-14T06:17:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8614000000000001e-05 2024-09-14T06:17:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38992 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 180224~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385931006, 2024-09-14T06:17:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086050225999999994, 2024-09-14T06:17:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38992 2024-09-14T06:17:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:17:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1199999999999998e-07 2024-09-14T06:17:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680199+0000", 2024-09-14T06:17:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:17:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680227+0000", 2024-09-14T06:17:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8238999999999998e-05 2024-09-14T06:17:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680234+0000", 2024-09-14T06:17:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0550000000000002e-06 2024-09-14T06:17:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766077+0000", 2024-09-14T06:17:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085843002000000002 2024-09-14T06:17:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766155+0000", 2024-09-14T06:17:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8112e-05 2024-09-14T06:17:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766246+0000", 2024-09-14T06:17:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0785999999999994e-05 2024-09-14T06:17:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38993 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 188416~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:17:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385839654999998, 2024-09-14T06:17:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086166131000000007, 2024-09-14T06:17:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38993 2024-09-14T06:17:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:17:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:17:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680288+0000", 2024-09-14T06:17:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7199999999999998e-07 2024-09-14T06:17:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680291+0000", 2024-09-14T06:17:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7280000000000001e-06 2024-09-14T06:17:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680301+0000", 2024-09-14T06:17:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3360000000000001e-06 2024-09-14T06:17:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680308+0000", 2024-09-14T06:17:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1389999999999998e-06 2024-09-14T06:17:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766280+0000", 2024-09-14T06:17:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085972111000000004 2024-09-14T06:17:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766357+0000", 2024-09-14T06:17:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7477000000000004e-05 2024-09-14T06:17:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766453+0000", 2024-09-14T06:17:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5668000000000006e-05 2024-09-14T06:17:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38994 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 196608~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385749848000003, 2024-09-14T06:17:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086290477000000004, 2024-09-14T06:17:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38994 2024-09-14T06:17:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:17:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3e-07 2024-09-14T06:17:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680381+0000", 2024-09-14T06:17:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.439e-06 2024-09-14T06:17:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680438+0000", 2024-09-14T06:17:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6693000000000001e-05 2024-09-14T06:17:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680446+0000", 2024-09-14T06:17:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9160000000000001e-06 2024-09-14T06:17:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766490+0000", 2024-09-14T06:17:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086044617000000004 2024-09-14T06:17:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766563+0000", 2024-09-14T06:17:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2430999999999998e-05 2024-09-14T06:17:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766667+0000", 2024-09-14T06:17:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000104651 2024-09-14T06:17:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38995 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 204800~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:17:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385666821999997, 2024-09-14T06:17:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086415360999999996, 2024-09-14T06:17:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38995 2024-09-14T06:17:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:17:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:17:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680461+0000", 2024-09-14T06:17:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:17:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680464+0000", 2024-09-14T06:17:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4929999999999998e-06 2024-09-14T06:17:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680479+0000", 2024-09-14T06:17:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5322999999999999e-05 2024-09-14T06:17:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680496+0000", 2024-09-14T06:17:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6801000000000001e-05 2024-09-14T06:17:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766699+0000", 2024-09-14T06:17:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086203265000000001 2024-09-14T06:17:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766777+0000", 2024-09-14T06:17:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7284000000000001e-05 2024-09-14T06:17:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766875+0000", 2024-09-14T06:17:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8461000000000005e-05 2024-09-14T06:17:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38996 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 212992~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:17:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385619382999998, 2024-09-14T06:17:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086576474, 2024-09-14T06:17:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38996 2024-09-14T06:17:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:17:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:17:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680508+0000", 2024-09-14T06:17:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1799999999999995e-07 2024-09-14T06:17:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680511+0000", 2024-09-14T06:17:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9739999999999998e-06 2024-09-14T06:17:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680520+0000", 2024-09-14T06:17:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.329e-06 2024-09-14T06:17:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680528+0000", 2024-09-14T06:17:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4660000000000002e-06 2024-09-14T06:17:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766907+0000", 2024-09-14T06:17:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086379529999999996 2024-09-14T06:17:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766987+0000", 2024-09-14T06:17:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0280999999999999e-05 2024-09-14T06:17:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767084+0000", 2024-09-14T06:17:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6275999999999998e-05 2024-09-14T06:17:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38997 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 221184~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385590041999997, 2024-09-14T06:17:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086757327999999995, 2024-09-14T06:17:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38997 2024-09-14T06:17:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:17:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:17:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680540+0000", 2024-09-14T06:17:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6910000000000002e-06 2024-09-14T06:17:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680548+0000", 2024-09-14T06:17:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7339999999999997e-06 2024-09-14T06:17:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680555+0000", 2024-09-14T06:17:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0900000000000001e-06 2024-09-14T06:17:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767116+0000", 2024-09-14T06:17:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086561551 2024-09-14T06:17:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767192+0000", 2024-09-14T06:17:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5504000000000001e-05 2024-09-14T06:17:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767294+0000", 2024-09-14T06:17:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000102308 2024-09-14T06:17:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38998 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 229376~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:17:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385563431000001, 2024-09-14T06:17:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087010193999999999, 2024-09-14T06:17:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38998 2024-09-14T06:17:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:17:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:17:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680564+0000", 2024-09-14T06:17:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1800000000000001e-07 2024-09-14T06:17:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680566+0000", 2024-09-14T06:17:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.503e-06 2024-09-14T06:17:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680574+0000", 2024-09-14T06:17:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3979999999999999e-06 2024-09-14T06:17:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680580+0000", 2024-09-14T06:17:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4859999999999997e-06 2024-09-14T06:17:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767359+0000", 2024-09-14T06:17:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086779447999999995 2024-09-14T06:17:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767445+0000", 2024-09-14T06:17:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6007000000000005e-05 2024-09-14T06:17:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767573+0000", 2024-09-14T06:17:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012793400000000001 2024-09-14T06:17:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38999 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 237568~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:17:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385534567000001, 2024-09-14T06:17:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087211699000000004, 2024-09-14T06:17:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38999 2024-09-14T06:17:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:17:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:17:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680593+0000", 2024-09-14T06:17:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7700000000000005e-07 2024-09-14T06:17:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680600+0000", 2024-09-14T06:17:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7130000000000005e-06 2024-09-14T06:17:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680610+0000", 2024-09-14T06:17:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.55e-06 2024-09-14T06:17:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680616+0000", 2024-09-14T06:17:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3069999999999999e-06 2024-09-14T06:17:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767607+0000", 2024-09-14T06:17:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086990703000000003 2024-09-14T06:17:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767682+0000", 2024-09-14T06:17:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4944999999999995e-05 2024-09-14T06:17:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767804+0000", 2024-09-14T06:17:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012200400000000001 2024-09-14T06:17:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39000 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 245760~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385500944999997, 2024-09-14T06:17:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087408649000000005, 2024-09-14T06:17:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39000 2024-09-14T06:17:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:56.133 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.133 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.133 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:17:56.133 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1699999999999998e-07 2024-09-14T06:17:56.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.133 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680629+0000", 2024-09-14T06:17:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5469999999999999e-06 2024-09-14T06:17:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680636+0000", 2024-09-14T06:17:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.973e-06 2024-09-14T06:17:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680642+0000", 2024-09-14T06:17:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0349999999999998e-06 2024-09-14T06:17:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767839+0000", 2024-09-14T06:17:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087197067000000003 2024-09-14T06:17:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767913+0000", 2024-09-14T06:17:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3832999999999996e-05 2024-09-14T06:17:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768034+0000", 2024-09-14T06:17:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012167699999999999 2024-09-14T06:17:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39001 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 253952~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:17:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385476502000003, 2024-09-14T06:17:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087623312999999994, 2024-09-14T06:17:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39001 2024-09-14T06:17:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:17:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:17:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680651+0000", 2024-09-14T06:17:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7700000000000004e-07 2024-09-14T06:17:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680653+0000", 2024-09-14T06:17:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5739999999999998e-06 2024-09-14T06:17:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680702+0000", 2024-09-14T06:17:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8476000000000001e-05 2024-09-14T06:17:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680709+0000", 2024-09-14T06:17:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8170000000000001e-06 2024-09-14T06:17:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768066+0000", 2024-09-14T06:17:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087357852999999999 2024-09-14T06:17:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768163+0000", 2024-09-14T06:17:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6869999999999999e-05 2024-09-14T06:17:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768273+0000", 2024-09-14T06:17:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000110146 2024-09-14T06:17:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39002 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 262144~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:17:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385396395999997, 2024-09-14T06:17:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088150478000000004, 2024-09-14T06:17:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39002 2024-09-14T06:17:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:17:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:17:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680739+0000", 2024-09-14T06:17:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3979999999999999e-06 2024-09-14T06:17:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680742+0000", 2024-09-14T06:17:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8349999999999997e-06 2024-09-14T06:17:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680751+0000", 2024-09-14T06:17:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8599999999999999e-06 2024-09-14T06:17:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680758+0000", 2024-09-14T06:17:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4790000000000001e-06 2024-09-14T06:17:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768307+0000", 2024-09-14T06:17:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087549342000000002 2024-09-14T06:17:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768392+0000", 2024-09-14T06:17:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.441e-05 2024-09-14T06:17:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768881+0000", 2024-09-14T06:17:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000489154 2024-09-14T06:17:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39003 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 270336~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385344105999998, 2024-09-14T06:17:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088375555999999994, 2024-09-14T06:17:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39003 2024-09-14T06:17:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:17:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7000000000000004e-07 2024-09-14T06:17:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680786+0000", 2024-09-14T06:17:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:17:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680794+0000", 2024-09-14T06:17:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9840000000000001e-06 2024-09-14T06:17:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680800+0000", 2024-09-14T06:17:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1550000000000004e-06 2024-09-14T06:17:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768965+0000", 2024-09-14T06:17:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088165091000000001 2024-09-14T06:17:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769059+0000", 2024-09-14T06:17:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3923000000000002e-05 2024-09-14T06:17:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769158+0000", 2024-09-14T06:17:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8820000000000006e-05 2024-09-14T06:17:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39004 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 278528~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:17:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385316418000002, 2024-09-14T06:17:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088499624999999998, 2024-09-14T06:17:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39004 2024-09-14T06:17:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:17:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:17:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680811+0000", 2024-09-14T06:17:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:17:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680814+0000", 2024-09-14T06:17:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:17:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680821+0000", 2024-09-14T06:17:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9009999999999997e-06 2024-09-14T06:17:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680826+0000", 2024-09-14T06:17:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5350000000000004e-06 2024-09-14T06:17:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769189+0000", 2024-09-14T06:17:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088363024999999998 2024-09-14T06:17:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769246+0000", 2024-09-14T06:17:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7017e-05 2024-09-14T06:17:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769310+0000", 2024-09-14T06:17:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3541999999999996e-05 2024-09-14T06:17:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39005 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 286720~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:17:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385292333999999, 2024-09-14T06:17:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088681092000000003, 2024-09-14T06:17:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39005 2024-09-14T06:17:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:17:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:17:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680835+0000", 2024-09-14T06:17:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2699999999999999e-07 2024-09-14T06:17:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680837+0000", 2024-09-14T06:17:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.413e-06 2024-09-14T06:17:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680844+0000", 2024-09-14T06:17:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8820000000000003e-06 2024-09-14T06:17:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680850+0000", 2024-09-14T06:17:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5169999999999999e-06 2024-09-14T06:17:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769337+0000", 2024-09-14T06:17:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088487471999999998 2024-09-14T06:17:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769404+0000", 2024-09-14T06:17:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7111000000000002e-05 2024-09-14T06:17:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769515+0000", 2024-09-14T06:17:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011116999999999999 2024-09-14T06:17:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39006 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 294912~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:17:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385269170999997, 2024-09-14T06:17:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088886572999999997, 2024-09-14T06:17:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39006 2024-09-14T06:17:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:17:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:17:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680858+0000", 2024-09-14T06:17:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7e-07 2024-09-14T06:17:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680860+0000", 2024-09-14T06:17:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:17:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680867+0000", 2024-09-14T06:17:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0430000000000002e-06 2024-09-14T06:17:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680873+0000", 2024-09-14T06:17:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7389999999999996e-06 2024-09-14T06:17:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769547+0000", 2024-09-14T06:17:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088673830999999995 2024-09-14T06:17:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769631+0000", 2024-09-14T06:17:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3660999999999997e-05 2024-09-14T06:17:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769744+0000", 2024-09-14T06:17:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000113482 2024-09-14T06:17:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39007 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 303104~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:17:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.385245759, 2024-09-14T06:17:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.089077397000000003, 2024-09-14T06:17:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39007 2024-09-14T06:17:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:17:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:17:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687953+0000", 2024-09-14T06:17:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00707203 2024-09-14T06:17:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687960+0000", 2024-09-14T06:17:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9890000000000003e-06 2024-09-14T06:17:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687974+0000", 2024-09-14T06:17:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4226999999999999e-05 2024-09-14T06:17:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687983+0000", 2024-09-14T06:17:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9800000000000004e-06 2024-09-14T06:17:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769775+0000", 2024-09-14T06:17:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.081792089999999998 2024-09-14T06:17:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769849+0000", 2024-09-14T06:17:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3925999999999997e-05 2024-09-14T06:17:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769958+0000", 2024-09-14T06:17:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010915499999999999 2024-09-14T06:17:56.169 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.169 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.169 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.169 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.169 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.169 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:56.170 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:17:56.171 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:56.168+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:56.266 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-09-14T06:17:56.271 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:56.269+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:56.364 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:56.365 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:56.365 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:56.365 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:56.372 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:56.372+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:56.384 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:17:56.473 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:56.469+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:56.483 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:17:56.483 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:17:56.483 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-14T06:17:56.483 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-14T06:17:56.512 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:56.512 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:56.512 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:56.512 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:56.512 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:56.527 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:17:56.574 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:56.572+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:56.639 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-14T06:17:56.639 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-14T06:17:56.639 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-14T06:17:56.639 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-14T06:17:56.639 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-14T06:17:56.639 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-14T06:17:56.639 INFO:teuthology.orchestra.run.smithi039.stderr:bench [] [] [] [] 2024-09-14T06:17:56.639 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:17:56.639 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-14T06:17:56.639 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-14T06:17:56.640 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-14T06:17:56.640 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2024-09-14T06:17:56.642 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:17:56.642 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:17:56.642 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-14T06:17:56.675 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:56.672+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:56.695 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:56.695 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:56.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:56.695 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:56.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3916 3.16 3:6ab8b1d8:::rbd_data.1152bd92e04f.0000000000000002:head [stat,truncate 1590169] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:17:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:17:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 89.114939696999997, 2024-09-14T06:17:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022958078999999999, 2024-09-14T06:17:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3916 2024-09-14T06:17:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:17:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:17:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579203+0000", 2024-09-14T06:17:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.748e-06 2024-09-14T06:17:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579208+0000", 2024-09-14T06:17:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:17:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579223+0000", 2024-09-14T06:17:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.508e-05 2024-09-14T06:17:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579232+0000", 2024-09-14T06:17:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9719999999999998e-06 2024-09-14T06:17:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579307+0000", 2024-09-14T06:17:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5028000000000001e-05 2024-09-14T06:17:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579412+0000", 2024-09-14T06:17:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010525400000000001 2024-09-14T06:17:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:17:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579535+0000", 2024-09-14T06:17:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122726 2024-09-14T06:17:56.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.700 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:56.700 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.581415+0000", 2024-09-14T06:17:56.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018800360000000001 2024-09-14T06:17:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602017+0000", 2024-09-14T06:17:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.020602425000000001 2024-09-14T06:17:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602096+0000", 2024-09-14T06:17:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8968999999999996e-05 2024-09-14T06:17:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602159+0000", 2024-09-14T06:17:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2824999999999996e-05 2024-09-14T06:17:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:7151 3.e 3:7793b17d:::rbd_data.1152f87cbac6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:17:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:17:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 85.364543830000002, 2024-09-14T06:17:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.029556783, 2024-09-14T06:17:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 7151 2024-09-14T06:17:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:17:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:17:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329599+0000", 2024-09-14T06:17:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5270000000000001e-06 2024-09-14T06:17:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329610+0000", 2024-09-14T06:17:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0431000000000001e-05 2024-09-14T06:17:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329634+0000", 2024-09-14T06:17:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3617000000000001e-05 2024-09-14T06:17:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329644+0000", 2024-09-14T06:17:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.095e-05 2024-09-14T06:17:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329712+0000", 2024-09-14T06:17:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7067e-05 2024-09-14T06:17:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329827+0000", 2024-09-14T06:17:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011498 2024-09-14T06:17:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:17:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.334271+0000", 2024-09-14T06:17:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044445839999999997 2024-09-14T06:17:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.358967+0000", 2024-09-14T06:17:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024696356999999999 2024-09-14T06:17:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359027+0000", 2024-09-14T06:17:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0006000000000001e-05 2024-09-14T06:17:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359099+0000", 2024-09-14T06:17:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1052999999999999e-05 2024-09-14T06:17:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359154+0000", 2024-09-14T06:17:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5210999999999997e-05 2024-09-14T06:17:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:7151 3.e e32/29 ondisk, result = 0)", 2024-09-14T06:17:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:17:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 85.344599396000007, 2024-09-14T06:17:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0096251340000000005, 2024-09-14T06:17:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:17:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:17:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349543+0000", 2024-09-14T06:17:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:17:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349552+0000", 2024-09-14T06:17:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1039999999999996e-06 2024-09-14T06:17:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349579+0000", 2024-09-14T06:17:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6449999999999999e-05 2024-09-14T06:17:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349593+0000", 2024-09-14T06:17:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4099e-05 2024-09-14T06:17:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359003+0000", 2024-09-14T06:17:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0094096119999999995 2024-09-14T06:17:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359014+0000", 2024-09-14T06:17:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0871e-05 2024-09-14T06:17:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359167+0000", 2024-09-14T06:17:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015300700000000001 2024-09-14T06:17:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15619 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'124, mlcod=36'119)", 2024-09-14T06:17:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 80.396286234000002, 2024-09-14T06:17:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061545389999999997, 2024-09-14T06:17:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:17:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3399999999999998e-07 2024-09-14T06:17:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297861+0000", 2024-09-14T06:17:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7640000000000002e-06 2024-09-14T06:17:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297876+0000", 2024-09-14T06:17:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4881e-05 2024-09-14T06:17:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297884+0000", 2024-09-14T06:17:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.588e-06 2024-09-14T06:17:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298294+0000", 2024-09-14T06:17:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00041011200000000002 2024-09-14T06:17:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298317+0000", 2024-09-14T06:17:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3417999999999998e-05 2024-09-14T06:17:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303972+0000", 2024-09-14T06:17:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056548680000000004 2024-09-14T06:17:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304009+0000", 2024-09-14T06:17:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7073999999999998e-05 2024-09-14T06:17:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15621 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'125, mlcod=36'119)", 2024-09-14T06:17:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 80.394775217000003, 2024-09-14T06:17:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0069496169999999999, 2024-09-14T06:17:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:17:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2200000000000003e-07 2024-09-14T06:17:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299372+0000", 2024-09-14T06:17:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3850000000000001e-06 2024-09-14T06:17:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299387+0000", 2024-09-14T06:17:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4905e-05 2024-09-14T06:17:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299393+0000", 2024-09-14T06:17:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9090000000000003e-06 2024-09-14T06:17:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303591+0000", 2024-09-14T06:17:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041976640000000003 2024-09-14T06:17:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303619+0000", 2024-09-14T06:17:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7651e-05 2024-09-14T06:17:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306287+0000", 2024-09-14T06:17:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026686689999999998 2024-09-14T06:17:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306315+0000", 2024-09-14T06:17:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7712000000000002e-05 2024-09-14T06:17:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:24229 3.5 e47/34 3:a30fd30b:::rbd_data.1152db28e106.0000000000000001:head v 47'55, mlcod=45'53)", 2024-09-14T06:17:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:17:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.784129618999998, 2024-09-14T06:17:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0092924199999999992, 2024-09-14T06:17:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:17:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:17:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910013+0000", 2024-09-14T06:17:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.812e-06 2024-09-14T06:17:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915374+0000", 2024-09-14T06:17:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053609240000000004 2024-09-14T06:17:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915395+0000", 2024-09-14T06:17:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1118e-05 2024-09-14T06:17:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915406+0000", 2024-09-14T06:17:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1303e-05 2024-09-14T06:17:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915490+0000", 2024-09-14T06:17:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4017000000000006e-05 2024-09-14T06:17:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915515+0000", 2024-09-14T06:17:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4712e-05 2024-09-14T06:17:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919240+0000", 2024-09-14T06:17:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037247690000000002 2024-09-14T06:17:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919304+0000", 2024-09-14T06:17:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3764999999999997e-05 2024-09-14T06:17:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:head, version: 31'47, data_included: [0~98304,360448~278528], data_size: 376832, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:head@31'47, size: 638976, copy_subset: [(0, 638976)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 638976, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:17:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:17:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 56.483866673999998, 2024-09-14T06:17:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062487460000000003, 2024-09-14T06:17:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:17:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:17:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210275+0000", 2024-09-14T06:17:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:17:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210752+0000", 2024-09-14T06:17:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047772100000000002 2024-09-14T06:17:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210781+0000", 2024-09-14T06:17:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8391999999999999e-05 2024-09-14T06:17:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210805+0000", 2024-09-14T06:17:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4142999999999999e-05 2024-09-14T06:17:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.213641+0000", 2024-09-14T06:17:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002835827 2024-09-14T06:17:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.213663+0000", 2024-09-14T06:17:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.243e-05 2024-09-14T06:17:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.216523+0000", 2024-09-14T06:17:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028597729999999999 2024-09-14T06:17:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:50564 3.16 3:689e8220:::rbd_header.1152757de986:head [call rbd.snapshot_add in=15b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-14T06:17:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:13.319284+0000", 2024-09-14T06:17:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 43.374856293999997, 2024-09-14T06:17:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.030881613, 2024-09-14T06:17:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50564 2024-09-14T06:17:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319284+0000", 2024-09-14T06:17:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319284+0000", 2024-09-14T06:17:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319286+0000", 2024-09-14T06:17:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.849e-06 2024-09-14T06:17:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319293+0000", 2024-09-14T06:17:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.72e-06 2024-09-14T06:17:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319313+0000", 2024-09-14T06:17:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9833000000000001e-05 2024-09-14T06:17:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319325+0000", 2024-09-14T06:17:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1924e-05 2024-09-14T06:17:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319441+0000", 2024-09-14T06:17:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000115952 2024-09-14T06:17:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op must wait for map", 2024-09-14T06:17:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319453+0000", 2024-09-14T06:17:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1810999999999999e-05 2024-09-14T06:17:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322905+0000", 2024-09-14T06:17:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034519680000000001 2024-09-14T06:17:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322917+0000", 2024-09-14T06:17:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1946999999999999e-05 2024-09-14T06:17:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.342215+0000", 2024-09-14T06:17:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0192988300000001 2024-09-14T06:17:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.342451+0000", 2024-09-14T06:17:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00023546 2024-09-14T06:17:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.342959+0000", 2024-09-14T06:17:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050845999999999997 2024-09-14T06:17:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.347472+0000", 2024-09-14T06:17:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045125310000000002 2024-09-14T06:17:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.349544+0000", 2024-09-14T06:17:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020721730000000001 2024-09-14T06:17:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.349727+0000", 2024-09-14T06:17:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000182776 2024-09-14T06:17:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.350166+0000", 2024-09-14T06:17:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000439379 2024-09-14T06:17:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:50565 3.16 3:689e8220:::rbd_header.1152757de986:head [watch reconnect cookie 140175519011472 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-14T06:17:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:13.319351+0000", 2024-09-14T06:17:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 43.374789716000002, 2024-09-14T06:17:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0352714430000001, 2024-09-14T06:17:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50565 2024-09-14T06:17:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319351+0000", 2024-09-14T06:17:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319351+0000", 2024-09-14T06:17:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319352+0000", 2024-09-14T06:17:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0500000000000003e-07 2024-09-14T06:17:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319355+0000", 2024-09-14T06:17:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3960000000000002e-06 2024-09-14T06:17:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319367+0000", 2024-09-14T06:17:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1518999999999999e-05 2024-09-14T06:17:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319373+0000", 2024-09-14T06:17:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7800000000000003e-06 2024-09-14T06:17:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319517+0000", 2024-09-14T06:17:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000143719 2024-09-14T06:17:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting_for_map not empty", 2024-09-14T06:17:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319520+0000", 2024-09-14T06:17:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1990000000000002e-06 2024-09-14T06:17:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322963+0000", 2024-09-14T06:17:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034424920000000001 2024-09-14T06:17:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322971+0000", 2024-09-14T06:17:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0860000000000002e-06 2024-09-14T06:17:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343565+0000", 2024-09-14T06:17:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0205935530000001 2024-09-14T06:17:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for rw locks", 2024-09-14T06:17:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343675+0000", 2024-09-14T06:17:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011048900000000001 2024-09-14T06:17:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.350295+0000", 2024-09-14T06:17:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006620094 2024-09-14T06:17:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.350584+0000", 2024-09-14T06:17:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00028850000000000002 2024-09-14T06:17:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.354623+0000", 2024-09-14T06:17:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040389110000000001 2024-09-14T06:17:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:50566 3.16 3:689e8220:::rbd_header.1152757de986:head [watch ping cookie 140175519011472 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-14T06:17:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:13.960177+0000", 2024-09-14T06:17:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.733963568999997, 2024-09-14T06:17:56.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.39354602500000002, 2024-09-14T06:17:56.746 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.746 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.746 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.746 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.746 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.746 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.747 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50566 2024-09-14T06:17:56.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.747 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960177+0000", 2024-09-14T06:17:56.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960177+0000", 2024-09-14T06:17:56.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960178+0000", 2024-09-14T06:17:56.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.136e-06 2024-09-14T06:17:56.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960186+0000", 2024-09-14T06:17:56.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0299999999999994e-06 2024-09-14T06:17:56.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960202+0000", 2024-09-14T06:17:56.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5238999999999999e-05 2024-09-14T06:17:56.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960218+0000", 2024-09-14T06:17:56.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.668e-05 2024-09-14T06:17:56.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960350+0000", 2024-09-14T06:17:56.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013201 2024-09-14T06:17:56.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.753 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:56.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960368+0000", 2024-09-14T06:17:56.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7526000000000001e-05 2024-09-14T06:17:56.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343805+0000", 2024-09-14T06:17:56.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.38343735400000001 2024-09-14T06:17:56.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for rw locks", 2024-09-14T06:17:56.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343907+0000", 2024-09-14T06:17:56.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010133599999999999 2024-09-14T06:17:56.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.352997+0000", 2024-09-14T06:17:56.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0090907409999999994 2024-09-14T06:17:56.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.353352+0000", 2024-09-14T06:17:56.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00035436000000000002 2024-09-14T06:17:56.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.353723+0000", 2024-09-14T06:17:56.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00037161300000000002 2024-09-14T06:17:56.757 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.757 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.757 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.758 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:51037 3.11 3:89b51a5c:::rbd_data.11522fd7a0c5.0000000000000059:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-14T06:17:56.758 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:15.038049+0000", 2024-09-14T06:17:56.758 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.656092227000002, 2024-09-14T06:17:56.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.075311116999999997, 2024-09-14T06:17:56.759 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.759 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.759 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:56.759 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:56.759 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:56.759 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 51037 2024-09-14T06:17:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038049+0000", 2024-09-14T06:17:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038049+0000", 2024-09-14T06:17:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038051+0000", 2024-09-14T06:17:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:17:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038065+0000", 2024-09-14T06:17:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3576e-05 2024-09-14T06:17:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038097+0000", 2024-09-14T06:17:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2566e-05 2024-09-14T06:17:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038121+0000", 2024-09-14T06:17:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3207e-05 2024-09-14T06:17:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038291+0000", 2024-09-14T06:17:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016996900000000001 2024-09-14T06:17:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038543+0000", 2024-09-14T06:17:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00025233999999999999 2024-09-14T06:17:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.053218+0000", 2024-09-14T06:17:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014674957000000001 2024-09-14T06:17:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113195+0000", 2024-09-14T06:17:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.059976697000000002 2024-09-14T06:17:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113212+0000", 2024-09-14T06:17:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7299000000000002e-05 2024-09-14T06:17:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113267+0000", 2024-09-14T06:17:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4632999999999997e-05 2024-09-14T06:17:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113360+0000", 2024-09-14T06:17:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3129999999999998e-05 2024-09-14T06:17:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:51037 3.11 e67/66 ondisk, result = 0)", 2024-09-14T06:17:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:15.093818+0000", 2024-09-14T06:17:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 41.600322435999999, 2024-09-14T06:17:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.019384384000000001, 2024-09-14T06:17:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093818+0000", 2024-09-14T06:17:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093818+0000", 2024-09-14T06:17:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093820+0000", 2024-09-14T06:17:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0169999999999999e-06 2024-09-14T06:17:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093829+0000", 2024-09-14T06:17:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6500000000000002e-06 2024-09-14T06:17:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093860+0000", 2024-09-14T06:17:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1275999999999997e-05 2024-09-14T06:17:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093883+0000", 2024-09-14T06:17:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2464999999999999e-05 2024-09-14T06:17:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113178+0000", 2024-09-14T06:17:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.019295643000000001 2024-09-14T06:17:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113189+0000", 2024-09-14T06:17:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0285999999999999e-05 2024-09-14T06:17:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113203+0000", 2024-09-14T06:17:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4047e-05 2024-09-14T06:17:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:52268 3.e e70/34 3:7013111b:::rbd_data.1152cd930a87.0000000000000001:head v 70'315, mlcod=70'313)", 2024-09-14T06:17:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:17.975434+0000", 2024-09-14T06:17:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.718707238, 2024-09-14T06:17:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014816765000000001, 2024-09-14T06:17:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.975434+0000", 2024-09-14T06:17:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.975434+0000", 2024-09-14T06:17:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.975436+0000", 2024-09-14T06:17:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3010000000000001e-06 2024-09-14T06:17:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982453+0000", 2024-09-14T06:17:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0070167909999999997 2024-09-14T06:17:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982489+0000", 2024-09-14T06:17:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6603000000000002e-05 2024-09-14T06:17:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982507+0000", 2024-09-14T06:17:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8074e-05 2024-09-14T06:17:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982610+0000", 2024-09-14T06:17:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010224 2024-09-14T06:17:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982639+0000", 2024-09-14T06:17:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9009999999999998e-05 2024-09-14T06:17:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.990188+0000", 2024-09-14T06:17:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0075497150000000002 2024-09-14T06:17:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.990250+0000", 2024-09-14T06:17:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2031000000000003e-05 2024-09-14T06:17:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:52269 3.1f e70/56 3:fd802c48:::rbd_data.1152cd930a87.0000000000000002:head v 70'321, mlcod=70'318)", 2024-09-14T06:17:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:17.982671+0000", 2024-09-14T06:17:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.711470255999998, 2024-09-14T06:17:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016453552, 2024-09-14T06:17:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982671+0000", 2024-09-14T06:17:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982671+0000", 2024-09-14T06:17:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982672+0000", 2024-09-14T06:17:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.246e-06 2024-09-14T06:17:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991501+0000", 2024-09-14T06:17:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0088292590000000008 2024-09-14T06:17:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991521+0000", 2024-09-14T06:17:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0211e-05 2024-09-14T06:17:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991531+0000", 2024-09-14T06:17:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0207000000000001e-05 2024-09-14T06:17:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991605+0000", 2024-09-14T06:17:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3066999999999997e-05 2024-09-14T06:17:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991633+0000", 2024-09-14T06:17:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8136e-05 2024-09-14T06:17:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.999052+0000", 2024-09-14T06:17:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0074192320000000004 2024-09-14T06:17:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.999124+0000", 2024-09-14T06:17:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2193999999999994e-05 2024-09-14T06:17:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:59899 3.9 e75/68 3:96b79e96:::rbd_data.115214b4562.0000000000000004:head v 75'319, mlcod=75'317)", 2024-09-14T06:17:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:24.822693+0000", 2024-09-14T06:17:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 31.871447405000001, 2024-09-14T06:17:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014246999, 2024-09-14T06:17:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.822693+0000", 2024-09-14T06:17:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.822693+0000", 2024-09-14T06:17:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.822695+0000", 2024-09-14T06:17:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.762e-06 2024-09-14T06:17:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831397+0000", 2024-09-14T06:17:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0087016230000000003 2024-09-14T06:17:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831424+0000", 2024-09-14T06:17:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7480000000000001e-05 2024-09-14T06:17:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831435+0000", 2024-09-14T06:17:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0943e-05 2024-09-14T06:17:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831509+0000", 2024-09-14T06:17:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3596000000000005e-05 2024-09-14T06:17:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831531+0000", 2024-09-14T06:17:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2628000000000001e-05 2024-09-14T06:17:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.836873+0000", 2024-09-14T06:17:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053417849999999999 2024-09-14T06:17:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.836940+0000", 2024-09-14T06:17:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7181999999999995e-05 2024-09-14T06:17:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:64148 3.2 e86/29 3:41d40d1e:::rbd_data.115267b2c56c.0000000000000000:head v 86'276, mlcod=80'274)", 2024-09-14T06:17:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:38.879703+0000", 2024-09-14T06:17:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 17.814437776999998, 2024-09-14T06:17:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014921665000000001, 2024-09-14T06:17:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.879703+0000", 2024-09-14T06:17:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.879703+0000", 2024-09-14T06:17:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.879704+0000", 2024-09-14T06:17:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0219999999999999e-06 2024-09-14T06:17:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885314+0000", 2024-09-14T06:17:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056102119999999998 2024-09-14T06:17:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885331+0000", 2024-09-14T06:17:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6696000000000001e-05 2024-09-14T06:17:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885342+0000", 2024-09-14T06:17:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0886999999999999e-05 2024-09-14T06:17:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885468+0000", 2024-09-14T06:17:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012655399999999999 2024-09-14T06:17:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885517+0000", 2024-09-14T06:17:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9057999999999997e-05 2024-09-14T06:17:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.894572+0000", 2024-09-14T06:17:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0090541130000000008 2024-09-14T06:17:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.894625+0000", 2024-09-14T06:17:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3122999999999999e-05 2024-09-14T06:17:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:64152 3.3 e86/29 3:c1c8d963:::rbd_data.115267b2c56c.0000000000000002:head v 86'180, mlcod=79'178)", 2024-09-14T06:17:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:38.884954+0000", 2024-09-14T06:17:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 17.809186315000002, 2024-09-14T06:17:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011982222000000001, 2024-09-14T06:17:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.884954+0000", 2024-09-14T06:17:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.884954+0000", 2024-09-14T06:17:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.884956+0000", 2024-09-14T06:17:56.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1179999999999999e-06 2024-09-14T06:17:56.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886783+0000", 2024-09-14T06:17:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018269289999999999 2024-09-14T06:17:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886795+0000", 2024-09-14T06:17:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1975e-05 2024-09-14T06:17:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886802+0000", 2024-09-14T06:17:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.824e-06 2024-09-14T06:17:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886881+0000", 2024-09-14T06:17:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8702000000000006e-05 2024-09-14T06:17:56.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886902+0000", 2024-09-14T06:17:56.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0854000000000001e-05 2024-09-14T06:17:56.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:56.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.896887+0000", 2024-09-14T06:17:56.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0099853700000000004 2024-09-14T06:17:56.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.896937+0000", 2024-09-14T06:17:56.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9450000000000003e-05 2024-09-14T06:17:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:64155 3.1 e86/29 3:86008c25:::rbd_data.115267b2c56c.0000000000000003:head v 86'451, mlcod=78'449)", 2024-09-14T06:17:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:17:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 17.791085229, 2024-09-14T06:17:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006333038, 2024-09-14T06:17:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:17:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:17:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:17:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:17:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906826+0000", 2024-09-14T06:17:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037703260000000001 2024-09-14T06:17:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906845+0000", 2024-09-14T06:17:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8488999999999999e-05 2024-09-14T06:17:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906857+0000", 2024-09-14T06:17:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2571e-05 2024-09-14T06:17:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906966+0000", 2024-09-14T06:17:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010832000000000001 2024-09-14T06:17:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906984+0000", 2024-09-14T06:17:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8068999999999999e-05 2024-09-14T06:17:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.909306+0000", 2024-09-14T06:17:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023221050000000001 2024-09-14T06:17:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.909389+0000", 2024-09-14T06:17:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2865e-05 2024-09-14T06:17:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(osd.5.0:1871 3.1e e99/29 3:783464bd:::rbd_data.11522fd7a0c5.000000000000005b:19 v 99'341, mlcod=95'338)", 2024-09-14T06:17:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:53.229369+0000", 2024-09-14T06:17:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.4647716979999998, 2024-09-14T06:17:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0063464239999999998, 2024-09-14T06:17:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229369+0000", 2024-09-14T06:17:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229369+0000", 2024-09-14T06:17:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229370+0000", 2024-09-14T06:17:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7300000000000005e-07 2024-09-14T06:17:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229388+0000", 2024-09-14T06:17:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8006999999999999e-05 2024-09-14T06:17:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229406+0000", 2024-09-14T06:17:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7663e-05 2024-09-14T06:17:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229417+0000", 2024-09-14T06:17:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1749999999999999e-05 2024-09-14T06:17:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230797+0000", 2024-09-14T06:17:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0013801589999999999 2024-09-14T06:17:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.233207+0000", 2024-09-14T06:17:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024092889999999998 2024-09-14T06:17:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.235660+0000", 2024-09-14T06:17:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024534370000000002 2024-09-14T06:17:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.235716+0000", 2024-09-14T06:17:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5346000000000002e-05 2024-09-14T06:17:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(osd.0.0:2754 3.e e99/34 3:70005258:::rbd_data.11522fd7a0c5.000000000000018a:19 v 99'690, mlcod=98'687)", 2024-09-14T06:17:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:53.230299+0000", 2024-09-14T06:17:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 3.4638417850000001, 2024-09-14T06:17:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064664880000000003, 2024-09-14T06:17:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230299+0000", 2024-09-14T06:17:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230299+0000", 2024-09-14T06:17:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230299+0000", 2024-09-14T06:17:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.32e-07 2024-09-14T06:17:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230304+0000", 2024-09-14T06:17:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8729999999999998e-06 2024-09-14T06:17:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230316+0000", 2024-09-14T06:17:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2104e-05 2024-09-14T06:17:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230325+0000", 2024-09-14T06:17:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4079999999999998e-06 2024-09-14T06:17:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.809 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.809 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:56.809 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230896+0000", 2024-09-14T06:17:56.809 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057151700000000001 2024-09-14T06:17:56.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.809 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.809 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:56.809 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.234280+0000", 2024-09-14T06:17:56.809 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033832060000000001 2024-09-14T06:17:56.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.810 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.810 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:56.810 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.236724+0000", 2024-09-14T06:17:56.810 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024447090000000002 2024-09-14T06:17:56.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:56.811 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:56.811 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:56.811 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.236765+0000", 2024-09-14T06:17:56.811 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1239000000000003e-05 2024-09-14T06:17:56.811 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.811 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.812 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.812 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:56.812 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:56.812 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:56.815 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:17:56.816 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:17:56.816 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-14T06:17:56.816 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-14T06:17:56.817 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-14T06:17:56.817 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-14T06:17:56.817 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-14T06:17:56.817 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-14T06:17:56.817 INFO:teuthology.orchestra.run.smithi039.stderr:bench [] [] [] [] 2024-09-14T06:17:56.817 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:17:56.818 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-14T06:17:56.818 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-14T06:17:56.818 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-14T06:17:56.818 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2024-09-14T06:17:56.818 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:17:56.819 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-14T06:17:56.819 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:56.812+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:56.915 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:56.909+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:56.993 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:56.994 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:56.994 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:17:56.994 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:57.010 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:17:57.025 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-14T06:17:57.025 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-14T06:17:57.025 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-14T06:17:57.026 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-14T06:17:57.026 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-14T06:17:57.026 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-14T06:17:57.026 INFO:teuthology.orchestra.run.smithi039.stderr:bench [] [] [] [] 2024-09-14T06:17:57.026 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:17:57.026 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-14T06:17:57.026 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-14T06:17:57.026 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-14T06:17:57.026 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2024-09-14T06:17:57.026 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:57.021+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:57.028 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:17:57.028 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:17:57.028 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-14T06:17:57.116 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:57.116+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:57.142 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:57.142 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:17:57.142 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:17:57.142 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:17:57.142 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:57.158 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:17:57.217 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:57.213+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-14T06:17:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-14T06:17:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-14T06:17:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-14T06:17:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-14T06:17:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-14T06:17:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:bench [] [] [] [] 2024-09-14T06:17:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:17:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-14T06:17:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-14T06:17:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-14T06:17:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2024-09-14T06:17:57.227 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:17:57.228 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:17:57.228 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-14T06:17:57.285 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:17:57.285 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:17:57.285 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:17:57.285 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:17:57.285 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.290661432999997, 2024-09-14T06:17:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:17:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:17:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:17:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:17:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:17:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:17:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:17:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:17:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:17:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:17:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:17:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:17:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:17:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:17:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:17:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:17:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.290648360999995, 2024-09-14T06:17:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:17:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:17:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:17:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:17:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:17:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:17:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:17:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:17:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:17:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:17:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:17:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:17:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:17:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:17:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:17:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:17:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:17:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.290630929000002, 2024-09-14T06:17:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:17:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.296 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:57.296 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.296 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.296 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.296 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:17:57.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.296 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:17:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:17:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:17:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:17:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:17:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:17:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:17:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:17:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:17:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:17:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:17:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:17:57.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.300 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.300 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.300 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:17:57.300 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:17:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.290609824000001, 2024-09-14T06:17:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:17:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:17:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:17:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:17:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:17:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:17:57.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.304 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.304 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.304 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:17:57.304 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:17:57.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.304 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.304 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.304 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:17:57.304 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:17:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:17:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:17:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:17:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:17:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:17:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:17:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.290588716000002, 2024-09-14T06:17:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:17:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:17:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:17:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:17:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:17:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:17:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:17:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:17:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:17:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:17:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:17:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:17:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:17:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:17:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:17:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:17:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48234 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.249317423999997, 2024-09-14T06:17:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051730353999999999, 2024-09-14T06:17:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48234 2024-09-14T06:17:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:17:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.313 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.313 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032937+0000", 2024-09-14T06:17:57.313 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:17:57.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.313 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.313 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032940+0000", 2024-09-14T06:17:57.313 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0580000000000002e-06 2024-09-14T06:17:57.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.314 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.314 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.314 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032946+0000", 2024-09-14T06:17:57.314 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2960000000000004e-06 2024-09-14T06:17:57.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.315 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.315 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032951+0000", 2024-09-14T06:17:57.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.938e-06 2024-09-14T06:17:57.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074419+0000", 2024-09-14T06:17:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041467979000000002 2024-09-14T06:17:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074490+0000", 2024-09-14T06:17:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1324999999999999e-05 2024-09-14T06:17:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074859+0000", 2024-09-14T06:17:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036914899999999999 2024-09-14T06:17:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078464+0000", 2024-09-14T06:17:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036043170000000001 2024-09-14T06:17:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084547+0000", 2024-09-14T06:17:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006083097 2024-09-14T06:17:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084614+0000", 2024-09-14T06:17:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7509999999999996e-05 2024-09-14T06:17:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084667+0000", 2024-09-14T06:17:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:17:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48237 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.24923768, 2024-09-14T06:17:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051772830999999998, 2024-09-14T06:17:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48237 2024-09-14T06:17:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:17:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0499999999999999e-07 2024-09-14T06:17:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033018+0000", 2024-09-14T06:17:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0949999999999998e-06 2024-09-14T06:17:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033023+0000", 2024-09-14T06:17:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1620000000000001e-06 2024-09-14T06:17:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033026+0000", 2024-09-14T06:17:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0120000000000001e-06 2024-09-14T06:17:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075349+0000", 2024-09-14T06:17:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042323373999999997 2024-09-14T06:17:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075394+0000", 2024-09-14T06:17:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4861000000000003e-05 2024-09-14T06:17:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075578+0000", 2024-09-14T06:17:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00018414899999999999 2024-09-14T06:17:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078894+0000", 2024-09-14T06:17:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033158480000000001 2024-09-14T06:17:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084677+0000", 2024-09-14T06:17:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0057832120000000002 2024-09-14T06:17:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084752+0000", 2024-09-14T06:17:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5123000000000006e-05 2024-09-14T06:17:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084789+0000", 2024-09-14T06:17:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6690000000000003e-05 2024-09-14T06:17:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48243 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.249156554999999, 2024-09-14T06:17:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051780221000000001, 2024-09-14T06:17:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48243 2024-09-14T06:17:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:17:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:17:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033100+0000", 2024-09-14T06:17:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:17:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033108+0000", 2024-09-14T06:17:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1760000000000004e-06 2024-09-14T06:17:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033111+0000", 2024-09-14T06:17:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0790000000000001e-06 2024-09-14T06:17:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075903+0000", 2024-09-14T06:17:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042791942999999999 2024-09-14T06:17:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075987+0000", 2024-09-14T06:17:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4467000000000003e-05 2024-09-14T06:17:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.076259+0000", 2024-09-14T06:17:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027201900000000003 2024-09-14T06:17:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078947+0000", 2024-09-14T06:17:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026875879999999999 2024-09-14T06:17:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084797+0000", 2024-09-14T06:17:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0058500380000000001 2024-09-14T06:17:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084843+0000", 2024-09-14T06:17:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6298999999999998e-05 2024-09-14T06:17:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084877+0000", 2024-09-14T06:17:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4378999999999997e-05 2024-09-14T06:17:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48287 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.249116755000003, 2024-09-14T06:17:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051924706000000001, 2024-09-14T06:17:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48287 2024-09-14T06:17:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:17:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033138+0000", 2024-09-14T06:17:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6599999999999996e-07 2024-09-14T06:17:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033141+0000", 2024-09-14T06:17:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0879999999999999e-06 2024-09-14T06:17:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033147+0000", 2024-09-14T06:17:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2249999999999997e-06 2024-09-14T06:17:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033153+0000", 2024-09-14T06:17:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.066e-06 2024-09-14T06:17:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077790+0000", 2024-09-14T06:17:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.044637255000000001 2024-09-14T06:17:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077845+0000", 2024-09-14T06:17:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4410999999999998e-05 2024-09-14T06:17:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078062+0000", 2024-09-14T06:17:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021755599999999999 2024-09-14T06:17:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.080991+0000", 2024-09-14T06:17:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029282969999999998 2024-09-14T06:17:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:57.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084885+0000", 2024-09-14T06:17:57.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003894495 2024-09-14T06:17:57.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:57.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085020+0000", 2024-09-14T06:17:57.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013500799999999999 2024-09-14T06:17:57.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085062+0000", 2024-09-14T06:17:57.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1638999999999999e-05 2024-09-14T06:17:57.337 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.337 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.337 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.337 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48288 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:57.337 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:57.337 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.249093056, 2024-09-14T06:17:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051994018000000003, 2024-09-14T06:17:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48288 2024-09-14T06:17:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.339 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:57.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.339 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:17:57.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.89e-07 2024-09-14T06:17:57.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033164+0000", 2024-09-14T06:17:57.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.779e-06 2024-09-14T06:17:57.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033171+0000", 2024-09-14T06:17:57.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6279999999999999e-06 2024-09-14T06:17:57.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033176+0000", 2024-09-14T06:17:57.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1610000000000002e-06 2024-09-14T06:17:57.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078991+0000", 2024-09-14T06:17:57.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045815348999999998 2024-09-14T06:17:57.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079086+0000", 2024-09-14T06:17:57.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5282e-05 2024-09-14T06:17:57.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:57.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079397+0000", 2024-09-14T06:17:57.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031055999999999998 2024-09-14T06:17:57.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084432+0000", 2024-09-14T06:17:57.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050355599999999997 2024-09-14T06:17:57.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:57.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085073+0000", 2024-09-14T06:17:57.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064015199999999995 2024-09-14T06:17:57.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:57.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085120+0000", 2024-09-14T06:17:57.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7107000000000001e-05 2024-09-14T06:17:57.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085155+0000", 2024-09-14T06:17:57.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4950999999999998e-05 2024-09-14T06:17:57.343 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.343 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.343 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.343 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48416 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:57.344 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:57.344 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.246852644000001, 2024-09-14T06:17:57.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053690144000000002, 2024-09-14T06:17:57.344 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.344 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.344 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.344 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.344 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.344 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.344 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48416 2024-09-14T06:17:57.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.344 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.345 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.345 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:57.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.345 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.345 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:17:57.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.345 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.345 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035402+0000", 2024-09-14T06:17:57.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7999999999999995e-07 2024-09-14T06:17:57.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035405+0000", 2024-09-14T06:17:57.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3960000000000002e-06 2024-09-14T06:17:57.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035413+0000", 2024-09-14T06:17:57.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2209999999999997e-06 2024-09-14T06:17:57.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035420+0000", 2024-09-14T06:17:57.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1539999999999996e-06 2024-09-14T06:17:57.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081021+0000", 2024-09-14T06:17:57.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045600389999999998 2024-09-14T06:17:57.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081091+0000", 2024-09-14T06:17:57.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0325000000000002e-05 2024-09-14T06:17:57.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:57.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081434+0000", 2024-09-14T06:17:57.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000342519 2024-09-14T06:17:57.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:57.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085254+0000", 2024-09-14T06:17:57.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038201609999999999 2024-09-14T06:17:57.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088977+0000", 2024-09-14T06:17:57.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003722805 2024-09-14T06:17:57.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:57.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089048+0000", 2024-09-14T06:17:57.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1266999999999994e-05 2024-09-14T06:17:57.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089091+0000", 2024-09-14T06:17:57.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3325999999999998e-05 2024-09-14T06:17:57.349 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.349 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.349 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.350 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48417 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:57.350 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:57.350 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.246823974999998, 2024-09-14T06:17:57.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053805032000000003, 2024-09-14T06:17:57.350 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.350 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.350 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.350 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.350 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.350 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.350 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48417 2024-09-14T06:17:57.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.350 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.351 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.351 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:57.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.351 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.351 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:57.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.351 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.351 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:17:57.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9500000000000003e-07 2024-09-14T06:17:57.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035433+0000", 2024-09-14T06:17:57.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.796e-06 2024-09-14T06:17:57.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035442+0000", 2024-09-14T06:17:57.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7120000000000006e-06 2024-09-14T06:17:57.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035448+0000", 2024-09-14T06:17:57.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.015e-06 2024-09-14T06:17:57.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081855+0000", 2024-09-14T06:17:57.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.046407634000000003 2024-09-14T06:17:57.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081938+0000", 2024-09-14T06:17:57.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2687000000000003e-05 2024-09-14T06:17:57.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:57.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082313+0000", 2024-09-14T06:17:57.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000374872 2024-09-14T06:17:57.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:57.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087616+0000", 2024-09-14T06:17:57.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053032670000000004 2024-09-14T06:17:57.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089140+0000", 2024-09-14T06:17:57.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0015234350000000001 2024-09-14T06:17:57.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:57.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089184+0000", 2024-09-14T06:17:57.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4804e-05 2024-09-14T06:17:57.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089235+0000", 2024-09-14T06:17:57.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0315000000000002e-05 2024-09-14T06:17:57.355 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.355 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.355 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.356 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48418 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:57.356 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:57.356 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.246779836000002, 2024-09-14T06:17:57.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053936425000000003, 2024-09-14T06:17:57.356 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.356 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.356 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.356 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.356 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.356 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.356 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48418 2024-09-14T06:17:57.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.357 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.357 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:57.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.357 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:17:57.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035475+0000", 2024-09-14T06:17:57.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7000000000000003e-07 2024-09-14T06:17:57.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035479+0000", 2024-09-14T06:17:57.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2350000000000001e-06 2024-09-14T06:17:57.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035488+0000", 2024-09-14T06:17:57.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8820000000000007e-06 2024-09-14T06:17:57.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035494+0000", 2024-09-14T06:17:57.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3060000000000003e-06 2024-09-14T06:17:57.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082826+0000", 2024-09-14T06:17:57.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.047332008000000002 2024-09-14T06:17:57.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082883+0000", 2024-09-14T06:17:57.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7124999999999999e-05 2024-09-14T06:17:57.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:57.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.083385+0000", 2024-09-14T06:17:57.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050134099999999998 2024-09-14T06:17:57.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:57.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088888+0000", 2024-09-14T06:17:57.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005503127 2024-09-14T06:17:57.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089294+0000", 2024-09-14T06:17:57.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040669700000000003 2024-09-14T06:17:57.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:57.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089348+0000", 2024-09-14T06:17:57.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3729e-05 2024-09-14T06:17:57.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089410+0000", 2024-09-14T06:17:57.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2105000000000003e-05 2024-09-14T06:17:57.362 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.362 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.363 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.363 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48419 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:57.363 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:57.363 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.246702206000002, 2024-09-14T06:17:57.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054007545999999997, 2024-09-14T06:17:57.363 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.363 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.363 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.363 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.363 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.363 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.363 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48419 2024-09-14T06:17:57.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.364 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:57.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:17:57.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035552+0000", 2024-09-14T06:17:57.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4099999999999998e-07 2024-09-14T06:17:57.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035557+0000", 2024-09-14T06:17:57.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1819999999999997e-06 2024-09-14T06:17:57.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035566+0000", 2024-09-14T06:17:57.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1120000000000002e-06 2024-09-14T06:17:57.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035572+0000", 2024-09-14T06:17:57.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0430000000000004e-06 2024-09-14T06:17:57.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085304+0000", 2024-09-14T06:17:57.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049731707 2024-09-14T06:17:57.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085371+0000", 2024-09-14T06:17:57.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6771000000000002e-05 2024-09-14T06:17:57.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:57.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085601+0000", 2024-09-14T06:17:57.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022968000000000001 2024-09-14T06:17:57.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:57.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088893+0000", 2024-09-14T06:17:57.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032927780000000001 2024-09-14T06:17:57.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089467+0000", 2024-09-14T06:17:57.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057330399999999996 2024-09-14T06:17:57.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:57.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089520+0000", 2024-09-14T06:17:57.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3133000000000001e-05 2024-09-14T06:17:57.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089559+0000", 2024-09-14T06:17:57.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9195e-05 2024-09-14T06:17:57.368 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.368 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.368 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.369 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48420 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:57.369 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:57.369 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.246612976000002, 2024-09-14T06:17:57.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056401932000000002, 2024-09-14T06:17:57.369 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.369 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.369 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.369 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.370 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.370 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.370 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48420 2024-09-14T06:17:57.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.370 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.370 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.370 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:57.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.370 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.370 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:17:57.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.371 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035642+0000", 2024-09-14T06:17:57.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1519999999999999e-06 2024-09-14T06:17:57.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.371 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035647+0000", 2024-09-14T06:17:57.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2299999999999999e-06 2024-09-14T06:17:57.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035660+0000", 2024-09-14T06:17:57.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3166999999999999e-05 2024-09-14T06:17:57.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035666+0000", 2024-09-14T06:17:57.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1720000000000004e-06 2024-09-14T06:17:57.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086104+0000", 2024-09-14T06:17:57.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050437956999999999 2024-09-14T06:17:57.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086164+0000", 2024-09-14T06:17:57.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9592999999999997e-05 2024-09-14T06:17:57.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:57.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086402+0000", 2024-09-14T06:17:57.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000238314 2024-09-14T06:17:57.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:57.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089103+0000", 2024-09-14T06:17:57.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027011600000000002 2024-09-14T06:17:57.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091962+0000", 2024-09-14T06:17:57.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028581259999999999 2024-09-14T06:17:57.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:57.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092005+0000", 2024-09-14T06:17:57.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3111000000000001e-05 2024-09-14T06:17:57.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092043+0000", 2024-09-14T06:17:57.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7950000000000001e-05 2024-09-14T06:17:57.375 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.375 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.375 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.375 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48424 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:57.375 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:57.375 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.246349031000001, 2024-09-14T06:17:57.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056273173000000003, 2024-09-14T06:17:57.375 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.375 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.375 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.375 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.375 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.376 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.376 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48424 2024-09-14T06:17:57.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.376 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:57.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:57.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:17:57.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2900000000000003e-07 2024-09-14T06:17:57.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035909+0000", 2024-09-14T06:17:57.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6490000000000001e-06 2024-09-14T06:17:57.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035918+0000", 2024-09-14T06:17:57.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5089999999999999e-06 2024-09-14T06:17:57.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.378 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.378 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035924+0000", 2024-09-14T06:17:57.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7089999999999999e-06 2024-09-14T06:17:57.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.378 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.378 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087662+0000", 2024-09-14T06:17:57.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737621999999997 2024-09-14T06:17:57.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.378 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087744+0000", 2024-09-14T06:17:57.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2174000000000003e-05 2024-09-14T06:17:57.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:57.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088106+0000", 2024-09-14T06:17:57.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000361947 2024-09-14T06:17:57.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:57.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092053+0000", 2024-09-14T06:17:57.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0039473080000000001 2024-09-14T06:17:57.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092087+0000", 2024-09-14T06:17:57.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4153000000000003e-05 2024-09-14T06:17:57.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:57.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092138+0000", 2024-09-14T06:17:57.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0804999999999999e-05 2024-09-14T06:17:57.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092178+0000", 2024-09-14T06:17:57.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9668e-05 2024-09-14T06:17:57.381 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.381 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.381 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.381 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48508 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:57.381 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:57.381 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.24127395, 2024-09-14T06:17:57.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054906017000000001, 2024-09-14T06:17:57.381 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.382 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.382 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.382 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.382 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.382 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.382 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48508 2024-09-14T06:17:57.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.382 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.382 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:57.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:17:57.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040981+0000", 2024-09-14T06:17:57.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7700000000000004e-07 2024-09-14T06:17:57.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040984+0000", 2024-09-14T06:17:57.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1480000000000002e-06 2024-09-14T06:17:57.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040991+0000", 2024-09-14T06:17:57.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4580000000000004e-06 2024-09-14T06:17:57.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040997+0000", 2024-09-14T06:17:57.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7799999999999997e-06 2024-09-14T06:17:57.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089601+0000", 2024-09-14T06:17:57.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048604264000000001 2024-09-14T06:17:57.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089708+0000", 2024-09-14T06:17:57.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010681200000000001 2024-09-14T06:17:57.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:57.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090384+0000", 2024-09-14T06:17:57.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00067557499999999998 2024-09-14T06:17:57.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093879+0000", 2024-09-14T06:17:57.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034954679999999998 2024-09-14T06:17:57.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.386 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:57.386 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095764+0000", 2024-09-14T06:17:57.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018848070000000001 2024-09-14T06:17:57.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.386 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:57.386 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095837+0000", 2024-09-14T06:17:57.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3433e-05 2024-09-14T06:17:57.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.387 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095886+0000", 2024-09-14T06:17:57.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8495000000000002e-05 2024-09-14T06:17:57.387 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.387 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.387 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.387 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48537 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:17:57.388 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:57.388 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.239910180000003, 2024-09-14T06:17:57.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053681833999999998, 2024-09-14T06:17:57.388 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.388 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.388 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.388 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.388 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.389 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.389 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48537 2024-09-14T06:17:57.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.389 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:57.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.391 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:17:57.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2299999999999999e-07 2024-09-14T06:17:57.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.391 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.391 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042350+0000", 2024-09-14T06:17:57.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5690000000000004e-06 2024-09-14T06:17:57.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042361+0000", 2024-09-14T06:17:57.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1337e-05 2024-09-14T06:17:57.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042369+0000", 2024-09-14T06:17:57.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8359999999999992e-06 2024-09-14T06:17:57.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090914+0000", 2024-09-14T06:17:57.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048544945999999999 2024-09-14T06:17:57.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090995+0000", 2024-09-14T06:17:57.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0548000000000002e-05 2024-09-14T06:17:57.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:17:57.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091508+0000", 2024-09-14T06:17:57.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000513723 2024-09-14T06:17:57.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093900+0000", 2024-09-14T06:17:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023920429999999999 2024-09-14T06:17:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:17:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095944+0000", 2024-09-14T06:17:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020437379999999998 2024-09-14T06:17:57.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.396 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:17:57.396 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095987+0000", 2024-09-14T06:17:57.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2521999999999997e-05 2024-09-14T06:17:57.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.096025+0000", 2024-09-14T06:17:57.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8649000000000002e-05 2024-09-14T06:17:57.397 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.397 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.397 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.398 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:63810 3.1d 3.c887d99d (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-14T06:17:57.398 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:17:57.398 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 23.702152294000001, 2024-09-14T06:17:57.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.61346978900000004, 2024-09-14T06:17:57.398 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.398 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.398 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:17:57.398 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.399 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.399 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.399 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 63810 2024-09-14T06:17:57.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.399 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:17:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:17:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580104+0000", 2024-09-14T06:17:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4650000000000001e-06 2024-09-14T06:17:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580112+0000", 2024-09-14T06:17:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9219999999999997e-06 2024-09-14T06:17:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580130+0000", 2024-09-14T06:17:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8128999999999999e-05 2024-09-14T06:17:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580140+0000", 2024-09-14T06:17:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0285000000000001e-05 2024-09-14T06:17:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580201+0000", 2024-09-14T06:17:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0517999999999999e-05 2024-09-14T06:17:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580209+0000", 2024-09-14T06:17:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6430000000000001e-06 2024-09-14T06:17:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:34.193554+0000", 2024-09-14T06:17:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.61334482899999998 2024-09-14T06:17:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:34.193571+0000", 2024-09-14T06:17:57.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6997999999999999e-05 2024-09-14T06:17:57.404 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.404 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.404 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.404 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:63832 3.1d 3:b99be113:::rbd_object_map.11525ae33f05:head [read 0~52] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T06:17:57.404 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:17:57.404 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 20.044155908, 2024-09-14T06:17:57.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.99066972200000003, 2024-09-14T06:17:57.404 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:17:57.405 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:17:57.405 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:17:57.405 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:17:57.405 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:17:57.405 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.405 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 63832 2024-09-14T06:17:57.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.405 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:17:57.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:17:57.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:17:57.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:17:57.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:17:57.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:17:57.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.406 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:17:57.406 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238100+0000", 2024-09-14T06:17:57.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0849999999999999e-06 2024-09-14T06:17:57.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:17:57.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238108+0000", 2024-09-14T06:17:57.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.089e-06 2024-09-14T06:17:57.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:17:57.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238123+0000", 2024-09-14T06:17:57.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5418e-05 2024-09-14T06:17:57.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238136+0000", 2024-09-14T06:17:57.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2204e-05 2024-09-14T06:17:57.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238913+0000", 2024-09-14T06:17:57.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00077720399999999998 2024-09-14T06:17:57.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:17:57.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238934+0000", 2024-09-14T06:17:57.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1311999999999999e-05 2024-09-14T06:17:57.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:17:57.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228134+0000", 2024-09-14T06:17:57.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.98919991200000001 2024-09-14T06:17:57.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:17:57.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228261+0000", 2024-09-14T06:17:57.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000126742 2024-09-14T06:17:57.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:17:57.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:17:57.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:17:57.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228768+0000", 2024-09-14T06:17:57.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050675600000000003 2024-09-14T06:17:57.409 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.409 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.409 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.409 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:17:57.410 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:17:57.410 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:17:57.411 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:17:57.412 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:57.405+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:57.460 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-14T06:17:57.460 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-14T06:17:57.460 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-14T06:17:57.460 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-14T06:17:57.461 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-14T06:17:57.461 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-14T06:17:57.461 INFO:teuthology.orchestra.run.smithi039.stderr:bench [] [] [] [] 2024-09-14T06:17:57.461 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:17:57.461 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-14T06:17:57.461 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-14T06:17:57.461 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-14T06:17:57.461 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2024-09-14T06:17:57.463 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:17:57.464 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:17:57.464 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-14T06:17:57.512 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:57.509+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:57.550 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:57.550 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:57.551 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:17:57.551 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:57.565 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:17:57.613 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-14T06:17:57.613 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-14T06:17:57.613 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-14T06:17:57.613 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-14T06:17:57.613 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-14T06:17:57.613 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-14T06:17:57.613 INFO:teuthology.orchestra.run.smithi039.stderr:bench [] [] [] [] 2024-09-14T06:17:57.613 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:17:57.613 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-14T06:17:57.614 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-14T06:17:57.614 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-14T06:17:57.614 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2024-09-14T06:17:57.614 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:57.609+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:57.616 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:17:57.617 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:17:57.617 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-14T06:17:57.687 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:57.687 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:17:57.687 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:17:57.687 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:17:57.687 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:57.701 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:17:57.730 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:17:57.748 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:57.748+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:57.814 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:57.809+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:57.818 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-14T06:17:57.818 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-14T06:17:57.818 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-14T06:17:57.818 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-14T06:17:57.818 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-14T06:17:57.818 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-14T06:17:57.818 INFO:teuthology.orchestra.run.smithi039.stderr:bench [] [] [] [] 2024-09-14T06:17:57.818 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:17:57.819 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-14T06:17:57.819 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-14T06:17:57.819 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-14T06:17:57.819 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2024-09-14T06:17:57.821 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:17:57.822 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:17:57.822 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-14T06:17:57.823 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:17:57.823 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:17:57.823 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:17:57.824 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:17:57.824 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.824 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13229 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.824 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:57.824 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.720275076999997, 2024-09-14T06:17:57.824 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081798868999999996, 2024-09-14T06:17:57.824 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.824 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.824 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.824 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.824 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.824 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.824 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13229 2024-09-14T06:17:57.825 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.825 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.825 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.825 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.825 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:57.825 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.825 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.825 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.825 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.825 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:17:57.825 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.825 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.825 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.826 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.826 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098710+0000", 2024-09-14T06:17:57.826 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8199999999999999e-07 2024-09-14T06:17:57.826 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.826 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.826 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.826 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098714+0000", 2024-09-14T06:17:57.826 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:17:57.826 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.826 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.826 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.826 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098727+0000", 2024-09-14T06:17:57.826 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2520999999999999e-05 2024-09-14T06:17:57.827 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.827 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.827 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.827 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098749+0000", 2024-09-14T06:17:57.827 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2773999999999999e-05 2024-09-14T06:17:57.827 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.827 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.827 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.827 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160234+0000", 2024-09-14T06:17:57.827 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.061484430999999999 2024-09-14T06:17:57.827 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.827 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.827 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.828 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160281+0000", 2024-09-14T06:17:57.828 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7437e-05 2024-09-14T06:17:57.828 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.828 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.828 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.828 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160569+0000", 2024-09-14T06:17:57.828 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028788700000000002 2024-09-14T06:17:57.828 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.828 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.828 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.828 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163121+0000", 2024-09-14T06:17:57.828 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025514090000000001 2024-09-14T06:17:57.828 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.828 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.829 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.829 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180410+0000", 2024-09-14T06:17:57.829 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.017289098999999999 2024-09-14T06:17:57.829 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.829 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.829 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.829 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180472+0000", 2024-09-14T06:17:57.829 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2547999999999998e-05 2024-09-14T06:17:57.829 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.829 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.829 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.829 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180508+0000", 2024-09-14T06:17:57.829 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5534000000000002e-05 2024-09-14T06:17:57.829 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.830 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.830 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.830 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.830 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.830 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13230 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.830 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:57.830 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.720193875000007, 2024-09-14T06:17:57.830 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081851558000000005, 2024-09-14T06:17:57.830 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.830 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.830 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.830 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.830 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.830 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.830 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13230 2024-09-14T06:17:57.831 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.831 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.831 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.831 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.831 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:57.831 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.831 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.831 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.831 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.831 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:17:57.831 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.831 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.831 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.832 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.832 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098791+0000", 2024-09-14T06:17:57.832 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8800000000000001e-07 2024-09-14T06:17:57.832 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.832 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.832 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.832 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098795+0000", 2024-09-14T06:17:57.832 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.391e-06 2024-09-14T06:17:57.832 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.832 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.832 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.832 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098804+0000", 2024-09-14T06:17:57.832 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3440000000000001e-06 2024-09-14T06:17:57.832 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.833 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.833 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.833 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098810+0000", 2024-09-14T06:17:57.833 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2339999999999999e-06 2024-09-14T06:17:57.833 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.833 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.833 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.833 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160930+0000", 2024-09-14T06:17:57.833 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062120018999999999 2024-09-14T06:17:57.833 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.833 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.833 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.833 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161017+0000", 2024-09-14T06:17:57.834 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6868999999999998e-05 2024-09-14T06:17:57.834 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.834 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.834 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.834 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161300+0000", 2024-09-14T06:17:57.834 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028349499999999998 2024-09-14T06:17:57.834 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.834 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.834 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.834 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164612+0000", 2024-09-14T06:17:57.834 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033119680000000002 2024-09-14T06:17:57.834 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.834 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.834 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.835 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180559+0000", 2024-09-14T06:17:57.835 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015946612999999998 2024-09-14T06:17:57.835 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.835 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.835 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.835 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180605+0000", 2024-09-14T06:17:57.835 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5890000000000003e-05 2024-09-14T06:17:57.835 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.835 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.835 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.835 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180642+0000", 2024-09-14T06:17:57.835 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:17:57.835 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.835 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.835 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.836 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.836 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.836 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13231 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.836 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:57.836 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.719931461000002, 2024-09-14T06:17:57.836 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081750685000000003, 2024-09-14T06:17:57.836 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.836 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.836 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.836 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.836 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.836 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.836 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13231 2024-09-14T06:17:57.836 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.836 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.837 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.837 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.837 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:57.837 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.837 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.837 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.837 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.837 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:17:57.837 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.837 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.837 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.837 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.837 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099053+0000", 2024-09-14T06:17:57.837 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:17:57.838 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.838 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.838 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.838 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099059+0000", 2024-09-14T06:17:57.838 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1789999999999999e-06 2024-09-14T06:17:57.838 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.838 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.838 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.838 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099072+0000", 2024-09-14T06:17:57.838 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3179999999999999e-05 2024-09-14T06:17:57.838 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.838 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.838 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.838 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099080+0000", 2024-09-14T06:17:57.839 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9030000000000004e-06 2024-09-14T06:17:57.839 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.839 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.839 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.839 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161681+0000", 2024-09-14T06:17:57.839 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062601641999999999 2024-09-14T06:17:57.839 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.839 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.839 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.840 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161766+0000", 2024-09-14T06:17:57.840 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4909999999999998e-05 2024-09-14T06:17:57.840 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.840 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.840 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.840 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162109+0000", 2024-09-14T06:17:57.840 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00034264899999999999 2024-09-14T06:17:57.840 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.840 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.840 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.840 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164615+0000", 2024-09-14T06:17:57.840 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025061979999999998 2024-09-14T06:17:57.840 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.840 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.841 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.841 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180718+0000", 2024-09-14T06:17:57.841 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016103240000000001 2024-09-14T06:17:57.841 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.841 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.841 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.841 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180768+0000", 2024-09-14T06:17:57.841 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9781999999999999e-05 2024-09-14T06:17:57.841 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.841 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.841 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.841 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180803+0000", 2024-09-14T06:17:57.841 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5021999999999998e-05 2024-09-14T06:17:57.841 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.841 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.842 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.842 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.842 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.842 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13232 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.842 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:57.842 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.719889687000006, 2024-09-14T06:17:57.842 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081861981, 2024-09-14T06:17:57.842 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.842 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.842 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.842 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.842 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.842 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.842 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13232 2024-09-14T06:17:57.843 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.843 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.843 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.843 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.843 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:57.843 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.843 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.843 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.843 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.843 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:17:57.843 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.843 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.843 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.843 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.843 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099095+0000", 2024-09-14T06:17:57.844 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:17:57.844 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.844 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.844 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.844 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099098+0000", 2024-09-14T06:17:57.844 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5420000000000001e-06 2024-09-14T06:17:57.844 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.844 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.844 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.844 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099106+0000", 2024-09-14T06:17:57.844 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.824e-06 2024-09-14T06:17:57.844 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.844 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.844 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.844 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099112+0000", 2024-09-14T06:17:57.845 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9789999999999998e-06 2024-09-14T06:17:57.845 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.845 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.845 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.845 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162576+0000", 2024-09-14T06:17:57.845 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.063463723999999999 2024-09-14T06:17:57.845 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.845 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.845 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.845 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162627+0000", 2024-09-14T06:17:57.845 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1530000000000003e-05 2024-09-14T06:17:57.845 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.845 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.845 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.846 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162854+0000", 2024-09-14T06:17:57.846 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000226159 2024-09-14T06:17:57.846 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.846 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.846 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.846 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166090+0000", 2024-09-14T06:17:57.846 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032362319999999999 2024-09-14T06:17:57.846 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.846 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.846 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.846 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180877+0000", 2024-09-14T06:17:57.846 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014786726 2024-09-14T06:17:57.846 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.846 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.846 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.847 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180920+0000", 2024-09-14T06:17:57.847 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3427000000000003e-05 2024-09-14T06:17:57.847 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.847 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.847 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.847 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180956+0000", 2024-09-14T06:17:57.847 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6195999999999997e-05 2024-09-14T06:17:57.847 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.847 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.847 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.847 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.847 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.847 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13233 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.847 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:57.848 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.719864423999994, 2024-09-14T06:17:57.848 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081924860000000002, 2024-09-14T06:17:57.848 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.848 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.848 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.848 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.848 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.848 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.848 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13233 2024-09-14T06:17:57.848 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.848 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.848 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.848 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.848 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:57.849 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.849 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.849 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.849 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.849 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:17:57.849 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.849 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.849 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.849 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.849 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099120+0000", 2024-09-14T06:17:57.849 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:17:57.849 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.849 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.850 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.850 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099122+0000", 2024-09-14T06:17:57.850 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:17:57.850 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.850 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.850 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.850 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099126+0000", 2024-09-14T06:17:57.850 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2540000000000003e-06 2024-09-14T06:17:57.850 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.850 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.850 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.850 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099130+0000", 2024-09-14T06:17:57.850 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4060000000000001e-06 2024-09-14T06:17:57.851 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.851 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.851 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.851 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163169+0000", 2024-09-14T06:17:57.851 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064039415000000002 2024-09-14T06:17:57.851 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.851 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.851 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.851 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163260+0000", 2024-09-14T06:17:57.851 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1038000000000005e-05 2024-09-14T06:17:57.851 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.851 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.851 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.852 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163640+0000", 2024-09-14T06:17:57.852 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037954699999999999 2024-09-14T06:17:57.852 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.852 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.852 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.852 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166093+0000", 2024-09-14T06:17:57.852 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024533229999999999 2024-09-14T06:17:57.852 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.852 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.852 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.852 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180993+0000", 2024-09-14T06:17:57.852 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01489997 2024-09-14T06:17:57.852 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.852 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.853 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.853 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181020+0000", 2024-09-14T06:17:57.853 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6806999999999999e-05 2024-09-14T06:17:57.853 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.853 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.853 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.853 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181044+0000", 2024-09-14T06:17:57.853 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4405000000000001e-05 2024-09-14T06:17:57.853 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.853 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.853 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.853 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.853 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13234 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.719822218999994, 2024-09-14T06:17:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082015597999999995, 2024-09-14T06:17:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13234 2024-09-14T06:17:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:17:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0999999999999998e-07 2024-09-14T06:17:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099166+0000", 2024-09-14T06:17:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:17:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099171+0000", 2024-09-14T06:17:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6949999999999997e-06 2024-09-14T06:17:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099175+0000", 2024-09-14T06:17:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9029999999999997e-06 2024-09-14T06:17:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164063+0000", 2024-09-14T06:17:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064887608999999999 2024-09-14T06:17:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164111+0000", 2024-09-14T06:17:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7648999999999997e-05 2024-09-14T06:17:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164351+0000", 2024-09-14T06:17:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024072899999999999 2024-09-14T06:17:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167561+0000", 2024-09-14T06:17:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032094179999999999 2024-09-14T06:17:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181093+0000", 2024-09-14T06:17:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013532245 2024-09-14T06:17:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181139+0000", 2024-09-14T06:17:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6483999999999997e-05 2024-09-14T06:17:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181177+0000", 2024-09-14T06:17:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7861000000000002e-05 2024-09-14T06:17:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13235 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.719714945000007, 2024-09-14T06:17:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081988088000000001, 2024-09-14T06:17:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13235 2024-09-14T06:17:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:17:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099270+0000", 2024-09-14T06:17:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7400000000000003e-07 2024-09-14T06:17:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099276+0000", 2024-09-14T06:17:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:17:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099284+0000", 2024-09-14T06:17:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7979999999999995e-06 2024-09-14T06:17:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099289+0000", 2024-09-14T06:17:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:17:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164664+0000", 2024-09-14T06:17:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065374901999999999 2024-09-14T06:17:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164755+0000", 2024-09-14T06:17:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0790000000000003e-05 2024-09-14T06:17:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165143+0000", 2024-09-14T06:17:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000387904 2024-09-14T06:17:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167564+0000", 2024-09-14T06:17:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024208989999999998 2024-09-14T06:17:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181212+0000", 2024-09-14T06:17:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013648664 2024-09-14T06:17:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181237+0000", 2024-09-14T06:17:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5177999999999998e-05 2024-09-14T06:17:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181257+0000", 2024-09-14T06:17:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9660000000000002e-05 2024-09-14T06:17:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13236 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.719608125999997, 2024-09-14T06:17:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082010939000000005, 2024-09-14T06:17:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13236 2024-09-14T06:17:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:17:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.61e-07 2024-09-14T06:17:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099382+0000", 2024-09-14T06:17:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3589999999999999e-06 2024-09-14T06:17:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099388+0000", 2024-09-14T06:17:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9619999999999998e-06 2024-09-14T06:17:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099393+0000", 2024-09-14T06:17:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3410000000000002e-06 2024-09-14T06:17:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165553+0000", 2024-09-14T06:17:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066159604999999996 2024-09-14T06:17:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165603+0000", 2024-09-14T06:17:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0427999999999999e-05 2024-09-14T06:17:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165827+0000", 2024-09-14T06:17:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022358299999999999 2024-09-14T06:17:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177005+0000", 2024-09-14T06:17:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011178044 2024-09-14T06:17:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181302+0000", 2024-09-14T06:17:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042970669999999999 2024-09-14T06:17:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181349+0000", 2024-09-14T06:17:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7287999999999999e-05 2024-09-14T06:17:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181387+0000", 2024-09-14T06:17:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7601000000000001e-05 2024-09-14T06:17:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13237 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.719449979000004, 2024-09-14T06:17:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081927299999999995, 2024-09-14T06:17:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13237 2024-09-14T06:17:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:17:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099535+0000", 2024-09-14T06:17:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:17:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099538+0000", 2024-09-14T06:17:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:17:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099545+0000", 2024-09-14T06:17:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:17:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099551+0000", 2024-09-14T06:17:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1770000000000003e-06 2024-09-14T06:17:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166131+0000", 2024-09-14T06:17:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066580499000000001 2024-09-14T06:17:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166221+0000", 2024-09-14T06:17:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0000000000000006e-05 2024-09-14T06:17:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166622+0000", 2024-09-14T06:17:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040029799999999999 2024-09-14T06:17:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177008+0000", 2024-09-14T06:17:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010386772000000001 2024-09-14T06:17:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181420+0000", 2024-09-14T06:17:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044119190000000003 2024-09-14T06:17:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181443+0000", 2024-09-14T06:17:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2603999999999999e-05 2024-09-14T06:17:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181461+0000", 2024-09-14T06:17:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8405000000000001e-05 2024-09-14T06:17:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13238 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.719413670999998, 2024-09-14T06:17:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082014927000000001, 2024-09-14T06:17:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13238 2024-09-14T06:17:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:17:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099571+0000", 2024-09-14T06:17:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9699999999999996e-07 2024-09-14T06:17:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099577+0000", 2024-09-14T06:17:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2890000000000003e-06 2024-09-14T06:17:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099587+0000", 2024-09-14T06:17:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0002e-05 2024-09-14T06:17:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099593+0000", 2024-09-14T06:17:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4539999999999998e-06 2024-09-14T06:17:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167024+0000", 2024-09-14T06:17:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067431480000000002 2024-09-14T06:17:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167074+0000", 2024-09-14T06:17:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9996e-05 2024-09-14T06:17:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167288+0000", 2024-09-14T06:17:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021410900000000001 2024-09-14T06:17:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177997+0000", 2024-09-14T06:17:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010708756 2024-09-14T06:17:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181508+0000", 2024-09-14T06:17:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003511363 2024-09-14T06:17:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181552+0000", 2024-09-14T06:17:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3609999999999998e-05 2024-09-14T06:17:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181585+0000", 2024-09-14T06:17:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3271e-05 2024-09-14T06:17:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13239 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.719321852999997, 2024-09-14T06:17:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081998772999999997, 2024-09-14T06:17:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13239 2024-09-14T06:17:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:17:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099663+0000", 2024-09-14T06:17:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6100000000000001e-07 2024-09-14T06:17:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099666+0000", 2024-09-14T06:17:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.056e-06 2024-09-14T06:17:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099672+0000", 2024-09-14T06:17:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8120000000000002e-06 2024-09-14T06:17:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099676+0000", 2024-09-14T06:17:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:17:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167602+0000", 2024-09-14T06:17:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067925867000000001 2024-09-14T06:17:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167693+0000", 2024-09-14T06:17:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0736000000000007e-05 2024-09-14T06:17:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168065+0000", 2024-09-14T06:17:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037245399999999998 2024-09-14T06:17:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178003+0000", 2024-09-14T06:17:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0099372360000000003 2024-09-14T06:17:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181621+0000", 2024-09-14T06:17:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036178489999999998 2024-09-14T06:17:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181643+0000", 2024-09-14T06:17:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2155e-05 2024-09-14T06:17:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181661+0000", 2024-09-14T06:17:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8147000000000001e-05 2024-09-14T06:17:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13240 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.719202760000002, 2024-09-14T06:17:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081942607000000001, 2024-09-14T06:17:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13240 2024-09-14T06:17:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:17:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099782+0000", 2024-09-14T06:17:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3900000000000004e-07 2024-09-14T06:17:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099787+0000", 2024-09-14T06:17:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6189999999999999e-06 2024-09-14T06:17:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099793+0000", 2024-09-14T06:17:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.998e-06 2024-09-14T06:17:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099797+0000", 2024-09-14T06:17:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8449999999999996e-06 2024-09-14T06:17:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.893 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.893 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168494+0000", 2024-09-14T06:17:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.068696500999999993 2024-09-14T06:17:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168590+0000", 2024-09-14T06:17:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6733999999999999e-05 2024-09-14T06:17:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168973+0000", 2024-09-14T06:17:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038229399999999998 2024-09-14T06:17:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178013+0000", 2024-09-14T06:17:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0090397199999999994 2024-09-14T06:17:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181683+0000", 2024-09-14T06:17:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036705819999999999 2024-09-14T06:17:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181705+0000", 2024-09-14T06:17:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1551000000000001e-05 2024-09-14T06:17:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181724+0000", 2024-09-14T06:17:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9123999999999999e-05 2024-09-14T06:17:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13243 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.718669030000001, 2024-09-14T06:17:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081767815999999993, 2024-09-14T06:17:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13243 2024-09-14T06:17:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:17:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1e-07 2024-09-14T06:17:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100317+0000", 2024-09-14T06:17:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.092e-06 2024-09-14T06:17:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100321+0000", 2024-09-14T06:17:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7809999999999999e-06 2024-09-14T06:17:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100324+0000", 2024-09-14T06:17:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.117e-06 2024-09-14T06:17:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171113+0000", 2024-09-14T06:17:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.070788816000000004 2024-09-14T06:17:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171202+0000", 2024-09-14T06:17:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8843000000000003e-05 2024-09-14T06:17:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171584+0000", 2024-09-14T06:17:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038231099999999998 2024-09-14T06:17:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178101+0000", 2024-09-14T06:17:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0065168329999999997 2024-09-14T06:17:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182041+0000", 2024-09-14T06:17:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039401810000000001 2024-09-14T06:17:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182065+0000", 2024-09-14T06:17:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3623000000000001e-05 2024-09-14T06:17:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182083+0000", 2024-09-14T06:17:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7909000000000001e-05 2024-09-14T06:17:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13244 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.718654209999997, 2024-09-14T06:17:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081837800000000002, 2024-09-14T06:17:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13244 2024-09-14T06:17:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:17:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:17:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100337+0000", 2024-09-14T06:17:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.939e-06 2024-09-14T06:17:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100341+0000", 2024-09-14T06:17:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2320000000000003e-06 2024-09-14T06:17:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100344+0000", 2024-09-14T06:17:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0759999999999999e-06 2024-09-14T06:17:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171996+0000", 2024-09-14T06:17:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.071651919999999994 2024-09-14T06:17:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172079+0000", 2024-09-14T06:17:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2579999999999999e-05 2024-09-14T06:17:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172457+0000", 2024-09-14T06:17:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037785100000000001 2024-09-14T06:17:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178104+0000", 2024-09-14T06:17:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056471730000000001 2024-09-14T06:17:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182123+0000", 2024-09-14T06:17:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040190019999999998 2024-09-14T06:17:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182147+0000", 2024-09-14T06:17:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4241000000000001e-05 2024-09-14T06:17:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182168+0000", 2024-09-14T06:17:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0492999999999998e-05 2024-09-14T06:17:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13245 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.718634703000006, 2024-09-14T06:17:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081893899000000006, 2024-09-14T06:17:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13245 2024-09-14T06:17:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:17:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100350+0000", 2024-09-14T06:17:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5499999999999999e-07 2024-09-14T06:17:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100353+0000", 2024-09-14T06:17:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:17:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100357+0000", 2024-09-14T06:17:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8500000000000004e-06 2024-09-14T06:17:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100360+0000", 2024-09-14T06:17:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8700000000000001e-06 2024-09-14T06:17:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172862+0000", 2024-09-14T06:17:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.072501992000000001 2024-09-14T06:17:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172950+0000", 2024-09-14T06:17:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8988999999999998e-05 2024-09-14T06:17:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173324+0000", 2024-09-14T06:17:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037366199999999997 2024-09-14T06:17:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178107+0000", 2024-09-14T06:17:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047830520000000003 2024-09-14T06:17:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182202+0000", 2024-09-14T06:17:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040949280000000003 2024-09-14T06:17:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182226+0000", 2024-09-14T06:17:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3535000000000001e-05 2024-09-14T06:17:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182243+0000", 2024-09-14T06:17:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7445e-05 2024-09-14T06:17:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13246 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.718620380999994, 2024-09-14T06:17:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081962276000000001, 2024-09-14T06:17:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13246 2024-09-14T06:17:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:17:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0800000000000001e-07 2024-09-14T06:17:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100367+0000", 2024-09-14T06:17:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8439999999999998e-06 2024-09-14T06:17:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100372+0000", 2024-09-14T06:17:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4009999999999996e-06 2024-09-14T06:17:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100378+0000", 2024-09-14T06:17:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7180000000000001e-06 2024-09-14T06:17:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173780+0000", 2024-09-14T06:17:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.073401991 2024-09-14T06:17:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173879+0000", 2024-09-14T06:17:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9449000000000003e-05 2024-09-14T06:17:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174240+0000", 2024-09-14T06:17:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036118100000000002 2024-09-14T06:17:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178109+0000", 2024-09-14T06:17:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038690619999999999 2024-09-14T06:17:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182281+0000", 2024-09-14T06:17:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041715900000000002 2024-09-14T06:17:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182306+0000", 2024-09-14T06:17:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4477e-05 2024-09-14T06:17:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182326+0000", 2024-09-14T06:17:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0254999999999998e-05 2024-09-14T06:17:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13247 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.718523203999993, 2024-09-14T06:17:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966593000000004, 2024-09-14T06:17:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13247 2024-09-14T06:17:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:17:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100462+0000", 2024-09-14T06:17:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9800000000000002e-07 2024-09-14T06:17:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100466+0000", 2024-09-14T06:17:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4039999999999997e-06 2024-09-14T06:17:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100475+0000", 2024-09-14T06:17:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3680000000000008e-06 2024-09-14T06:17:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100482+0000", 2024-09-14T06:17:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8630000000000002e-06 2024-09-14T06:17:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174669+0000", 2024-09-14T06:17:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074186849999999999 2024-09-14T06:17:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174758+0000", 2024-09-14T06:17:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9021e-05 2024-09-14T06:17:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175137+0000", 2024-09-14T06:17:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378511 2024-09-14T06:17:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178158+0000", 2024-09-14T06:17:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003020956 2024-09-14T06:17:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182386+0000", 2024-09-14T06:17:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042278840000000003 2024-09-14T06:17:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182410+0000", 2024-09-14T06:17:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4026e-05 2024-09-14T06:17:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182427+0000", 2024-09-14T06:17:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7711999999999999e-05 2024-09-14T06:17:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13248 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.718441123000005, 2024-09-14T06:17:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966527999999997, 2024-09-14T06:17:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13248 2024-09-14T06:17:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:17:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:17:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100548+0000", 2024-09-14T06:17:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:17:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100557+0000", 2024-09-14T06:17:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:17:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100563+0000", 2024-09-14T06:17:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2519999999999996e-06 2024-09-14T06:17:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175572+0000", 2024-09-14T06:17:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075008459 2024-09-14T06:17:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175656+0000", 2024-09-14T06:17:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3906999999999995e-05 2024-09-14T06:17:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176010+0000", 2024-09-14T06:17:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00035447499999999998 2024-09-14T06:17:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178304+0000", 2024-09-14T06:17:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0022941379999999998 2024-09-14T06:17:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182467+0000", 2024-09-14T06:17:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041632020000000004 2024-09-14T06:17:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182491+0000", 2024-09-14T06:17:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3733000000000001e-05 2024-09-14T06:17:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182509+0000", 2024-09-14T06:17:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8113000000000001e-05 2024-09-14T06:17:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13249 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.718406189000007, 2024-09-14T06:17:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082037225000000005, 2024-09-14T06:17:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13249 2024-09-14T06:17:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:17:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4600000000000005e-07 2024-09-14T06:17:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100584+0000", 2024-09-14T06:17:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.947e-06 2024-09-14T06:17:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100593+0000", 2024-09-14T06:17:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9760000000000001e-06 2024-09-14T06:17:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100597+0000", 2024-09-14T06:17:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9689999999999996e-06 2024-09-14T06:17:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176461+0000", 2024-09-14T06:17:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075864132000000001 2024-09-14T06:17:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176516+0000", 2024-09-14T06:17:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4416999999999998e-05 2024-09-14T06:17:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176734+0000", 2024-09-14T06:17:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021859399999999999 2024-09-14T06:17:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178308+0000", 2024-09-14T06:17:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015733660000000001 2024-09-14T06:17:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182573+0000", 2024-09-14T06:17:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042649200000000002 2024-09-14T06:17:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182597+0000", 2024-09-14T06:17:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4355e-05 2024-09-14T06:17:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182615+0000", 2024-09-14T06:17:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8003e-05 2024-09-14T06:17:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13250 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:17:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 81.718241583999998, 2024-09-14T06:17:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081983890000000004, 2024-09-14T06:17:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:17:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:17:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:17:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:17:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:17:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:17:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13250 2024-09-14T06:17:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:17:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:17:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:17:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:17:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:17:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:17:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100743+0000", 2024-09-14T06:17:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:17:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:17:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100753+0000", 2024-09-14T06:17:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9979999999999998e-06 2024-09-14T06:17:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:17:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100772+0000", 2024-09-14T06:17:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8664e-05 2024-09-14T06:17:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:17:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100779+0000", 2024-09-14T06:17:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.046e-06 2024-09-14T06:17:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:17:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177049+0000", 2024-09-14T06:17:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076270819000000004 2024-09-14T06:17:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:17:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177140+0000", 2024-09-14T06:17:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0746000000000002e-05 2024-09-14T06:17:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:17:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177519+0000", 2024-09-14T06:17:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378656 2024-09-14T06:17:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:17:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178349+0000", 2024-09-14T06:17:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00083061599999999995 2024-09-14T06:17:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:17:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182679+0000", 2024-09-14T06:17:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043298379999999999 2024-09-14T06:17:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:17:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182705+0000", 2024-09-14T06:17:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5602999999999999e-05 2024-09-14T06:17:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:17:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:17:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:17:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182726+0000", 2024-09-14T06:17:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1297e-05 2024-09-14T06:17:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:17:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:17:57.945 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:17:57.946 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:17:57.947 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:17:57.947 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:17:57.948 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:17:57.948 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:17:57.948 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:17:57.949 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:17:57.949 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:17:57.949 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:17:57.949 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:57.941+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:57.950 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:17:57.950 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:17:57.967 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:17:57.968 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:17:57.988 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:17:57.989 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:17:58.010 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:17:58.010 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:17:58.071 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:58.069+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:58.095 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-14T06:17:58.095 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-14T06:17:58.095 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-14T06:17:58.095 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-14T06:17:58.095 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-14T06:17:58.096 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-14T06:17:58.096 INFO:teuthology.orchestra.run.smithi039.stderr:bench [] [] [] [] 2024-09-14T06:17:58.096 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:17:58.096 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-14T06:17:58.096 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-14T06:17:58.096 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-14T06:17:58.096 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2024-09-14T06:17:58.099 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:17:58.099 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:17:58.123 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:58.124 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:58.139 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:17:58.149 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:58.145+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:58.249 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:58.248+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:58.259 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:58.259 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:58.273 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:17:58.294 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:17:58.359 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:17:58.359+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:17:58.449 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:58.450 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:58.451 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:58.447+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:58.466 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:17:58.552 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:58.549+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:58.632 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:58.632 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:58.646 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:17:58.692 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:17:58.691+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:17:58.754 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:58.749+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:58.769 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:58.769 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:58.784 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:17:58.855 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:58.849+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:58.945 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:58.945 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:58.957 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:58.953+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:58.959 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:17:59.057 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:59.053+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:59.076 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:59.077 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:59.091 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:17:59.159 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:17:59.153+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:17:59.184 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-09-14T06:17:59.214 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:59.214 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:59.230 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:17:59.263 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:59.261+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:59.360 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:59.360 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:59.361 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:17:59.357+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:17:59.374 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:17:59.462 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:59.457+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:59.509 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:59.509 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:59.523 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:17:59.568 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:59.565+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:17:59.663 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:17:59.661+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:17:59.693 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:59.693 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:59.707 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:17:59.765 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:17:59.763+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:17:59.865 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:17:59.863+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:17:59.878 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:17:59.878 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:17:59.892 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:17:59.966 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:17:59.961+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:00.012 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:00.012 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:00.027 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:18:00.028 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:18:00.067 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:00.063+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:00.169 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:00.165+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:00.191 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:00.191 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:00.207 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:18:00.212 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:18:00.212 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:18:00.230 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:18:00.230 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:18:00.247 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:18:00.247 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:18:00.263 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:18:00.264 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:18:00.269 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:00.267+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:00.284 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:18:00.284 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:18:00.305 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:18:00.305 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:18:00.331 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:18:00.331 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:18:00.337 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:00.337 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:18:00.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:18:00.337 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:18:00.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.337 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4170.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:18:00.337 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:18:00.337 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 116.257616243, 2024-09-14T06:18:00.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.56009052699999995, 2024-09-14T06:18:00.337 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.337 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.337 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:00.338 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.338 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4170", 2024-09-14T06:18:00.338 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1567029427", 2024-09-14T06:18:00.338 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 1 2024-09-14T06:18:00.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.338 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:18:00.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077062+0000", 2024-09-14T06:18:00.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.339 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077065+0000", 2024-09-14T06:18:00.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6429999999999999e-06 2024-09-14T06:18:00.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.339 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077079+0000", 2024-09-14T06:18:00.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4447000000000001e-05 2024-09-14T06:18:00.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077103+0000", 2024-09-14T06:18:00.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3856e-05 2024-09-14T06:18:00.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077120+0000", 2024-09-14T06:18:00.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7337e-05 2024-09-14T06:18:00.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077255+0000", 2024-09-14T06:18:00.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000134652 2024-09-14T06:18:00.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:18:00.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.077274+0000", 2024-09-14T06:18:00.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9003000000000001e-05 2024-09-14T06:18:00.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634272+0000", 2024-09-14T06:18:00.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.55699813099999995 2024-09-14T06:18:00.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634591+0000", 2024-09-14T06:18:00.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031910800000000002 2024-09-14T06:18:00.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 5", 2024-09-14T06:18:00.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.634892+0000", 2024-09-14T06:18:00.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00030066400000000002 2024-09-14T06:18:00.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:00.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636345+0000", 2024-09-14T06:18:00.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.001452896 2024-09-14T06:18:00.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:00.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.636974+0000", 2024-09-14T06:18:00.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00062927800000000004 2024-09-14T06:18:00.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:00.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637071+0000", 2024-09-14T06:18:00.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7178999999999999e-05 2024-09-14T06:18:00.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.344 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.344 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:04.637153+0000", 2024-09-14T06:18:00.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1333000000000002e-05 2024-09-14T06:18:00.344 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.344 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.344 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.344 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38947 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 159744~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.344 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:18:00.344 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.654557955999998, 2024-09-14T06:18:00.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085496275999999996, 2024-09-14T06:18:00.345 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.345 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.345 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.345 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.345 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.345 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.345 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38947 2024-09-14T06:18:00.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.345 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.345 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:18:00.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680120+0000", 2024-09-14T06:18:00.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680121+0000", 2024-09-14T06:18:00.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.82e-07 2024-09-14T06:18:00.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680123+0000", 2024-09-14T06:18:00.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5629999999999999e-06 2024-09-14T06:18:00.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680133+0000", 2024-09-14T06:18:00.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0890000000000006e-06 2024-09-14T06:18:00.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680138+0000", 2024-09-14T06:18:00.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.728e-06 2024-09-14T06:18:00.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765422+0000", 2024-09-14T06:18:00.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085283701000000003 2024-09-14T06:18:00.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.351 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.351 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765496+0000", 2024-09-14T06:18:00.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4276000000000005e-05 2024-09-14T06:18:00.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.351 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.351 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765617+0000", 2024-09-14T06:18:00.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000120437 2024-09-14T06:18:00.351 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.351 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.351 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.352 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38990 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 163840~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.352 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:18:00.352 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.654531972999997, 2024-09-14T06:18:00.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085695970999999996, 2024-09-14T06:18:00.352 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.352 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.352 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.352 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.352 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.352 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.352 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38990 2024-09-14T06:18:00.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.353 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:18:00.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680146+0000", 2024-09-14T06:18:00.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680147+0000", 2024-09-14T06:18:00.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7599999999999997e-07 2024-09-14T06:18:00.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680149+0000", 2024-09-14T06:18:00.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4700000000000001e-06 2024-09-14T06:18:00.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680156+0000", 2024-09-14T06:18:00.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.058e-06 2024-09-14T06:18:00.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680162+0000", 2024-09-14T06:18:00.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7080000000000002e-06 2024-09-14T06:18:00.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765657+0000", 2024-09-14T06:18:00.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085494566999999994 2024-09-14T06:18:00.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.356 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765741+0000", 2024-09-14T06:18:00.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4782000000000002e-05 2024-09-14T06:18:00.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.356 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765842+0000", 2024-09-14T06:18:00.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010090999999999999 2024-09-14T06:18:00.356 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.356 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.356 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.357 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38991 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 172032~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.357 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:18:00.357 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.654508016000001, 2024-09-14T06:18:00.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085875878000000003, 2024-09-14T06:18:00.357 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.357 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.357 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.357 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.357 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.357 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.357 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38991 2024-09-14T06:18:00.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.358 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:18:00.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680170+0000", 2024-09-14T06:18:00.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680171+0000", 2024-09-14T06:18:00.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4299999999999998e-07 2024-09-14T06:18:00.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680173+0000", 2024-09-14T06:18:00.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1639999999999999e-06 2024-09-14T06:18:00.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680181+0000", 2024-09-14T06:18:00.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0539999999999995e-06 2024-09-14T06:18:00.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680187+0000", 2024-09-14T06:18:00.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7529999999999998e-06 2024-09-14T06:18:00.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765875+0000", 2024-09-14T06:18:00.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085688722999999994 2024-09-14T06:18:00.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.765948+0000", 2024-09-14T06:18:00.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2126999999999996e-05 2024-09-14T06:18:00.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766046+0000", 2024-09-14T06:18:00.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8614000000000001e-05 2024-09-14T06:18:00.364 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.364 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.364 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.364 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38992 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 180224~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.364 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:18:00.364 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.654482739999999, 2024-09-14T06:18:00.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086050225999999994, 2024-09-14T06:18:00.364 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.364 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.364 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.364 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.365 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.365 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.365 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38992 2024-09-14T06:18:00.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.365 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:18:00.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:18:00.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680196+0000", 2024-09-14T06:18:00.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1199999999999998e-07 2024-09-14T06:18:00.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680199+0000", 2024-09-14T06:18:00.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:18:00.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680227+0000", 2024-09-14T06:18:00.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8238999999999998e-05 2024-09-14T06:18:00.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680234+0000", 2024-09-14T06:18:00.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0550000000000002e-06 2024-09-14T06:18:00.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766077+0000", 2024-09-14T06:18:00.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085843002000000002 2024-09-14T06:18:00.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766155+0000", 2024-09-14T06:18:00.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8112e-05 2024-09-14T06:18:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766246+0000", 2024-09-14T06:18:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0785999999999994e-05 2024-09-14T06:18:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38993 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 188416~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:18:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.654391388999997, 2024-09-14T06:18:00.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086166131000000007, 2024-09-14T06:18:00.369 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.369 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.369 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.369 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.369 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.369 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.369 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38993 2024-09-14T06:18:00.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.369 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:18:00.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.370 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.370 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680287+0000", 2024-09-14T06:18:00.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.370 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.370 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680288+0000", 2024-09-14T06:18:00.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7199999999999998e-07 2024-09-14T06:18:00.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.370 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.370 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680291+0000", 2024-09-14T06:18:00.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7280000000000001e-06 2024-09-14T06:18:00.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.371 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680301+0000", 2024-09-14T06:18:00.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3360000000000001e-06 2024-09-14T06:18:00.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.371 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680308+0000", 2024-09-14T06:18:00.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1389999999999998e-06 2024-09-14T06:18:00.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766280+0000", 2024-09-14T06:18:00.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.085972111000000004 2024-09-14T06:18:00.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766357+0000", 2024-09-14T06:18:00.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7477000000000004e-05 2024-09-14T06:18:00.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766453+0000", 2024-09-14T06:18:00.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5668000000000006e-05 2024-09-14T06:18:00.372 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.372 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.373 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.373 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38994 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 196608~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.373 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:18:00.373 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.654301582000002, 2024-09-14T06:18:00.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086290477000000004, 2024-09-14T06:18:00.373 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.373 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.373 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.373 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.373 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.373 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.374 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38994 2024-09-14T06:18:00.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.374 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:18:00.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:18:00.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680377+0000", 2024-09-14T06:18:00.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3e-07 2024-09-14T06:18:00.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680381+0000", 2024-09-14T06:18:00.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.439e-06 2024-09-14T06:18:00.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680438+0000", 2024-09-14T06:18:00.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6693000000000001e-05 2024-09-14T06:18:00.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680446+0000", 2024-09-14T06:18:00.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9160000000000001e-06 2024-09-14T06:18:00.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766490+0000", 2024-09-14T06:18:00.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086044617000000004 2024-09-14T06:18:00.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.376 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766563+0000", 2024-09-14T06:18:00.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2430999999999998e-05 2024-09-14T06:18:00.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766667+0000", 2024-09-14T06:18:00.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000104651 2024-09-14T06:18:00.377 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.377 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.377 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.377 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38995 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 204800~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.377 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:18:00.377 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.654218556000004, 2024-09-14T06:18:00.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086415360999999996, 2024-09-14T06:18:00.378 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.378 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.378 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.378 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.378 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.378 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.378 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38995 2024-09-14T06:18:00.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.378 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.378 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.378 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:18:00.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680460+0000", 2024-09-14T06:18:00.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680461+0000", 2024-09-14T06:18:00.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:18:00.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680464+0000", 2024-09-14T06:18:00.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4929999999999998e-06 2024-09-14T06:18:00.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680479+0000", 2024-09-14T06:18:00.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5322999999999999e-05 2024-09-14T06:18:00.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680496+0000", 2024-09-14T06:18:00.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6801000000000001e-05 2024-09-14T06:18:00.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766699+0000", 2024-09-14T06:18:00.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086203265000000001 2024-09-14T06:18:00.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766777+0000", 2024-09-14T06:18:00.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7284000000000001e-05 2024-09-14T06:18:00.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766875+0000", 2024-09-14T06:18:00.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8461000000000005e-05 2024-09-14T06:18:00.381 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.382 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.382 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.382 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38996 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 212992~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.382 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:18:00.382 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.654171116999997, 2024-09-14T06:18:00.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086576474, 2024-09-14T06:18:00.382 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.382 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.382 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.382 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.382 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.383 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.383 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38996 2024-09-14T06:18:00.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.383 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:18:00.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680507+0000", 2024-09-14T06:18:00.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680508+0000", 2024-09-14T06:18:00.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1799999999999995e-07 2024-09-14T06:18:00.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680511+0000", 2024-09-14T06:18:00.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9739999999999998e-06 2024-09-14T06:18:00.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680520+0000", 2024-09-14T06:18:00.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.329e-06 2024-09-14T06:18:00.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680528+0000", 2024-09-14T06:18:00.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4660000000000002e-06 2024-09-14T06:18:00.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766907+0000", 2024-09-14T06:18:00.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086379529999999996 2024-09-14T06:18:00.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.386 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.766987+0000", 2024-09-14T06:18:00.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0280999999999999e-05 2024-09-14T06:18:00.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.386 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.386 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767084+0000", 2024-09-14T06:18:00.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6275999999999998e-05 2024-09-14T06:18:00.386 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.386 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.386 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.387 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38997 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 221184~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.387 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:18:00.387 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.654141776000003, 2024-09-14T06:18:00.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086757327999999995, 2024-09-14T06:18:00.387 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.387 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.387 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.387 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.387 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.387 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.387 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38997 2024-09-14T06:18:00.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.387 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:18:00.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.388 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:18:00.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680537+0000", 2024-09-14T06:18:00.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:18:00.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680540+0000", 2024-09-14T06:18:00.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6910000000000002e-06 2024-09-14T06:18:00.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680548+0000", 2024-09-14T06:18:00.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7339999999999997e-06 2024-09-14T06:18:00.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680555+0000", 2024-09-14T06:18:00.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0900000000000001e-06 2024-09-14T06:18:00.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767116+0000", 2024-09-14T06:18:00.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086561551 2024-09-14T06:18:00.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767192+0000", 2024-09-14T06:18:00.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5504000000000001e-05 2024-09-14T06:18:00.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.391 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.391 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767294+0000", 2024-09-14T06:18:00.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000102308 2024-09-14T06:18:00.391 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.391 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.391 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.391 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38998 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 229376~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.391 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:18:00.391 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.654115165, 2024-09-14T06:18:00.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087010193999999999, 2024-09-14T06:18:00.392 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.392 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.392 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.392 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.392 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.392 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.392 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38998 2024-09-14T06:18:00.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.392 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:18:00.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680563+0000", 2024-09-14T06:18:00.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680564+0000", 2024-09-14T06:18:00.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1800000000000001e-07 2024-09-14T06:18:00.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680566+0000", 2024-09-14T06:18:00.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.503e-06 2024-09-14T06:18:00.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680574+0000", 2024-09-14T06:18:00.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3979999999999999e-06 2024-09-14T06:18:00.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.394 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.394 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680580+0000", 2024-09-14T06:18:00.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4859999999999997e-06 2024-09-14T06:18:00.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767359+0000", 2024-09-14T06:18:00.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086779447999999995 2024-09-14T06:18:00.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767445+0000", 2024-09-14T06:18:00.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6007000000000005e-05 2024-09-14T06:18:00.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767573+0000", 2024-09-14T06:18:00.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012793400000000001 2024-09-14T06:18:00.396 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.396 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.396 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.396 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:38999 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 237568~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.396 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:18:00.396 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.654086301, 2024-09-14T06:18:00.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087211699000000004, 2024-09-14T06:18:00.396 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.396 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.396 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.396 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.397 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.397 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.397 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 38999 2024-09-14T06:18:00.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.397 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:18:00.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680592+0000", 2024-09-14T06:18:00.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680593+0000", 2024-09-14T06:18:00.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7700000000000005e-07 2024-09-14T06:18:00.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680600+0000", 2024-09-14T06:18:00.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7130000000000005e-06 2024-09-14T06:18:00.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680610+0000", 2024-09-14T06:18:00.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.55e-06 2024-09-14T06:18:00.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680616+0000", 2024-09-14T06:18:00.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3069999999999999e-06 2024-09-14T06:18:00.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767607+0000", 2024-09-14T06:18:00.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.086990703000000003 2024-09-14T06:18:00.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.400 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.400 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767682+0000", 2024-09-14T06:18:00.400 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4944999999999995e-05 2024-09-14T06:18:00.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.400 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.400 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767804+0000", 2024-09-14T06:18:00.400 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012200400000000001 2024-09-14T06:18:00.400 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.400 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.401 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.401 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39000 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 245760~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.401 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:18:00.401 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.654052679000003, 2024-09-14T06:18:00.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087408649000000005, 2024-09-14T06:18:00.401 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.401 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.401 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.401 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.401 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.401 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.402 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39000 2024-09-14T06:18:00.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.402 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:18:00.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:18:00.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680626+0000", 2024-09-14T06:18:00.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1699999999999998e-07 2024-09-14T06:18:00.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680629+0000", 2024-09-14T06:18:00.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5469999999999999e-06 2024-09-14T06:18:00.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680636+0000", 2024-09-14T06:18:00.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.973e-06 2024-09-14T06:18:00.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680642+0000", 2024-09-14T06:18:00.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0349999999999998e-06 2024-09-14T06:18:00.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767839+0000", 2024-09-14T06:18:00.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087197067000000003 2024-09-14T06:18:00.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.767913+0000", 2024-09-14T06:18:00.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3832999999999996e-05 2024-09-14T06:18:00.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768034+0000", 2024-09-14T06:18:00.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012167699999999999 2024-09-14T06:18:00.405 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.405 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.406 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.406 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39001 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 253952~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.406 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:18:00.406 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.654028236000002, 2024-09-14T06:18:00.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087623312999999994, 2024-09-14T06:18:00.406 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.406 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.406 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.406 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.406 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.406 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.407 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39001 2024-09-14T06:18:00.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.407 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:18:00.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680650+0000", 2024-09-14T06:18:00.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680651+0000", 2024-09-14T06:18:00.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7700000000000004e-07 2024-09-14T06:18:00.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680653+0000", 2024-09-14T06:18:00.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5739999999999998e-06 2024-09-14T06:18:00.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680702+0000", 2024-09-14T06:18:00.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8476000000000001e-05 2024-09-14T06:18:00.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680709+0000", 2024-09-14T06:18:00.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8170000000000001e-06 2024-09-14T06:18:00.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768066+0000", 2024-09-14T06:18:00.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087357852999999999 2024-09-14T06:18:00.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768163+0000", 2024-09-14T06:18:00.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6869999999999999e-05 2024-09-14T06:18:00.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768273+0000", 2024-09-14T06:18:00.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000110146 2024-09-14T06:18:00.410 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.410 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.410 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.411 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39002 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 262144~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.411 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:18:00.411 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.653948130000003, 2024-09-14T06:18:00.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088150478000000004, 2024-09-14T06:18:00.411 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.411 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.411 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.411 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.411 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.411 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.411 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39002 2024-09-14T06:18:00.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.412 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:18:00.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680730+0000", 2024-09-14T06:18:00.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.413 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680739+0000", 2024-09-14T06:18:00.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3979999999999999e-06 2024-09-14T06:18:00.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.413 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.413 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680742+0000", 2024-09-14T06:18:00.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8349999999999997e-06 2024-09-14T06:18:00.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.413 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.413 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680751+0000", 2024-09-14T06:18:00.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8599999999999999e-06 2024-09-14T06:18:00.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680758+0000", 2024-09-14T06:18:00.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4790000000000001e-06 2024-09-14T06:18:00.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768307+0000", 2024-09-14T06:18:00.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.087549342000000002 2024-09-14T06:18:00.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768392+0000", 2024-09-14T06:18:00.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.441e-05 2024-09-14T06:18:00.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768881+0000", 2024-09-14T06:18:00.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000489154 2024-09-14T06:18:00.415 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.415 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.415 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.415 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39003 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 270336~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.415 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:18:00.416 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.653895839999997, 2024-09-14T06:18:00.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088375555999999994, 2024-09-14T06:18:00.416 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.416 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.416 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.416 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.416 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.416 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.416 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39003 2024-09-14T06:18:00.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.416 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:18:00.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:18:00.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680783+0000", 2024-09-14T06:18:00.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7000000000000004e-07 2024-09-14T06:18:00.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680786+0000", 2024-09-14T06:18:00.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:18:00.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680794+0000", 2024-09-14T06:18:00.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9840000000000001e-06 2024-09-14T06:18:00.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680800+0000", 2024-09-14T06:18:00.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1550000000000004e-06 2024-09-14T06:18:00.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.768965+0000", 2024-09-14T06:18:00.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088165091000000001 2024-09-14T06:18:00.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769059+0000", 2024-09-14T06:18:00.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3923000000000002e-05 2024-09-14T06:18:00.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769158+0000", 2024-09-14T06:18:00.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8820000000000006e-05 2024-09-14T06:18:00.420 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.420 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.420 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.420 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39004 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 278528~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.420 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:18:00.420 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.653868152000001, 2024-09-14T06:18:00.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088499624999999998, 2024-09-14T06:18:00.420 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.421 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.421 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.421 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.421 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.421 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.421 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39004 2024-09-14T06:18:00.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.421 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:18:00.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680810+0000", 2024-09-14T06:18:00.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680811+0000", 2024-09-14T06:18:00.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:18:00.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680814+0000", 2024-09-14T06:18:00.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:18:00.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680821+0000", 2024-09-14T06:18:00.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9009999999999997e-06 2024-09-14T06:18:00.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.423 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.423 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680826+0000", 2024-09-14T06:18:00.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5350000000000004e-06 2024-09-14T06:18:00.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769189+0000", 2024-09-14T06:18:00.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088363024999999998 2024-09-14T06:18:00.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769246+0000", 2024-09-14T06:18:00.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7017e-05 2024-09-14T06:18:00.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769310+0000", 2024-09-14T06:18:00.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3541999999999996e-05 2024-09-14T06:18:00.425 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.425 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.425 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.425 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39005 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 286720~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.425 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:18:00.425 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.653844067999998, 2024-09-14T06:18:00.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088681092000000003, 2024-09-14T06:18:00.425 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.425 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.425 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.425 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.426 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.426 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.426 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39005 2024-09-14T06:18:00.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.426 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.426 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:18:00.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.426 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680834+0000", 2024-09-14T06:18:00.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680835+0000", 2024-09-14T06:18:00.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2699999999999999e-07 2024-09-14T06:18:00.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680837+0000", 2024-09-14T06:18:00.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.413e-06 2024-09-14T06:18:00.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680844+0000", 2024-09-14T06:18:00.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8820000000000003e-06 2024-09-14T06:18:00.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680850+0000", 2024-09-14T06:18:00.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5169999999999999e-06 2024-09-14T06:18:00.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769337+0000", 2024-09-14T06:18:00.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088487471999999998 2024-09-14T06:18:00.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769404+0000", 2024-09-14T06:18:00.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7111000000000002e-05 2024-09-14T06:18:00.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769515+0000", 2024-09-14T06:18:00.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011116999999999999 2024-09-14T06:18:00.429 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.429 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.429 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.429 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39006 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 294912~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.429 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:18:00.429 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.653820905000003, 2024-09-14T06:18:00.430 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088886572999999997, 2024-09-14T06:18:00.430 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.430 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.430 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.430 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.430 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.430 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.430 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39006 2024-09-14T06:18:00.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.430 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.430 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.430 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.430 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:18:00.430 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.431 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.431 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680857+0000", 2024-09-14T06:18:00.431 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.431 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.431 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680858+0000", 2024-09-14T06:18:00.431 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7e-07 2024-09-14T06:18:00.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.431 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.431 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680860+0000", 2024-09-14T06:18:00.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:18:00.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680867+0000", 2024-09-14T06:18:00.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0430000000000002e-06 2024-09-14T06:18:00.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680873+0000", 2024-09-14T06:18:00.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7389999999999996e-06 2024-09-14T06:18:00.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769547+0000", 2024-09-14T06:18:00.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.088673830999999995 2024-09-14T06:18:00.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769631+0000", 2024-09-14T06:18:00.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3660999999999997e-05 2024-09-14T06:18:00.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769744+0000", 2024-09-14T06:18:00.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000113482 2024-09-14T06:18:00.433 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.434 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.434 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.434 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:39007 3.14 3:2aeebcdc:::rbd_data.115274ed0aa8.0000000000000005:14 [read 303104~4096] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:00.434 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:18:00.434 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.653797492999999, 2024-09-14T06:18:00.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.089077397000000003, 2024-09-14T06:18:00.434 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.434 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.434 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.434 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.434 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.435 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.435 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 39007 2024-09-14T06:18:00.435 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.435 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.435 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.435 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.435 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:18:00.435 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.435 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.435 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.435 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.435 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.680881+0000", 2024-09-14T06:18:00.435 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.435 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.436 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.436 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687953+0000", 2024-09-14T06:18:00.436 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00707203 2024-09-14T06:18:00.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.436 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.436 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687960+0000", 2024-09-14T06:18:00.436 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9890000000000003e-06 2024-09-14T06:18:00.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.436 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.437 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687974+0000", 2024-09-14T06:18:00.437 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4226999999999999e-05 2024-09-14T06:18:00.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.437 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.437 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.437 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.687983+0000", 2024-09-14T06:18:00.437 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9800000000000004e-06 2024-09-14T06:18:00.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.437 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.437 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.437 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769775+0000", 2024-09-14T06:18:00.437 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.081792089999999998 2024-09-14T06:18:00.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.438 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.438 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769849+0000", 2024-09-14T06:18:00.438 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3925999999999997e-05 2024-09-14T06:18:00.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.438 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.438 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:09.769958+0000", 2024-09-14T06:18:00.438 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010915499999999999 2024-09-14T06:18:00.438 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.438 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.438 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.438 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.438 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.438 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:00.440 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:18:00.440 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:18:00.441 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:18:00.442 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:00.437+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:00.541 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:00.539+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:00.619 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:00.619 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:18:00.619 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:18:00.619 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:00.633 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:18:00.642 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:00.639+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:00.743 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:00.739+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:00.760 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:00.760 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:18:00.760 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:18:00.760 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:18:00.760 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:00.774 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:18:00.844 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:00.841+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:00.903 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:00.903 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:18:00.903 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:18:00.903 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:18:00.904 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.904 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3916 3.16 3:6ab8b1d8:::rbd_data.1152bd92e04f.0000000000000002:head [stat,truncate 1590169] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:18:00.904 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:18:00.904 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 93.321456518000005, 2024-09-14T06:18:00.904 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022958078999999999, 2024-09-14T06:18:00.904 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.904 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.904 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:00.904 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.904 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.904 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.904 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3916 2024-09-14T06:18:00.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.905 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.905 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.905 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.905 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:18:00.905 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.905 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.905 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.905 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:18:00.905 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.905 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.905 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.906 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579203+0000", 2024-09-14T06:18:00.906 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.748e-06 2024-09-14T06:18:00.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.906 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.906 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.906 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579208+0000", 2024-09-14T06:18:00.906 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:18:00.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.906 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.906 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.906 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579223+0000", 2024-09-14T06:18:00.906 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.508e-05 2024-09-14T06:18:00.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.906 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.907 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.907 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579232+0000", 2024-09-14T06:18:00.907 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9719999999999998e-06 2024-09-14T06:18:00.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.907 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.907 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.907 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579307+0000", 2024-09-14T06:18:00.907 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5028000000000001e-05 2024-09-14T06:18:00.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.907 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.907 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.907 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579412+0000", 2024-09-14T06:18:00.907 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010525400000000001 2024-09-14T06:18:00.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.908 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.908 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:18:00.908 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579535+0000", 2024-09-14T06:18:00.908 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122726 2024-09-14T06:18:00.908 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.908 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.908 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:00.908 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.581415+0000", 2024-09-14T06:18:00.908 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018800360000000001 2024-09-14T06:18:00.908 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.908 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.908 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:00.908 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602017+0000", 2024-09-14T06:18:00.908 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.020602425000000001 2024-09-14T06:18:00.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.909 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.909 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:00.909 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602096+0000", 2024-09-14T06:18:00.909 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8968999999999996e-05 2024-09-14T06:18:00.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.909 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.909 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.909 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602159+0000", 2024-09-14T06:18:00.909 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2824999999999996e-05 2024-09-14T06:18:00.909 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.909 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.909 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.910 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.910 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:7151 3.e 3:7793b17d:::rbd_data.1152f87cbac6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:18:00.910 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:18:00.910 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 89.571060650999996, 2024-09-14T06:18:00.910 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.029556783, 2024-09-14T06:18:00.910 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.910 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.910 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:00.910 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.910 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.910 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.910 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 7151 2024-09-14T06:18:00.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.911 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.911 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.911 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.911 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:18:00.911 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.911 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.911 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.911 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:18:00.911 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.911 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.911 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.912 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329599+0000", 2024-09-14T06:18:00.912 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5270000000000001e-06 2024-09-14T06:18:00.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.912 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.912 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.912 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329610+0000", 2024-09-14T06:18:00.912 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0431000000000001e-05 2024-09-14T06:18:00.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.912 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.912 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.912 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329634+0000", 2024-09-14T06:18:00.912 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3617000000000001e-05 2024-09-14T06:18:00.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.912 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.913 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.913 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329644+0000", 2024-09-14T06:18:00.913 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.095e-05 2024-09-14T06:18:00.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.913 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.913 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.913 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329712+0000", 2024-09-14T06:18:00.913 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7067e-05 2024-09-14T06:18:00.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.913 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.913 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.913 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329827+0000", 2024-09-14T06:18:00.913 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011498 2024-09-14T06:18:00.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.913 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.914 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:18:00.914 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.334271+0000", 2024-09-14T06:18:00.914 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044445839999999997 2024-09-14T06:18:00.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.914 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.914 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:00.914 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.358967+0000", 2024-09-14T06:18:00.914 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024696356999999999 2024-09-14T06:18:00.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.914 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.914 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:00.914 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359027+0000", 2024-09-14T06:18:00.914 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0006000000000001e-05 2024-09-14T06:18:00.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.915 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.915 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:00.915 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359099+0000", 2024-09-14T06:18:00.915 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1052999999999999e-05 2024-09-14T06:18:00.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.915 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.915 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.915 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359154+0000", 2024-09-14T06:18:00.915 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5210999999999997e-05 2024-09-14T06:18:00.915 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.915 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.915 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.915 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.916 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:7151 3.e e32/29 ondisk, result = 0)", 2024-09-14T06:18:00.916 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:18:00.916 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 89.551116217000001, 2024-09-14T06:18:00.916 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0096251340000000005, 2024-09-14T06:18:00.916 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.916 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.916 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.916 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.916 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.916 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.916 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:18:00.916 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.916 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.917 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.917 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:18:00.917 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.917 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.917 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.917 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.917 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349543+0000", 2024-09-14T06:18:00.917 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:18:00.917 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.917 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.917 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.917 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349552+0000", 2024-09-14T06:18:00.917 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1039999999999996e-06 2024-09-14T06:18:00.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.918 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.918 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.918 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349579+0000", 2024-09-14T06:18:00.918 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6449999999999999e-05 2024-09-14T06:18:00.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.918 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.918 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.918 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349593+0000", 2024-09-14T06:18:00.918 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4099e-05 2024-09-14T06:18:00.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.918 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.918 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.918 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359003+0000", 2024-09-14T06:18:00.918 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0094096119999999995 2024-09-14T06:18:00.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.919 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.919 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.919 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359014+0000", 2024-09-14T06:18:00.919 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0871e-05 2024-09-14T06:18:00.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.919 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.919 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.919 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359167+0000", 2024-09-14T06:18:00.919 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015300700000000001 2024-09-14T06:18:00.919 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.919 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.919 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.919 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.919 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15619 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'124, mlcod=36'119)", 2024-09-14T06:18:00.920 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:18:00.920 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 84.602803054999995, 2024-09-14T06:18:00.920 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061545389999999997, 2024-09-14T06:18:00.920 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.920 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.920 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:00.920 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.920 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.920 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.920 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:18:00.920 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.920 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.920 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.921 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:18:00.921 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.921 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.921 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.921 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:18:00.921 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3399999999999998e-07 2024-09-14T06:18:00.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.921 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.921 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.921 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297861+0000", 2024-09-14T06:18:00.921 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7640000000000002e-06 2024-09-14T06:18:00.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.921 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.922 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.922 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297876+0000", 2024-09-14T06:18:00.922 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4881e-05 2024-09-14T06:18:00.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.922 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.922 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.922 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297884+0000", 2024-09-14T06:18:00.922 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.588e-06 2024-09-14T06:18:00.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.922 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.922 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.922 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298294+0000", 2024-09-14T06:18:00.922 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00041011200000000002 2024-09-14T06:18:00.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.923 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.923 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.923 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298317+0000", 2024-09-14T06:18:00.923 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3417999999999998e-05 2024-09-14T06:18:00.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.923 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.923 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:00.923 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303972+0000", 2024-09-14T06:18:00.923 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056548680000000004 2024-09-14T06:18:00.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.923 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.923 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.923 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304009+0000", 2024-09-14T06:18:00.923 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7073999999999998e-05 2024-09-14T06:18:00.923 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.924 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.924 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.924 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.924 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15621 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'125, mlcod=36'119)", 2024-09-14T06:18:00.924 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:18:00.924 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 84.601292037999997, 2024-09-14T06:18:00.924 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0069496169999999999, 2024-09-14T06:18:00.924 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.924 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.924 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:00.924 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.924 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.924 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.925 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:18:00.925 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.925 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.925 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.925 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:18:00.925 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.925 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.925 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.925 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:18:00.925 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2200000000000003e-07 2024-09-14T06:18:00.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.925 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.926 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.926 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299372+0000", 2024-09-14T06:18:00.926 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3850000000000001e-06 2024-09-14T06:18:00.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.926 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.926 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.926 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299387+0000", 2024-09-14T06:18:00.926 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4905e-05 2024-09-14T06:18:00.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.926 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.926 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.926 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299393+0000", 2024-09-14T06:18:00.926 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9090000000000003e-06 2024-09-14T06:18:00.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.927 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.927 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.927 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303591+0000", 2024-09-14T06:18:00.927 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041976640000000003 2024-09-14T06:18:00.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.927 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.927 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.927 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303619+0000", 2024-09-14T06:18:00.927 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7651e-05 2024-09-14T06:18:00.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.927 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.927 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:00.927 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306287+0000", 2024-09-14T06:18:00.927 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026686689999999998 2024-09-14T06:18:00.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.928 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.928 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.928 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306315+0000", 2024-09-14T06:18:00.928 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7712000000000002e-05 2024-09-14T06:18:00.928 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.928 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.928 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.928 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.928 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:24229 3.5 e47/34 3:a30fd30b:::rbd_data.1152db28e106.0000000000000001:head v 47'55, mlcod=45'53)", 2024-09-14T06:18:00.928 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:18:00.928 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 68.990646440000006, 2024-09-14T06:18:00.929 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0092924199999999992, 2024-09-14T06:18:00.929 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.929 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.929 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:00.929 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.929 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.929 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.929 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:18:00.929 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.929 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.929 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.929 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:18:00.930 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.930 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.930 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.930 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910013+0000", 2024-09-14T06:18:00.930 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.812e-06 2024-09-14T06:18:00.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.930 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.930 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.930 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915374+0000", 2024-09-14T06:18:00.930 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053609240000000004 2024-09-14T06:18:00.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.930 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.930 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.931 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915395+0000", 2024-09-14T06:18:00.931 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1118e-05 2024-09-14T06:18:00.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.931 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.931 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.931 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915406+0000", 2024-09-14T06:18:00.931 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1303e-05 2024-09-14T06:18:00.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.931 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.931 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.931 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915490+0000", 2024-09-14T06:18:00.931 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4017000000000006e-05 2024-09-14T06:18:00.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.932 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.932 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.932 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915515+0000", 2024-09-14T06:18:00.932 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4712e-05 2024-09-14T06:18:00.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.932 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.932 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:00.932 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919240+0000", 2024-09-14T06:18:00.932 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037247690000000002 2024-09-14T06:18:00.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.932 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.932 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.932 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919304+0000", 2024-09-14T06:18:00.932 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3764999999999997e-05 2024-09-14T06:18:00.933 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.933 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.933 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.933 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.933 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:head, version: 31'47, data_included: [0~98304,360448~278528], data_size: 376832, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:head@31'47, size: 638976, copy_subset: [(0, 638976)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 638976, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:18:00.933 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:18:00.933 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 60.690383494999999, 2024-09-14T06:18:00.933 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062487460000000003, 2024-09-14T06:18:00.933 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.933 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.933 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.933 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:18:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:18:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210275+0000", 2024-09-14T06:18:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:18:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.935 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.935 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.935 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210752+0000", 2024-09-14T06:18:00.935 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047772100000000002 2024-09-14T06:18:00.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.935 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.935 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.935 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210781+0000", 2024-09-14T06:18:00.935 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8391999999999999e-05 2024-09-14T06:18:00.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.935 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.935 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.935 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210805+0000", 2024-09-14T06:18:00.936 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4142999999999999e-05 2024-09-14T06:18:00.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.936 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.936 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.936 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.213641+0000", 2024-09-14T06:18:00.936 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002835827 2024-09-14T06:18:00.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.936 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.936 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.936 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.213663+0000", 2024-09-14T06:18:00.936 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.243e-05 2024-09-14T06:18:00.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.936 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.936 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.937 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.216523+0000", 2024-09-14T06:18:00.937 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028597729999999999 2024-09-14T06:18:00.937 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.937 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.937 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.937 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.937 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:50564 3.16 3:689e8220:::rbd_header.1152757de986:head [call rbd.snapshot_add in=15b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-14T06:18:00.937 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:13.319284+0000", 2024-09-14T06:18:00.937 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.581373114999998, 2024-09-14T06:18:00.937 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.030881613, 2024-09-14T06:18:00.937 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.937 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.938 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:00.938 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.938 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.938 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.938 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50564 2024-09-14T06:18:00.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.938 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.938 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.938 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.938 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319284+0000", 2024-09-14T06:18:00.938 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.938 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.938 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.938 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319284+0000", 2024-09-14T06:18:00.939 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.939 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.939 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.939 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319286+0000", 2024-09-14T06:18:00.939 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.849e-06 2024-09-14T06:18:00.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.939 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.939 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.939 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319293+0000", 2024-09-14T06:18:00.939 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.72e-06 2024-09-14T06:18:00.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.939 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.939 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.940 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319313+0000", 2024-09-14T06:18:00.940 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9833000000000001e-05 2024-09-14T06:18:00.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.940 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.940 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.940 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319325+0000", 2024-09-14T06:18:00.940 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1924e-05 2024-09-14T06:18:00.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.940 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.940 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.940 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319441+0000", 2024-09-14T06:18:00.940 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000115952 2024-09-14T06:18:00.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.941 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.941 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op must wait for map", 2024-09-14T06:18:00.941 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319453+0000", 2024-09-14T06:18:00.941 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1810999999999999e-05 2024-09-14T06:18:00.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.941 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.941 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.941 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322905+0000", 2024-09-14T06:18:00.941 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034519680000000001 2024-09-14T06:18:00.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.941 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.941 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:18:00.941 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322917+0000", 2024-09-14T06:18:00.942 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1946999999999999e-05 2024-09-14T06:18:00.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.942 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.942 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.942 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.342215+0000", 2024-09-14T06:18:00.942 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0192988300000001 2024-09-14T06:18:00.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.942 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.942 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.942 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.342451+0000", 2024-09-14T06:18:00.942 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00023546 2024-09-14T06:18:00.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.942 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.943 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:00.943 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.342959+0000", 2024-09-14T06:18:00.943 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050845999999999997 2024-09-14T06:18:00.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.943 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.943 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:00.943 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.347472+0000", 2024-09-14T06:18:00.943 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045125310000000002 2024-09-14T06:18:00.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.943 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.943 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:00.943 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.349544+0000", 2024-09-14T06:18:00.943 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020721730000000001 2024-09-14T06:18:00.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.944 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.944 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:00.944 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.349727+0000", 2024-09-14T06:18:00.944 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000182776 2024-09-14T06:18:00.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.944 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.944 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.944 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.350166+0000", 2024-09-14T06:18:00.944 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000439379 2024-09-14T06:18:00.945 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.945 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.945 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.945 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.945 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:50565 3.16 3:689e8220:::rbd_header.1152757de986:head [watch reconnect cookie 140175519011472 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-14T06:18:00.945 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:13.319351+0000", 2024-09-14T06:18:00.945 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.581306537000003, 2024-09-14T06:18:00.945 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0352714430000001, 2024-09-14T06:18:00.945 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.945 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.945 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.945 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.946 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.946 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.946 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50565 2024-09-14T06:18:00.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.946 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.946 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.946 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.946 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319351+0000", 2024-09-14T06:18:00.946 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.946 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.946 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.946 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319351+0000", 2024-09-14T06:18:00.946 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.947 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.947 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.947 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319352+0000", 2024-09-14T06:18:00.947 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0500000000000003e-07 2024-09-14T06:18:00.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.947 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.947 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.947 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319355+0000", 2024-09-14T06:18:00.947 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3960000000000002e-06 2024-09-14T06:18:00.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.947 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.947 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.947 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319367+0000", 2024-09-14T06:18:00.948 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1518999999999999e-05 2024-09-14T06:18:00.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.948 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.948 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.948 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319373+0000", 2024-09-14T06:18:00.948 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7800000000000003e-06 2024-09-14T06:18:00.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.948 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.948 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.948 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319517+0000", 2024-09-14T06:18:00.948 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000143719 2024-09-14T06:18:00.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.948 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.948 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting_for_map not empty", 2024-09-14T06:18:00.949 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319520+0000", 2024-09-14T06:18:00.949 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1990000000000002e-06 2024-09-14T06:18:00.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.949 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.949 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322963+0000", 2024-09-14T06:18:00.949 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034424920000000001 2024-09-14T06:18:00.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.949 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:18:00.949 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322971+0000", 2024-09-14T06:18:00.949 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0860000000000002e-06 2024-09-14T06:18:00.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.950 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.950 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343565+0000", 2024-09-14T06:18:00.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0205935530000001 2024-09-14T06:18:00.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.950 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.950 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for rw locks", 2024-09-14T06:18:00.950 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343675+0000", 2024-09-14T06:18:00.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011048900000000001 2024-09-14T06:18:00.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.950 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.950 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.950 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.350295+0000", 2024-09-14T06:18:00.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006620094 2024-09-14T06:18:00.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.951 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.951 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.951 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.350584+0000", 2024-09-14T06:18:00.951 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00028850000000000002 2024-09-14T06:18:00.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.951 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.951 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.951 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.354623+0000", 2024-09-14T06:18:00.951 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040389110000000001 2024-09-14T06:18:00.951 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.951 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.951 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.951 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.952 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:50566 3.16 3:689e8220:::rbd_header.1152757de986:head [watch ping cookie 140175519011472 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-14T06:18:00.952 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:13.960177+0000", 2024-09-14T06:18:00.952 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 46.940480389999998, 2024-09-14T06:18:00.952 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.39354602500000002, 2024-09-14T06:18:00.952 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.952 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.952 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.952 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.952 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.952 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.952 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50566 2024-09-14T06:18:00.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.952 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.952 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.953 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.953 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960177+0000", 2024-09-14T06:18:00.953 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.953 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.953 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.953 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960177+0000", 2024-09-14T06:18:00.953 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.953 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.953 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.953 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960178+0000", 2024-09-14T06:18:00.953 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.136e-06 2024-09-14T06:18:00.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.954 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.954 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.954 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960186+0000", 2024-09-14T06:18:00.954 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0299999999999994e-06 2024-09-14T06:18:00.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.954 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.954 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.954 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960202+0000", 2024-09-14T06:18:00.954 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5238999999999999e-05 2024-09-14T06:18:00.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.954 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.954 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.954 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960218+0000", 2024-09-14T06:18:00.954 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.668e-05 2024-09-14T06:18:00.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.955 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.955 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960350+0000", 2024-09-14T06:18:00.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013201 2024-09-14T06:18:00.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.955 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:18:00.955 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960368+0000", 2024-09-14T06:18:00.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7526000000000001e-05 2024-09-14T06:18:00.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.955 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.955 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343805+0000", 2024-09-14T06:18:00.956 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.38343735400000001 2024-09-14T06:18:00.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.956 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.956 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for rw locks", 2024-09-14T06:18:00.956 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343907+0000", 2024-09-14T06:18:00.956 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010133599999999999 2024-09-14T06:18:00.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.956 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.956 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.956 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.352997+0000", 2024-09-14T06:18:00.956 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0090907409999999994 2024-09-14T06:18:00.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.956 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.956 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.957 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.353352+0000", 2024-09-14T06:18:00.957 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00035436000000000002 2024-09-14T06:18:00.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.957 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.957 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.353723+0000", 2024-09-14T06:18:00.957 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00037161300000000002 2024-09-14T06:18:00.960 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.960 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.960 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.960 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:51037 3.11 3:89b51a5c:::rbd_data.11522fd7a0c5.0000000000000059:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-14T06:18:00.960 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:15.038049+0000", 2024-09-14T06:18:00.960 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 45.862609048000003, 2024-09-14T06:18:00.960 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.075311116999999997, 2024-09-14T06:18:00.960 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.960 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.961 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:00.961 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:00.961 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:00.961 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:00.961 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 51037 2024-09-14T06:18:00.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.961 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.961 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.961 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.961 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038049+0000", 2024-09-14T06:18:00.961 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.962 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.962 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038049+0000", 2024-09-14T06:18:00.962 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.962 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.962 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038051+0000", 2024-09-14T06:18:00.962 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:18:00.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.962 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.963 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038065+0000", 2024-09-14T06:18:00.963 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3576e-05 2024-09-14T06:18:00.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.963 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.963 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038097+0000", 2024-09-14T06:18:00.963 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2566e-05 2024-09-14T06:18:00.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.963 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.963 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038121+0000", 2024-09-14T06:18:00.963 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3207e-05 2024-09-14T06:18:00.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.963 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.963 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038291+0000", 2024-09-14T06:18:00.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016996900000000001 2024-09-14T06:18:00.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.964 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.964 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038543+0000", 2024-09-14T06:18:00.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00025233999999999999 2024-09-14T06:18:00.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.964 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:00.964 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.053218+0000", 2024-09-14T06:18:00.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014674957000000001 2024-09-14T06:18:00.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.964 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:00.965 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113195+0000", 2024-09-14T06:18:00.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.059976697000000002 2024-09-14T06:18:00.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.965 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:00.965 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113212+0000", 2024-09-14T06:18:00.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7299000000000002e-05 2024-09-14T06:18:00.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.965 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:00.965 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113267+0000", 2024-09-14T06:18:00.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4632999999999997e-05 2024-09-14T06:18:00.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.966 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.966 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.966 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113360+0000", 2024-09-14T06:18:00.966 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3129999999999998e-05 2024-09-14T06:18:00.966 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.966 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.966 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.966 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.966 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:51037 3.11 e67/66 ondisk, result = 0)", 2024-09-14T06:18:00.966 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:15.093818+0000", 2024-09-14T06:18:00.966 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 45.806839257, 2024-09-14T06:18:00.966 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.019384384000000001, 2024-09-14T06:18:00.966 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.966 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.967 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:00.967 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.967 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.967 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093818+0000", 2024-09-14T06:18:00.967 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.967 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.967 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093818+0000", 2024-09-14T06:18:00.967 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.967 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.968 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093820+0000", 2024-09-14T06:18:00.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0169999999999999e-06 2024-09-14T06:18:00.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.968 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.968 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093829+0000", 2024-09-14T06:18:00.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6500000000000002e-06 2024-09-14T06:18:00.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.968 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.968 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093860+0000", 2024-09-14T06:18:00.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1275999999999997e-05 2024-09-14T06:18:00.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093883+0000", 2024-09-14T06:18:00.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2464999999999999e-05 2024-09-14T06:18:00.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113178+0000", 2024-09-14T06:18:00.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.019295643000000001 2024-09-14T06:18:00.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113189+0000", 2024-09-14T06:18:00.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0285999999999999e-05 2024-09-14T06:18:00.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.970 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.970 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113203+0000", 2024-09-14T06:18:00.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4047e-05 2024-09-14T06:18:00.970 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.970 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.970 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.970 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:52268 3.e e70/34 3:7013111b:::rbd_data.1152cd930a87.0000000000000001:head v 70'315, mlcod=70'313)", 2024-09-14T06:18:00.970 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:17.975434+0000", 2024-09-14T06:18:00.970 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.925224059000001, 2024-09-14T06:18:00.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014816765000000001, 2024-09-14T06:18:00.971 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.971 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.971 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:00.971 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.971 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.971 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.975434+0000", 2024-09-14T06:18:00.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.971 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.971 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.975434+0000", 2024-09-14T06:18:00.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.975436+0000", 2024-09-14T06:18:00.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3010000000000001e-06 2024-09-14T06:18:00.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982453+0000", 2024-09-14T06:18:00.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0070167909999999997 2024-09-14T06:18:00.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982489+0000", 2024-09-14T06:18:00.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6603000000000002e-05 2024-09-14T06:18:00.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.973 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.973 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982507+0000", 2024-09-14T06:18:00.973 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8074e-05 2024-09-14T06:18:00.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.973 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.973 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982610+0000", 2024-09-14T06:18:00.973 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010224 2024-09-14T06:18:00.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.973 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.973 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982639+0000", 2024-09-14T06:18:00.973 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9009999999999998e-05 2024-09-14T06:18:00.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:00.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.990188+0000", 2024-09-14T06:18:00.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0075497150000000002 2024-09-14T06:18:00.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.990250+0000", 2024-09-14T06:18:00.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2031000000000003e-05 2024-09-14T06:18:00.974 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.974 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.974 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.975 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:52269 3.1f e70/56 3:fd802c48:::rbd_data.1152cd930a87.0000000000000002:head v 70'321, mlcod=70'318)", 2024-09-14T06:18:00.975 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:17.982671+0000", 2024-09-14T06:18:00.975 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 42.917987076999999, 2024-09-14T06:18:00.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016453552, 2024-09-14T06:18:00.975 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.975 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.975 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:00.975 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982671+0000", 2024-09-14T06:18:00.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982671+0000", 2024-09-14T06:18:00.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982672+0000", 2024-09-14T06:18:00.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.246e-06 2024-09-14T06:18:00.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991501+0000", 2024-09-14T06:18:00.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0088292590000000008 2024-09-14T06:18:00.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.977 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.977 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991521+0000", 2024-09-14T06:18:00.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0211e-05 2024-09-14T06:18:00.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.977 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.977 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.977 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991531+0000", 2024-09-14T06:18:00.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0207000000000001e-05 2024-09-14T06:18:00.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.977 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.977 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.977 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991605+0000", 2024-09-14T06:18:00.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3066999999999997e-05 2024-09-14T06:18:00.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991633+0000", 2024-09-14T06:18:00.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8136e-05 2024-09-14T06:18:00.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:00.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.999052+0000", 2024-09-14T06:18:00.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0074192320000000004 2024-09-14T06:18:00.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.999124+0000", 2024-09-14T06:18:00.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2193999999999994e-05 2024-09-14T06:18:00.978 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.979 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.979 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.979 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:59899 3.9 e75/68 3:96b79e96:::rbd_data.115214b4562.0000000000000004:head v 75'319, mlcod=75'317)", 2024-09-14T06:18:00.979 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:24.822693+0000", 2024-09-14T06:18:00.979 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.077964225999999, 2024-09-14T06:18:00.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014246999, 2024-09-14T06:18:00.979 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.979 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.979 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:00.979 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.979 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.979 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.822693+0000", 2024-09-14T06:18:00.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.822693+0000", 2024-09-14T06:18:00.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.822695+0000", 2024-09-14T06:18:00.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.762e-06 2024-09-14T06:18:00.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831397+0000", 2024-09-14T06:18:00.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0087016230000000003 2024-09-14T06:18:00.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831424+0000", 2024-09-14T06:18:00.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7480000000000001e-05 2024-09-14T06:18:00.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831435+0000", 2024-09-14T06:18:00.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0943e-05 2024-09-14T06:18:00.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831509+0000", 2024-09-14T06:18:00.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3596000000000005e-05 2024-09-14T06:18:00.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831531+0000", 2024-09-14T06:18:00.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2628000000000001e-05 2024-09-14T06:18:00.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:00.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.836873+0000", 2024-09-14T06:18:00.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053417849999999999 2024-09-14T06:18:00.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.983 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.983 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.836940+0000", 2024-09-14T06:18:00.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7181999999999995e-05 2024-09-14T06:18:00.983 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.983 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.983 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.983 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:64148 3.2 e86/29 3:41d40d1e:::rbd_data.115267b2c56c.0000000000000000:head v 86'276, mlcod=80'274)", 2024-09-14T06:18:00.983 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:38.879703+0000", 2024-09-14T06:18:00.983 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.020954597999999, 2024-09-14T06:18:00.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014921665000000001, 2024-09-14T06:18:00.983 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.984 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.984 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:00.984 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.984 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.879703+0000", 2024-09-14T06:18:00.984 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.984 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.879703+0000", 2024-09-14T06:18:00.984 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.879704+0000", 2024-09-14T06:18:00.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0219999999999999e-06 2024-09-14T06:18:00.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885314+0000", 2024-09-14T06:18:00.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056102119999999998 2024-09-14T06:18:00.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885331+0000", 2024-09-14T06:18:00.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6696000000000001e-05 2024-09-14T06:18:00.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885342+0000", 2024-09-14T06:18:00.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0886999999999999e-05 2024-09-14T06:18:00.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885468+0000", 2024-09-14T06:18:00.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012655399999999999 2024-09-14T06:18:00.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885517+0000", 2024-09-14T06:18:00.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9057999999999997e-05 2024-09-14T06:18:00.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:00.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.894572+0000", 2024-09-14T06:18:00.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0090541130000000008 2024-09-14T06:18:00.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.894625+0000", 2024-09-14T06:18:00.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3122999999999999e-05 2024-09-14T06:18:00.987 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.987 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.987 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.987 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:64152 3.3 e86/29 3:c1c8d963:::rbd_data.115267b2c56c.0000000000000002:head v 86'180, mlcod=79'178)", 2024-09-14T06:18:00.987 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:38.884954+0000", 2024-09-14T06:18:00.987 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.015703135999999, 2024-09-14T06:18:00.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011982222000000001, 2024-09-14T06:18:00.988 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.988 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.988 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:00.988 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.884954+0000", 2024-09-14T06:18:00.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.884954+0000", 2024-09-14T06:18:00.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.884956+0000", 2024-09-14T06:18:00.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1179999999999999e-06 2024-09-14T06:18:00.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886783+0000", 2024-09-14T06:18:00.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018269289999999999 2024-09-14T06:18:00.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886795+0000", 2024-09-14T06:18:00.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1975e-05 2024-09-14T06:18:00.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886802+0000", 2024-09-14T06:18:00.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.824e-06 2024-09-14T06:18:00.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886881+0000", 2024-09-14T06:18:00.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8702000000000006e-05 2024-09-14T06:18:00.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886902+0000", 2024-09-14T06:18:00.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0854000000000001e-05 2024-09-14T06:18:00.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:00.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.896887+0000", 2024-09-14T06:18:00.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0099853700000000004 2024-09-14T06:18:00.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.991 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.896937+0000", 2024-09-14T06:18:00.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9450000000000003e-05 2024-09-14T06:18:00.991 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.991 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.992 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.992 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:64155 3.1 e86/29 3:86008c25:::rbd_data.115267b2c56c.0000000000000003:head v 86'451, mlcod=78'449)", 2024-09-14T06:18:00.992 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:18:00.992 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 21.997602050000001, 2024-09-14T06:18:00.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006333038, 2024-09-14T06:18:00.992 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.992 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.992 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:00.992 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:18:00.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:18:00.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:18:00.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:18:00.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906826+0000", 2024-09-14T06:18:00.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037703260000000001 2024-09-14T06:18:00.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906845+0000", 2024-09-14T06:18:00.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8488999999999999e-05 2024-09-14T06:18:00.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906857+0000", 2024-09-14T06:18:00.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2571e-05 2024-09-14T06:18:00.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906966+0000", 2024-09-14T06:18:00.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010832000000000001 2024-09-14T06:18:00.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906984+0000", 2024-09-14T06:18:00.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8068999999999999e-05 2024-09-14T06:18:00.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:00.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.909306+0000", 2024-09-14T06:18:00.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023221050000000001 2024-09-14T06:18:00.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:00.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.909389+0000", 2024-09-14T06:18:00.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2865e-05 2024-09-14T06:18:00.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.996 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:00.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:00.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.996 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(osd.5.0:1871 3.1e e99/29 3:783464bd:::rbd_data.11522fd7a0c5.000000000000005b:19 v 99'341, mlcod=95'338)", 2024-09-14T06:18:00.996 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:53.229369+0000", 2024-09-14T06:18:00.996 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 7.671288519, 2024-09-14T06:18:00.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0063464239999999998, 2024-09-14T06:18:00.997 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:00.997 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:00.997 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:00.997 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:00.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.997 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:00.997 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229369+0000", 2024-09-14T06:18:00.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.997 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:00.997 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229369+0000", 2024-09-14T06:18:00.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:00.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:00.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229370+0000", 2024-09-14T06:18:00.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7300000000000005e-07 2024-09-14T06:18:00.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:00.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229388+0000", 2024-09-14T06:18:00.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8006999999999999e-05 2024-09-14T06:18:00.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:00.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229406+0000", 2024-09-14T06:18:00.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7663e-05 2024-09-14T06:18:00.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:00.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229417+0000", 2024-09-14T06:18:00.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1749999999999999e-05 2024-09-14T06:18:00.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:00.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230797+0000", 2024-09-14T06:18:00.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0013801589999999999 2024-09-14T06:18:00.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:00.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:00.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:00.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.233207+0000", 2024-09-14T06:18:00.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024092889999999998 2024-09-14T06:18:00.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:01.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.235660+0000", 2024-09-14T06:18:01.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024534370000000002 2024-09-14T06:18:01.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.235716+0000", 2024-09-14T06:18:01.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5346000000000002e-05 2024-09-14T06:18:01.000 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.000 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.000 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.000 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(osd.0.0:2754 3.e e99/34 3:70005258:::rbd_data.11522fd7a0c5.000000000000018a:19 v 99'690, mlcod=98'687)", 2024-09-14T06:18:01.001 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:53.230299+0000", 2024-09-14T06:18:01.001 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 7.6703586059999997, 2024-09-14T06:18:01.001 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064664880000000003, 2024-09-14T06:18:01.001 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.001 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.001 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:01.001 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.001 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.001 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230299+0000", 2024-09-14T06:18:01.001 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.001 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.001 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230299+0000", 2024-09-14T06:18:01.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230299+0000", 2024-09-14T06:18:01.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.32e-07 2024-09-14T06:18:01.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230304+0000", 2024-09-14T06:18:01.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8729999999999998e-06 2024-09-14T06:18:01.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230316+0000", 2024-09-14T06:18:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2104e-05 2024-09-14T06:18:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230325+0000", 2024-09-14T06:18:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4079999999999998e-06 2024-09-14T06:18:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230896+0000", 2024-09-14T06:18:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057151700000000001 2024-09-14T06:18:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.234280+0000", 2024-09-14T06:18:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033832060000000001 2024-09-14T06:18:01.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:01.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.236724+0000", 2024-09-14T06:18:01.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024447090000000002 2024-09-14T06:18:01.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.236765+0000", 2024-09-14T06:18:01.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1239000000000003e-05 2024-09-14T06:18:01.004 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.004 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.004 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.004 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.005 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.005 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:01.006 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:18:01.007 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:01.003+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:01.107 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:01.105+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:01.178 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:01.178 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:18:01.178 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:18:01.178 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:01.192 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:18:01.215 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:18:01.236 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:01.233+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:01.309 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:01.307+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:01.362 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:01.362 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:18:01.363 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:18:01.363 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:18:01.363 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:01.382 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:18:01.409 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:01.407+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:01.509 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:01.505+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:01.513 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:01.514 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:18:01.514 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:18:01.514 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:18:01.514 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.514 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:01.514 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:18:01.514 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 85.520889722999996, 2024-09-14T06:18:01.514 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:18:01.514 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.514 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.514 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:01.515 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.515 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.515 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.515 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:18:01.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.515 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.515 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.515 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.515 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:18:01.515 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.515 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.515 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.515 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:18:01.516 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.516 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.516 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.516 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:18:01.516 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:18:01.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.516 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.516 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.516 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:18:01.516 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:18:01.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.516 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.516 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.517 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:18:01.517 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:18:01.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.517 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.517 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.517 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:18:01.517 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:18:01.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.517 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.517 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.517 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:18:01.517 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:18:01.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.518 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.518 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.518 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:18:01.518 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:18:01.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.518 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.518 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.518 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:18:01.518 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:18:01.518 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.518 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.518 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.519 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.519 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:01.519 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:18:01.519 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 85.520876650999995, 2024-09-14T06:18:01.519 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:18:01.519 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.519 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.519 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:01.519 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.519 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.519 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.519 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:18:01.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.519 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.520 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.520 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.520 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:18:01.520 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.520 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.520 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.520 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:18:01.520 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.520 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.520 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.520 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:18:01.520 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:18:01.521 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.521 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.521 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.521 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:18:01.521 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:18:01.521 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.521 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.521 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.521 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:18:01.521 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:18:01.521 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.521 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.521 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.521 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:18:01.521 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:18:01.522 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.522 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.522 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.522 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:18:01.522 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:18:01.522 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.522 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.522 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.522 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:18:01.522 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:18:01.522 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.522 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.522 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.522 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:18:01.523 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:18:01.523 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.523 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.523 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.523 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.523 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:01.523 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:18:01.523 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 85.520859219000002, 2024-09-14T06:18:01.523 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:18:01.523 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.523 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.523 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:01.523 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.523 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.523 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.524 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:18:01.524 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.524 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.524 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.524 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.524 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:18:01.524 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.524 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.524 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.524 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.524 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:18:01.524 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.524 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.524 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.524 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.525 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:18:01.525 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:18:01.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.525 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.525 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.525 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:18:01.525 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:18:01.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.525 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.525 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.525 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:18:01.525 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:18:01.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.525 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.526 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.526 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:18:01.526 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:18:01.526 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.526 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.526 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.526 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:18:01.526 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:18:01.526 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.526 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.526 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.526 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:18:01.526 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:18:01.526 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.526 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.527 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.527 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:18:01.527 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:18:01.527 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.527 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.527 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.527 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.527 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:01.527 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:18:01.527 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 85.520838114, 2024-09-14T06:18:01.527 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:18:01.527 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.527 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.528 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:01.528 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.528 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.528 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.528 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:18:01.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.528 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.528 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.528 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.528 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:18:01.528 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.528 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.529 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.529 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:18:01.529 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.529 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.529 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.529 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:18:01.529 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:18:01.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.529 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.529 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.529 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:18:01.529 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:18:01.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.530 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.530 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.530 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:18:01.530 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:18:01.530 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.530 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.530 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.530 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:18:01.530 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:18:01.530 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.530 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.530 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.530 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:18:01.530 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:18:01.530 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.531 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.531 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.531 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:18:01.531 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:18:01.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.531 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.531 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.531 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:18:01.531 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:18:01.531 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.531 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.531 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.531 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.532 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:01.532 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:18:01.532 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 85.520817006000001, 2024-09-14T06:18:01.532 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:18:01.532 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.532 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.532 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:01.532 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.532 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.532 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.532 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:18:01.532 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.532 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.533 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.533 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.533 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:18:01.533 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.533 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.533 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.533 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:18:01.533 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.533 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.533 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.533 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:18:01.534 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:18:01.534 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.534 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.534 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.534 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:18:01.534 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:18:01.534 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.534 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.534 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.534 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:18:01.534 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:18:01.534 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.534 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.535 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.535 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:18:01.535 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:18:01.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.535 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.535 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.535 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:18:01.535 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:18:01.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.535 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.535 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.535 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:18:01.535 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:18:01.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.536 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.536 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.536 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:18:01.536 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:18:01.536 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.536 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.536 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.536 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.536 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48234 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:01.536 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:18:01.536 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 51.479545713999997, 2024-09-14T06:18:01.536 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051730353999999999, 2024-09-14T06:18:01.537 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.537 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.537 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:01.537 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.537 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.537 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.537 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48234 2024-09-14T06:18:01.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.537 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.537 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.537 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.537 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:18:01.537 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.538 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.538 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.538 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:18:01.538 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.538 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.538 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.538 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032937+0000", 2024-09-14T06:18:01.538 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:18:01.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.538 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.538 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.538 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032940+0000", 2024-09-14T06:18:01.539 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0580000000000002e-06 2024-09-14T06:18:01.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.539 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.539 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.539 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032946+0000", 2024-09-14T06:18:01.539 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2960000000000004e-06 2024-09-14T06:18:01.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.539 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.539 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.539 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032951+0000", 2024-09-14T06:18:01.539 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.938e-06 2024-09-14T06:18:01.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.539 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.539 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.540 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074419+0000", 2024-09-14T06:18:01.540 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041467979000000002 2024-09-14T06:18:01.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.540 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.540 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.540 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074490+0000", 2024-09-14T06:18:01.540 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1324999999999999e-05 2024-09-14T06:18:01.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.540 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.540 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:01.540 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074859+0000", 2024-09-14T06:18:01.540 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036914899999999999 2024-09-14T06:18:01.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.541 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.541 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:01.541 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078464+0000", 2024-09-14T06:18:01.541 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036043170000000001 2024-09-14T06:18:01.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.541 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.541 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:01.541 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084547+0000", 2024-09-14T06:18:01.541 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006083097 2024-09-14T06:18:01.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.541 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.541 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:01.541 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084614+0000", 2024-09-14T06:18:01.542 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7509999999999996e-05 2024-09-14T06:18:01.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.542 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.542 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.542 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084667+0000", 2024-09-14T06:18:01.542 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:18:01.542 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.542 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.542 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.542 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.542 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48237 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:01.542 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:18:01.543 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 51.47946597, 2024-09-14T06:18:01.543 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051772830999999998, 2024-09-14T06:18:01.543 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.543 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.543 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:01.543 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.543 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.543 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.543 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48237 2024-09-14T06:18:01.543 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.543 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.543 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.543 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.543 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:18:01.544 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.544 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.544 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.544 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:18:01.544 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.544 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.544 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.544 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:18:01.544 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0499999999999999e-07 2024-09-14T06:18:01.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.544 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.545 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.545 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033018+0000", 2024-09-14T06:18:01.545 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0949999999999998e-06 2024-09-14T06:18:01.545 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.545 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.545 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.545 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033023+0000", 2024-09-14T06:18:01.545 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1620000000000001e-06 2024-09-14T06:18:01.545 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.545 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.545 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.545 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033026+0000", 2024-09-14T06:18:01.545 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0120000000000001e-06 2024-09-14T06:18:01.545 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.546 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.546 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.546 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075349+0000", 2024-09-14T06:18:01.546 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042323373999999997 2024-09-14T06:18:01.546 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.546 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.546 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.546 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075394+0000", 2024-09-14T06:18:01.546 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4861000000000003e-05 2024-09-14T06:18:01.546 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.547 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.547 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:01.547 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075578+0000", 2024-09-14T06:18:01.547 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00018414899999999999 2024-09-14T06:18:01.547 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.547 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.547 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:01.547 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078894+0000", 2024-09-14T06:18:01.547 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033158480000000001 2024-09-14T06:18:01.547 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.547 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.547 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:01.547 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084677+0000", 2024-09-14T06:18:01.547 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0057832120000000002 2024-09-14T06:18:01.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.548 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.548 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:01.548 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084752+0000", 2024-09-14T06:18:01.548 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5123000000000006e-05 2024-09-14T06:18:01.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.548 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.548 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.548 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084789+0000", 2024-09-14T06:18:01.548 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6690000000000003e-05 2024-09-14T06:18:01.548 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.548 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.548 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.549 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.549 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48243 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:01.549 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:18:01.549 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 51.479384844999998, 2024-09-14T06:18:01.549 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051780221000000001, 2024-09-14T06:18:01.549 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.549 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.549 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:01.549 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.549 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.549 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.549 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48243 2024-09-14T06:18:01.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.549 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.550 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.550 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.550 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:18:01.550 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.550 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.550 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.550 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:18:01.550 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.550 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.550 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.550 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:18:01.550 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:18:01.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.551 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.551 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.551 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033100+0000", 2024-09-14T06:18:01.551 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:18:01.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.551 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.551 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.551 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033108+0000", 2024-09-14T06:18:01.551 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1760000000000004e-06 2024-09-14T06:18:01.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.551 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.551 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.551 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033111+0000", 2024-09-14T06:18:01.551 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0790000000000001e-06 2024-09-14T06:18:01.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.551 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.552 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.552 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075903+0000", 2024-09-14T06:18:01.552 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042791942999999999 2024-09-14T06:18:01.552 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.552 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.552 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.552 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075987+0000", 2024-09-14T06:18:01.552 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4467000000000003e-05 2024-09-14T06:18:01.552 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.552 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.552 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:01.552 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.076259+0000", 2024-09-14T06:18:01.552 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027201900000000003 2024-09-14T06:18:01.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.553 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.553 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:01.553 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078947+0000", 2024-09-14T06:18:01.553 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026875879999999999 2024-09-14T06:18:01.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.553 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.553 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:01.553 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084797+0000", 2024-09-14T06:18:01.553 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0058500380000000001 2024-09-14T06:18:01.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.554 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.554 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:01.554 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084843+0000", 2024-09-14T06:18:01.554 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6298999999999998e-05 2024-09-14T06:18:01.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.554 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.554 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.554 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084877+0000", 2024-09-14T06:18:01.554 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4378999999999997e-05 2024-09-14T06:18:01.554 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.554 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.554 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.554 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.554 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48287 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:01.555 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:18:01.555 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 51.479345045000002, 2024-09-14T06:18:01.555 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051924706000000001, 2024-09-14T06:18:01.555 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.555 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.555 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:01.555 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.555 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.555 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.555 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48287 2024-09-14T06:18:01.555 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.555 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.555 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.555 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.555 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:18:01.556 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.556 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.556 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.556 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.556 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:18:01.556 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.556 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.556 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.556 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.556 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033138+0000", 2024-09-14T06:18:01.556 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6599999999999996e-07 2024-09-14T06:18:01.556 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.556 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.556 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.556 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033141+0000", 2024-09-14T06:18:01.557 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0879999999999999e-06 2024-09-14T06:18:01.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.557 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.557 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.557 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033147+0000", 2024-09-14T06:18:01.557 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2249999999999997e-06 2024-09-14T06:18:01.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.557 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.557 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.557 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033153+0000", 2024-09-14T06:18:01.557 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.066e-06 2024-09-14T06:18:01.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.557 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.557 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.557 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077790+0000", 2024-09-14T06:18:01.558 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.044637255000000001 2024-09-14T06:18:01.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.558 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.558 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.558 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077845+0000", 2024-09-14T06:18:01.558 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4410999999999998e-05 2024-09-14T06:18:01.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.558 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.558 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:01.558 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078062+0000", 2024-09-14T06:18:01.558 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021755599999999999 2024-09-14T06:18:01.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.558 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.558 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:01.558 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.080991+0000", 2024-09-14T06:18:01.558 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029282969999999998 2024-09-14T06:18:01.559 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.559 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.559 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:01.559 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084885+0000", 2024-09-14T06:18:01.559 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003894495 2024-09-14T06:18:01.559 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.559 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.559 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:01.559 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085020+0000", 2024-09-14T06:18:01.559 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013500799999999999 2024-09-14T06:18:01.559 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.559 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.559 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.560 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085062+0000", 2024-09-14T06:18:01.560 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1638999999999999e-05 2024-09-14T06:18:01.560 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.560 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.560 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.560 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.560 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48288 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:01.560 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:18:01.560 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 51.479321345999999, 2024-09-14T06:18:01.560 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051994018000000003, 2024-09-14T06:18:01.560 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.560 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.560 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:01.560 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.561 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.561 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.561 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48288 2024-09-14T06:18:01.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.561 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.561 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.561 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.561 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:18:01.561 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.561 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.561 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.561 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:18:01.561 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.562 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.562 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.562 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:18:01.562 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.89e-07 2024-09-14T06:18:01.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.562 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.562 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.562 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033164+0000", 2024-09-14T06:18:01.562 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.779e-06 2024-09-14T06:18:01.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.562 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.562 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.562 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033171+0000", 2024-09-14T06:18:01.562 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6279999999999999e-06 2024-09-14T06:18:01.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.563 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.563 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.563 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033176+0000", 2024-09-14T06:18:01.563 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1610000000000002e-06 2024-09-14T06:18:01.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.563 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.563 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.563 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078991+0000", 2024-09-14T06:18:01.563 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045815348999999998 2024-09-14T06:18:01.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.563 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.563 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.563 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079086+0000", 2024-09-14T06:18:01.563 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5282e-05 2024-09-14T06:18:01.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.564 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.564 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:01.564 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079397+0000", 2024-09-14T06:18:01.564 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031055999999999998 2024-09-14T06:18:01.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.564 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.564 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:01.564 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084432+0000", 2024-09-14T06:18:01.564 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050355599999999997 2024-09-14T06:18:01.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.564 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.564 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:01.564 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085073+0000", 2024-09-14T06:18:01.564 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064015199999999995 2024-09-14T06:18:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085120+0000", 2024-09-14T06:18:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7107000000000001e-05 2024-09-14T06:18:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085155+0000", 2024-09-14T06:18:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4950999999999998e-05 2024-09-14T06:18:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48416 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:18:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 51.477080934, 2024-09-14T06:18:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053690144000000002, 2024-09-14T06:18:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48416 2024-09-14T06:18:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:18:01.567 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.567 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.567 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.567 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.567 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:18:01.567 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.567 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.567 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.567 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.567 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035402+0000", 2024-09-14T06:18:01.567 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7999999999999995e-07 2024-09-14T06:18:01.567 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.567 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.567 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.567 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035405+0000", 2024-09-14T06:18:01.568 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3960000000000002e-06 2024-09-14T06:18:01.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.568 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.568 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.568 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035413+0000", 2024-09-14T06:18:01.568 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2209999999999997e-06 2024-09-14T06:18:01.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.568 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.568 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.568 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035420+0000", 2024-09-14T06:18:01.568 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1539999999999996e-06 2024-09-14T06:18:01.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.568 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.568 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.568 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081021+0000", 2024-09-14T06:18:01.569 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045600389999999998 2024-09-14T06:18:01.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.569 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.569 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.569 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081091+0000", 2024-09-14T06:18:01.569 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0325000000000002e-05 2024-09-14T06:18:01.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.569 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.569 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:01.569 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081434+0000", 2024-09-14T06:18:01.569 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000342519 2024-09-14T06:18:01.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.569 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.569 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:01.570 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085254+0000", 2024-09-14T06:18:01.570 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038201609999999999 2024-09-14T06:18:01.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.570 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.570 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:01.570 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088977+0000", 2024-09-14T06:18:01.570 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003722805 2024-09-14T06:18:01.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.570 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.570 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:01.570 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089048+0000", 2024-09-14T06:18:01.570 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1266999999999994e-05 2024-09-14T06:18:01.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.570 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.571 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.571 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089091+0000", 2024-09-14T06:18:01.571 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3325999999999998e-05 2024-09-14T06:18:01.571 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.571 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.571 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.571 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.571 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.571 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48417 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:01.571 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:18:01.571 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 51.477052264999998, 2024-09-14T06:18:01.572 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053805032000000003, 2024-09-14T06:18:01.572 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.572 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.572 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:01.572 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.572 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.572 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.573 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48417 2024-09-14T06:18:01.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.573 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.573 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.573 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.573 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:18:01.573 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.574 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.574 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.574 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:18:01.574 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.574 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.574 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.574 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:18:01.574 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9500000000000003e-07 2024-09-14T06:18:01.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.574 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.574 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.574 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035433+0000", 2024-09-14T06:18:01.574 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.796e-06 2024-09-14T06:18:01.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.575 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.575 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035442+0000", 2024-09-14T06:18:01.575 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7120000000000006e-06 2024-09-14T06:18:01.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.575 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.575 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035448+0000", 2024-09-14T06:18:01.575 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.015e-06 2024-09-14T06:18:01.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.575 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.575 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081855+0000", 2024-09-14T06:18:01.576 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.046407634000000003 2024-09-14T06:18:01.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.576 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.576 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081938+0000", 2024-09-14T06:18:01.576 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2687000000000003e-05 2024-09-14T06:18:01.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.576 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:01.576 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082313+0000", 2024-09-14T06:18:01.576 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000374872 2024-09-14T06:18:01.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.576 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:01.577 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087616+0000", 2024-09-14T06:18:01.577 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053032670000000004 2024-09-14T06:18:01.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.577 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:01.577 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089140+0000", 2024-09-14T06:18:01.577 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0015234350000000001 2024-09-14T06:18:01.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.577 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:01.577 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089184+0000", 2024-09-14T06:18:01.577 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4804e-05 2024-09-14T06:18:01.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.578 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.578 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089235+0000", 2024-09-14T06:18:01.578 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0315000000000002e-05 2024-09-14T06:18:01.578 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.578 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.578 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.578 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.578 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48418 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:01.578 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:18:01.578 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 51.477008126000001, 2024-09-14T06:18:01.578 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053936425000000003, 2024-09-14T06:18:01.578 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.578 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.579 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:01.579 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.579 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.579 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.579 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48418 2024-09-14T06:18:01.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.579 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.579 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.579 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.579 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:18:01.579 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.579 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.579 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.580 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:18:01.580 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.580 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.580 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035475+0000", 2024-09-14T06:18:01.580 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7000000000000003e-07 2024-09-14T06:18:01.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.580 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.580 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035479+0000", 2024-09-14T06:18:01.580 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2350000000000001e-06 2024-09-14T06:18:01.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.581 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.581 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035488+0000", 2024-09-14T06:18:01.581 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8820000000000007e-06 2024-09-14T06:18:01.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.581 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.581 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.581 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035494+0000", 2024-09-14T06:18:01.581 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3060000000000003e-06 2024-09-14T06:18:01.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.581 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.581 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.581 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082826+0000", 2024-09-14T06:18:01.581 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.047332008000000002 2024-09-14T06:18:01.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.582 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.582 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082883+0000", 2024-09-14T06:18:01.582 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7124999999999999e-05 2024-09-14T06:18:01.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.582 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:01.582 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.083385+0000", 2024-09-14T06:18:01.582 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050134099999999998 2024-09-14T06:18:01.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.582 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:01.582 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088888+0000", 2024-09-14T06:18:01.582 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005503127 2024-09-14T06:18:01.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.583 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:01.583 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089294+0000", 2024-09-14T06:18:01.583 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040669700000000003 2024-09-14T06:18:01.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.583 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:01.583 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089348+0000", 2024-09-14T06:18:01.583 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3729e-05 2024-09-14T06:18:01.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.583 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.584 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089410+0000", 2024-09-14T06:18:01.584 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2105000000000003e-05 2024-09-14T06:18:01.584 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.584 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.584 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.584 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48419 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:01.584 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:18:01.584 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 51.476930496000001, 2024-09-14T06:18:01.584 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054007545999999997, 2024-09-14T06:18:01.584 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.584 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.585 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:01.585 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.585 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.585 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.585 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48419 2024-09-14T06:18:01.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.585 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.585 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.585 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.585 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:18:01.585 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.585 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.585 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.586 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:18:01.586 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.586 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.586 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.586 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035552+0000", 2024-09-14T06:18:01.586 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4099999999999998e-07 2024-09-14T06:18:01.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.586 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.586 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.586 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035557+0000", 2024-09-14T06:18:01.586 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1819999999999997e-06 2024-09-14T06:18:01.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.586 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.587 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.587 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035566+0000", 2024-09-14T06:18:01.587 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1120000000000002e-06 2024-09-14T06:18:01.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.587 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.587 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035572+0000", 2024-09-14T06:18:01.587 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0430000000000004e-06 2024-09-14T06:18:01.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.587 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.587 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085304+0000", 2024-09-14T06:18:01.587 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049731707 2024-09-14T06:18:01.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.588 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.588 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085371+0000", 2024-09-14T06:18:01.588 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6771000000000002e-05 2024-09-14T06:18:01.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.588 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:01.588 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085601+0000", 2024-09-14T06:18:01.588 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022968000000000001 2024-09-14T06:18:01.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.588 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:01.588 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088893+0000", 2024-09-14T06:18:01.588 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032927780000000001 2024-09-14T06:18:01.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.589 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:01.589 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089467+0000", 2024-09-14T06:18:01.589 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057330399999999996 2024-09-14T06:18:01.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.589 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:01.589 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089520+0000", 2024-09-14T06:18:01.589 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3133000000000001e-05 2024-09-14T06:18:01.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.589 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.589 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089559+0000", 2024-09-14T06:18:01.590 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9195e-05 2024-09-14T06:18:01.590 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.590 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.590 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.590 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.590 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48420 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:01.590 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:18:01.590 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 51.476841266000001, 2024-09-14T06:18:01.590 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056401932000000002, 2024-09-14T06:18:01.590 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.590 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.590 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:01.590 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.591 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.591 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.591 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48420 2024-09-14T06:18:01.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.591 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.591 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.591 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.591 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:18:01.591 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.591 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.591 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.591 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:18:01.591 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035642+0000", 2024-09-14T06:18:01.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1519999999999999e-06 2024-09-14T06:18:01.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035647+0000", 2024-09-14T06:18:01.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2299999999999999e-06 2024-09-14T06:18:01.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035660+0000", 2024-09-14T06:18:01.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3166999999999999e-05 2024-09-14T06:18:01.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.593 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035666+0000", 2024-09-14T06:18:01.593 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1720000000000004e-06 2024-09-14T06:18:01.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.593 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086104+0000", 2024-09-14T06:18:01.593 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050437956999999999 2024-09-14T06:18:01.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.593 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086164+0000", 2024-09-14T06:18:01.593 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9592999999999997e-05 2024-09-14T06:18:01.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.594 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:01.594 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086402+0000", 2024-09-14T06:18:01.594 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000238314 2024-09-14T06:18:01.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.594 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:01.594 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089103+0000", 2024-09-14T06:18:01.594 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027011600000000002 2024-09-14T06:18:01.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.594 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:01.594 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091962+0000", 2024-09-14T06:18:01.595 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028581259999999999 2024-09-14T06:18:01.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.595 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:01.595 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092005+0000", 2024-09-14T06:18:01.595 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3111000000000001e-05 2024-09-14T06:18:01.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.595 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.595 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092043+0000", 2024-09-14T06:18:01.595 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7950000000000001e-05 2024-09-14T06:18:01.595 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.595 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.595 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.596 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.596 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48424 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:01.596 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:18:01.596 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 51.476577321000001, 2024-09-14T06:18:01.596 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056273173000000003, 2024-09-14T06:18:01.596 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.596 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.596 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:01.596 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.596 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.596 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.597 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48424 2024-09-14T06:18:01.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.597 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.597 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.597 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:18:01.597 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.597 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.597 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:18:01.597 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.598 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.598 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:18:01.598 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2900000000000003e-07 2024-09-14T06:18:01.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.598 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.598 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035909+0000", 2024-09-14T06:18:01.598 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6490000000000001e-06 2024-09-14T06:18:01.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.598 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.598 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035918+0000", 2024-09-14T06:18:01.598 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5089999999999999e-06 2024-09-14T06:18:01.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.599 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.599 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.599 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035924+0000", 2024-09-14T06:18:01.599 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7089999999999999e-06 2024-09-14T06:18:01.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.599 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.599 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.599 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087662+0000", 2024-09-14T06:18:01.599 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737621999999997 2024-09-14T06:18:01.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.599 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.599 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.599 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087744+0000", 2024-09-14T06:18:01.599 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2174000000000003e-05 2024-09-14T06:18:01.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.600 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.600 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:01.600 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088106+0000", 2024-09-14T06:18:01.600 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000361947 2024-09-14T06:18:01.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.600 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.600 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:01.600 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092053+0000", 2024-09-14T06:18:01.600 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0039473080000000001 2024-09-14T06:18:01.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.600 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.600 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:01.600 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092087+0000", 2024-09-14T06:18:01.601 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4153000000000003e-05 2024-09-14T06:18:01.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.601 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.601 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:01.601 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092138+0000", 2024-09-14T06:18:01.601 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0804999999999999e-05 2024-09-14T06:18:01.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.601 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.601 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.601 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092178+0000", 2024-09-14T06:18:01.601 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9668e-05 2024-09-14T06:18:01.601 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.601 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.601 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.602 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.602 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48508 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:01.602 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:18:01.602 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 51.47150224, 2024-09-14T06:18:01.602 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054906017000000001, 2024-09-14T06:18:01.602 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.602 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.602 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:01.602 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.602 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.602 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.602 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48508 2024-09-14T06:18:01.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.603 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.603 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.603 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.603 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:18:01.603 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.603 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.603 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.603 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.603 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:18:01.603 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.603 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.603 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.603 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.603 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040981+0000", 2024-09-14T06:18:01.604 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7700000000000004e-07 2024-09-14T06:18:01.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.604 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.604 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.604 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040984+0000", 2024-09-14T06:18:01.604 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1480000000000002e-06 2024-09-14T06:18:01.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.604 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.604 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.604 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040991+0000", 2024-09-14T06:18:01.604 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4580000000000004e-06 2024-09-14T06:18:01.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.604 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.604 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.605 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040997+0000", 2024-09-14T06:18:01.605 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7799999999999997e-06 2024-09-14T06:18:01.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.605 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.605 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.605 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089601+0000", 2024-09-14T06:18:01.605 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048604264000000001 2024-09-14T06:18:01.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.605 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.605 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.605 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089708+0000", 2024-09-14T06:18:01.605 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010681200000000001 2024-09-14T06:18:01.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090384+0000", 2024-09-14T06:18:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00067557499999999998 2024-09-14T06:18:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093879+0000", 2024-09-14T06:18:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034954679999999998 2024-09-14T06:18:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095764+0000", 2024-09-14T06:18:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018848070000000001 2024-09-14T06:18:01.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.607 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.607 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:01.607 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095837+0000", 2024-09-14T06:18:01.607 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3433e-05 2024-09-14T06:18:01.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.607 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.607 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.607 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095886+0000", 2024-09-14T06:18:01.607 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8495000000000002e-05 2024-09-14T06:18:01.607 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.607 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.607 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.608 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.608 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48537 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:01.608 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:18:01.608 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 51.470138470000002, 2024-09-14T06:18:01.608 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053681833999999998, 2024-09-14T06:18:01.608 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.608 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.608 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:01.608 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.608 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.608 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.608 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48537 2024-09-14T06:18:01.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.608 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.609 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.609 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:18:01.609 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.609 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.609 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:18:01.609 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.609 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.609 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:18:01.610 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2299999999999999e-07 2024-09-14T06:18:01.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.610 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.610 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042350+0000", 2024-09-14T06:18:01.610 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5690000000000004e-06 2024-09-14T06:18:01.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.610 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.610 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042361+0000", 2024-09-14T06:18:01.610 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1337e-05 2024-09-14T06:18:01.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.610 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.611 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042369+0000", 2024-09-14T06:18:01.611 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8359999999999992e-06 2024-09-14T06:18:01.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.611 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.611 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.611 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090914+0000", 2024-09-14T06:18:01.611 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048544945999999999 2024-09-14T06:18:01.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.611 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.611 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.611 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090995+0000", 2024-09-14T06:18:01.611 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0548000000000002e-05 2024-09-14T06:18:01.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.612 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.612 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:01.612 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091508+0000", 2024-09-14T06:18:01.612 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000513723 2024-09-14T06:18:01.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.612 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.612 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:01.612 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093900+0000", 2024-09-14T06:18:01.612 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023920429999999999 2024-09-14T06:18:01.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.612 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.612 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:01.612 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095944+0000", 2024-09-14T06:18:01.613 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020437379999999998 2024-09-14T06:18:01.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.613 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.613 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:01.613 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095987+0000", 2024-09-14T06:18:01.613 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2521999999999997e-05 2024-09-14T06:18:01.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.613 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.613 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.613 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.096025+0000", 2024-09-14T06:18:01.613 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8649000000000002e-05 2024-09-14T06:18:01.613 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.613 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.613 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.614 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.614 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:63810 3.1d 3.c887d99d (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-14T06:18:01.614 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:18:01.614 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 27.932380584000001, 2024-09-14T06:18:01.614 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.61346978900000004, 2024-09-14T06:18:01.614 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.614 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.614 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:18:01.614 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.614 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.614 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.614 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 63810 2024-09-14T06:18:01.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.615 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.615 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.615 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:18:01.615 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.615 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.615 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:18:01.615 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.615 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.615 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580104+0000", 2024-09-14T06:18:01.616 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4650000000000001e-06 2024-09-14T06:18:01.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.616 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.616 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580112+0000", 2024-09-14T06:18:01.616 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9219999999999997e-06 2024-09-14T06:18:01.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.616 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.616 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580130+0000", 2024-09-14T06:18:01.616 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8128999999999999e-05 2024-09-14T06:18:01.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.616 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.617 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580140+0000", 2024-09-14T06:18:01.617 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0285000000000001e-05 2024-09-14T06:18:01.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.617 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.617 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580201+0000", 2024-09-14T06:18:01.617 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0517999999999999e-05 2024-09-14T06:18:01.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.617 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:18:01.617 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580209+0000", 2024-09-14T06:18:01.617 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6430000000000001e-06 2024-09-14T06:18:01.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.617 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.618 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:34.193554+0000", 2024-09-14T06:18:01.618 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.61334482899999998 2024-09-14T06:18:01.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.618 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.618 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.618 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:34.193571+0000", 2024-09-14T06:18:01.618 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6997999999999999e-05 2024-09-14T06:18:01.618 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.618 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.618 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.618 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.618 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:63832 3.1d 3:b99be113:::rbd_object_map.11525ae33f05:head [read 0~52] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T06:18:01.619 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:18:01.619 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 24.274384198, 2024-09-14T06:18:01.619 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.99066972200000003, 2024-09-14T06:18:01.619 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:01.619 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:01.619 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:01.619 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:01.619 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:01.619 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:01.619 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 63832 2024-09-14T06:18:01.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.619 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:01.619 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.619 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:01.620 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:18:01.620 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.620 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:01.620 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:18:01.620 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:01.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.620 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:01.620 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238100+0000", 2024-09-14T06:18:01.620 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0849999999999999e-06 2024-09-14T06:18:01.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.621 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:01.621 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238108+0000", 2024-09-14T06:18:01.621 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.089e-06 2024-09-14T06:18:01.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.621 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:01.621 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238123+0000", 2024-09-14T06:18:01.621 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5418e-05 2024-09-14T06:18:01.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.621 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:01.621 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238136+0000", 2024-09-14T06:18:01.621 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2204e-05 2024-09-14T06:18:01.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.622 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.622 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238913+0000", 2024-09-14T06:18:01.622 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00077720399999999998 2024-09-14T06:18:01.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.622 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:18:01.622 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238934+0000", 2024-09-14T06:18:01.622 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1311999999999999e-05 2024-09-14T06:18:01.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.622 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:01.622 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228134+0000", 2024-09-14T06:18:01.622 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.98919991200000001 2024-09-14T06:18:01.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.623 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.623 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:01.623 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228261+0000", 2024-09-14T06:18:01.623 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000126742 2024-09-14T06:18:01.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:01.623 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:01.623 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:01.623 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228768+0000", 2024-09-14T06:18:01.623 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050675600000000003 2024-09-14T06:18:01.623 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.623 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.623 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.623 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:01.623 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:01.623 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:01.625 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:18:01.625 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:01.623+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:01.727 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:01.721+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:01.757 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:01.757 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:01.757 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:18:01.757 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:01.771 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:18:01.827 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:01.823+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:01.892 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:01.892 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:01.892 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:18:01.892 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:18:01.892 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:01.905 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:18:01.928 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:01.925+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:02.029 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:02.027+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:02.072 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:18:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:18:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:18:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13229 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:18:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.969905768999993, 2024-09-14T06:18:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081798868999999996, 2024-09-14T06:18:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13229 2024-09-14T06:18:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:18:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:18:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098710+0000", 2024-09-14T06:18:02.075 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8199999999999999e-07 2024-09-14T06:18:02.075 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.075 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.075 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.075 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098714+0000", 2024-09-14T06:18:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:18:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098727+0000", 2024-09-14T06:18:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2520999999999999e-05 2024-09-14T06:18:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098749+0000", 2024-09-14T06:18:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2773999999999999e-05 2024-09-14T06:18:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160234+0000", 2024-09-14T06:18:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.061484430999999999 2024-09-14T06:18:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160281+0000", 2024-09-14T06:18:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7437e-05 2024-09-14T06:18:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160569+0000", 2024-09-14T06:18:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028788700000000002 2024-09-14T06:18:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163121+0000", 2024-09-14T06:18:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025514090000000001 2024-09-14T06:18:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180410+0000", 2024-09-14T06:18:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.017289098999999999 2024-09-14T06:18:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180472+0000", 2024-09-14T06:18:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2547999999999998e-05 2024-09-14T06:18:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180508+0000", 2024-09-14T06:18:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5534000000000002e-05 2024-09-14T06:18:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13230 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:18:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.969824567000003, 2024-09-14T06:18:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081851558000000005, 2024-09-14T06:18:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13230 2024-09-14T06:18:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:18:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:18:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098791+0000", 2024-09-14T06:18:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8800000000000001e-07 2024-09-14T06:18:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098795+0000", 2024-09-14T06:18:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.391e-06 2024-09-14T06:18:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098804+0000", 2024-09-14T06:18:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3440000000000001e-06 2024-09-14T06:18:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098810+0000", 2024-09-14T06:18:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2339999999999999e-06 2024-09-14T06:18:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160930+0000", 2024-09-14T06:18:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062120018999999999 2024-09-14T06:18:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161017+0000", 2024-09-14T06:18:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6868999999999998e-05 2024-09-14T06:18:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161300+0000", 2024-09-14T06:18:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028349499999999998 2024-09-14T06:18:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164612+0000", 2024-09-14T06:18:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033119680000000002 2024-09-14T06:18:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180559+0000", 2024-09-14T06:18:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015946612999999998 2024-09-14T06:18:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180605+0000", 2024-09-14T06:18:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5890000000000003e-05 2024-09-14T06:18:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180642+0000", 2024-09-14T06:18:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:18:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13231 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:18:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.969562152999998, 2024-09-14T06:18:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081750685000000003, 2024-09-14T06:18:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13231 2024-09-14T06:18:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:18:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:18:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099053+0000", 2024-09-14T06:18:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:18:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099059+0000", 2024-09-14T06:18:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1789999999999999e-06 2024-09-14T06:18:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099072+0000", 2024-09-14T06:18:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3179999999999999e-05 2024-09-14T06:18:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099080+0000", 2024-09-14T06:18:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9030000000000004e-06 2024-09-14T06:18:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161681+0000", 2024-09-14T06:18:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062601641999999999 2024-09-14T06:18:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161766+0000", 2024-09-14T06:18:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4909999999999998e-05 2024-09-14T06:18:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162109+0000", 2024-09-14T06:18:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00034264899999999999 2024-09-14T06:18:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164615+0000", 2024-09-14T06:18:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025061979999999998 2024-09-14T06:18:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180718+0000", 2024-09-14T06:18:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016103240000000001 2024-09-14T06:18:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180768+0000", 2024-09-14T06:18:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9781999999999999e-05 2024-09-14T06:18:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180803+0000", 2024-09-14T06:18:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5021999999999998e-05 2024-09-14T06:18:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13232 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:18:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.969520379000002, 2024-09-14T06:18:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081861981, 2024-09-14T06:18:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13232 2024-09-14T06:18:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:18:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:18:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099095+0000", 2024-09-14T06:18:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:18:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099098+0000", 2024-09-14T06:18:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5420000000000001e-06 2024-09-14T06:18:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099106+0000", 2024-09-14T06:18:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.824e-06 2024-09-14T06:18:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099112+0000", 2024-09-14T06:18:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9789999999999998e-06 2024-09-14T06:18:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162576+0000", 2024-09-14T06:18:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.063463723999999999 2024-09-14T06:18:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162627+0000", 2024-09-14T06:18:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1530000000000003e-05 2024-09-14T06:18:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162854+0000", 2024-09-14T06:18:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000226159 2024-09-14T06:18:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166090+0000", 2024-09-14T06:18:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032362319999999999 2024-09-14T06:18:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180877+0000", 2024-09-14T06:18:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014786726 2024-09-14T06:18:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180920+0000", 2024-09-14T06:18:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3427000000000003e-05 2024-09-14T06:18:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180956+0000", 2024-09-14T06:18:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6195999999999997e-05 2024-09-14T06:18:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13233 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:18:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.969495116000004, 2024-09-14T06:18:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081924860000000002, 2024-09-14T06:18:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13233 2024-09-14T06:18:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:18:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:18:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099120+0000", 2024-09-14T06:18:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:18:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099122+0000", 2024-09-14T06:18:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:18:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099126+0000", 2024-09-14T06:18:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2540000000000003e-06 2024-09-14T06:18:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099130+0000", 2024-09-14T06:18:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4060000000000001e-06 2024-09-14T06:18:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163169+0000", 2024-09-14T06:18:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064039415000000002 2024-09-14T06:18:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163260+0000", 2024-09-14T06:18:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1038000000000005e-05 2024-09-14T06:18:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163640+0000", 2024-09-14T06:18:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037954699999999999 2024-09-14T06:18:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166093+0000", 2024-09-14T06:18:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024533229999999999 2024-09-14T06:18:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180993+0000", 2024-09-14T06:18:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01489997 2024-09-14T06:18:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181020+0000", 2024-09-14T06:18:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6806999999999999e-05 2024-09-14T06:18:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181044+0000", 2024-09-14T06:18:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4405000000000001e-05 2024-09-14T06:18:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13234 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:18:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.969452911000005, 2024-09-14T06:18:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082015597999999995, 2024-09-14T06:18:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13234 2024-09-14T06:18:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:18:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:18:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:18:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0999999999999998e-07 2024-09-14T06:18:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099166+0000", 2024-09-14T06:18:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:18:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099171+0000", 2024-09-14T06:18:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6949999999999997e-06 2024-09-14T06:18:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099175+0000", 2024-09-14T06:18:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9029999999999997e-06 2024-09-14T06:18:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164063+0000", 2024-09-14T06:18:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064887608999999999 2024-09-14T06:18:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164111+0000", 2024-09-14T06:18:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7648999999999997e-05 2024-09-14T06:18:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164351+0000", 2024-09-14T06:18:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024072899999999999 2024-09-14T06:18:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167561+0000", 2024-09-14T06:18:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032094179999999999 2024-09-14T06:18:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181093+0000", 2024-09-14T06:18:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013532245 2024-09-14T06:18:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181139+0000", 2024-09-14T06:18:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6483999999999997e-05 2024-09-14T06:18:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181177+0000", 2024-09-14T06:18:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7861000000000002e-05 2024-09-14T06:18:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13235 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:18:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.969345637000004, 2024-09-14T06:18:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081988088000000001, 2024-09-14T06:18:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.112 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.112 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13235 2024-09-14T06:18:02.112 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.112 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.112 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.112 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.112 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:18:02.112 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.112 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.112 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.112 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.113 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:18:02.113 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.113 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.113 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.113 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099270+0000", 2024-09-14T06:18:02.113 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7400000000000003e-07 2024-09-14T06:18:02.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.113 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.113 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.113 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099276+0000", 2024-09-14T06:18:02.113 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:18:02.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099284+0000", 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7979999999999995e-06 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099289+0000", 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164664+0000", 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065374901999999999 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.114 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.115 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164755+0000", 2024-09-14T06:18:02.115 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0790000000000003e-05 2024-09-14T06:18:02.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.115 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.115 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.115 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165143+0000", 2024-09-14T06:18:02.115 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000387904 2024-09-14T06:18:02.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.115 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.115 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.115 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167564+0000", 2024-09-14T06:18:02.115 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024208989999999998 2024-09-14T06:18:02.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.116 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.116 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.116 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181212+0000", 2024-09-14T06:18:02.116 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013648664 2024-09-14T06:18:02.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.116 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.116 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.116 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181237+0000", 2024-09-14T06:18:02.116 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5177999999999998e-05 2024-09-14T06:18:02.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.116 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.116 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.116 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181257+0000", 2024-09-14T06:18:02.117 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9660000000000002e-05 2024-09-14T06:18:02.117 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.117 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.117 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.117 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.117 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13236 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.117 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:18:02.117 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.969238817999994, 2024-09-14T06:18:02.117 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082010939000000005, 2024-09-14T06:18:02.117 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.117 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.117 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.117 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.118 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.118 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.118 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13236 2024-09-14T06:18:02.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.118 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.118 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:18:02.118 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.118 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:18:02.118 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.119 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.119 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.119 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:18:02.119 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.61e-07 2024-09-14T06:18:02.119 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.119 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.119 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.119 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099382+0000", 2024-09-14T06:18:02.119 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3589999999999999e-06 2024-09-14T06:18:02.119 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.119 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.119 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.119 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099388+0000", 2024-09-14T06:18:02.119 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9619999999999998e-06 2024-09-14T06:18:02.119 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.119 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.120 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.120 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099393+0000", 2024-09-14T06:18:02.120 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3410000000000002e-06 2024-09-14T06:18:02.120 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.120 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.120 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.120 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165553+0000", 2024-09-14T06:18:02.120 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066159604999999996 2024-09-14T06:18:02.120 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.120 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.120 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.120 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165603+0000", 2024-09-14T06:18:02.120 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0427999999999999e-05 2024-09-14T06:18:02.120 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.121 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.121 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165827+0000", 2024-09-14T06:18:02.121 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022358299999999999 2024-09-14T06:18:02.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.121 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.121 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177005+0000", 2024-09-14T06:18:02.121 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011178044 2024-09-14T06:18:02.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.121 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.121 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181302+0000", 2024-09-14T06:18:02.121 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042970669999999999 2024-09-14T06:18:02.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.122 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.122 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.122 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181349+0000", 2024-09-14T06:18:02.122 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7287999999999999e-05 2024-09-14T06:18:02.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.122 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.122 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.122 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181387+0000", 2024-09-14T06:18:02.122 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7601000000000001e-05 2024-09-14T06:18:02.122 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.122 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.122 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.123 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.123 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13237 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.123 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:18:02.123 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.969080671, 2024-09-14T06:18:02.123 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081927299999999995, 2024-09-14T06:18:02.123 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.123 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.123 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.123 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.123 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.123 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.123 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13237 2024-09-14T06:18:02.123 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.123 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.124 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.124 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.124 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:18:02.124 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.124 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.124 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.124 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.124 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:18:02.124 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.124 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.124 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.124 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.124 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099535+0000", 2024-09-14T06:18:02.124 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:18:02.124 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.125 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.125 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.125 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099538+0000", 2024-09-14T06:18:02.125 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:18:02.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.125 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.125 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.125 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099545+0000", 2024-09-14T06:18:02.125 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:18:02.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.125 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.125 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.125 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099551+0000", 2024-09-14T06:18:02.126 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1770000000000003e-06 2024-09-14T06:18:02.126 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.126 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.126 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.126 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166131+0000", 2024-09-14T06:18:02.126 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066580499000000001 2024-09-14T06:18:02.126 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.126 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.126 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.126 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166221+0000", 2024-09-14T06:18:02.126 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0000000000000006e-05 2024-09-14T06:18:02.126 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.127 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.127 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.127 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166622+0000", 2024-09-14T06:18:02.127 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040029799999999999 2024-09-14T06:18:02.127 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.127 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.127 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.127 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177008+0000", 2024-09-14T06:18:02.127 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010386772000000001 2024-09-14T06:18:02.127 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.127 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.127 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.127 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181420+0000", 2024-09-14T06:18:02.127 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044119190000000003 2024-09-14T06:18:02.128 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.128 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.128 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.128 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181443+0000", 2024-09-14T06:18:02.128 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2603999999999999e-05 2024-09-14T06:18:02.128 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.128 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.128 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.128 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181461+0000", 2024-09-14T06:18:02.128 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8405000000000001e-05 2024-09-14T06:18:02.129 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.129 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.129 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.129 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.129 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.129 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13238 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.129 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:18:02.129 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.969044362999995, 2024-09-14T06:18:02.129 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082014927000000001, 2024-09-14T06:18:02.129 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.129 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.129 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.129 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.129 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.130 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.130 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13238 2024-09-14T06:18:02.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.130 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.130 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.130 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.130 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:18:02.130 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.130 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.130 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.130 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:18:02.130 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.131 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.131 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.131 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099571+0000", 2024-09-14T06:18:02.131 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9699999999999996e-07 2024-09-14T06:18:02.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.131 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.131 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.131 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099577+0000", 2024-09-14T06:18:02.131 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2890000000000003e-06 2024-09-14T06:18:02.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.131 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.131 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.131 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099587+0000", 2024-09-14T06:18:02.132 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0002e-05 2024-09-14T06:18:02.132 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.132 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.132 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.132 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099593+0000", 2024-09-14T06:18:02.132 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4539999999999998e-06 2024-09-14T06:18:02.132 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.133 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.133 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.133 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167024+0000", 2024-09-14T06:18:02.133 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067431480000000002 2024-09-14T06:18:02.133 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.133 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.133 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.133 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167074+0000", 2024-09-14T06:18:02.133 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9996e-05 2024-09-14T06:18:02.133 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.133 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.134 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.134 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167288+0000", 2024-09-14T06:18:02.134 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021410900000000001 2024-09-14T06:18:02.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.134 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.134 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.134 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177997+0000", 2024-09-14T06:18:02.134 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010708756 2024-09-14T06:18:02.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.134 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.134 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.134 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181508+0000", 2024-09-14T06:18:02.134 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003511363 2024-09-14T06:18:02.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.135 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.135 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.135 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181552+0000", 2024-09-14T06:18:02.135 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3609999999999998e-05 2024-09-14T06:18:02.135 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.135 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.135 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.135 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181585+0000", 2024-09-14T06:18:02.135 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3271e-05 2024-09-14T06:18:02.135 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.135 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.135 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.135 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.136 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.136 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13239 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.136 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:18:02.136 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.968952544999993, 2024-09-14T06:18:02.136 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081998772999999997, 2024-09-14T06:18:02.136 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.136 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.136 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.136 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.136 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.136 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.136 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13239 2024-09-14T06:18:02.136 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.136 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.137 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.137 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.137 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:18:02.137 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.137 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.137 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.137 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.137 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:18:02.137 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.137 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.137 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.137 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.137 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099663+0000", 2024-09-14T06:18:02.138 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6100000000000001e-07 2024-09-14T06:18:02.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.138 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.138 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.138 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099666+0000", 2024-09-14T06:18:02.138 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.056e-06 2024-09-14T06:18:02.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.138 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.138 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.138 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099672+0000", 2024-09-14T06:18:02.138 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8120000000000002e-06 2024-09-14T06:18:02.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.138 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.138 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.139 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099676+0000", 2024-09-14T06:18:02.139 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:18:02.139 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.139 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.139 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.139 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167602+0000", 2024-09-14T06:18:02.139 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067925867000000001 2024-09-14T06:18:02.139 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.139 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.139 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.139 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167693+0000", 2024-09-14T06:18:02.139 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0736000000000007e-05 2024-09-14T06:18:02.139 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.139 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.140 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.140 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168065+0000", 2024-09-14T06:18:02.140 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037245399999999998 2024-09-14T06:18:02.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.140 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.140 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.140 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178003+0000", 2024-09-14T06:18:02.140 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0099372360000000003 2024-09-14T06:18:02.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.140 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.140 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.140 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181621+0000", 2024-09-14T06:18:02.140 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036178489999999998 2024-09-14T06:18:02.141 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.141 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.141 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.141 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181643+0000", 2024-09-14T06:18:02.141 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2155e-05 2024-09-14T06:18:02.141 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.141 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.141 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.141 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181661+0000", 2024-09-14T06:18:02.141 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8147000000000001e-05 2024-09-14T06:18:02.141 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.141 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.141 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.141 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.142 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.142 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13240 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.142 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:18:02.142 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.968833451999998, 2024-09-14T06:18:02.142 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081942607000000001, 2024-09-14T06:18:02.142 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.142 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.142 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.142 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.142 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.143 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.143 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13240 2024-09-14T06:18:02.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.143 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.143 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.143 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.143 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:18:02.144 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.144 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.144 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.144 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.144 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:18:02.144 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.145 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.145 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.145 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099782+0000", 2024-09-14T06:18:02.145 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3900000000000004e-07 2024-09-14T06:18:02.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.145 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.146 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.146 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099787+0000", 2024-09-14T06:18:02.146 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6189999999999999e-06 2024-09-14T06:18:02.146 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.146 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.146 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.147 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099793+0000", 2024-09-14T06:18:02.147 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.998e-06 2024-09-14T06:18:02.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.147 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.147 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.147 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099797+0000", 2024-09-14T06:18:02.147 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8449999999999996e-06 2024-09-14T06:18:02.148 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.148 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.148 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.148 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168494+0000", 2024-09-14T06:18:02.149 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.068696500999999993 2024-09-14T06:18:02.149 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.149 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.149 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.149 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168590+0000", 2024-09-14T06:18:02.149 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6733999999999999e-05 2024-09-14T06:18:02.150 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.150 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.150 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.150 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168973+0000", 2024-09-14T06:18:02.150 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038229399999999998 2024-09-14T06:18:02.151 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.151 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.151 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.151 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178013+0000", 2024-09-14T06:18:02.151 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0090397199999999994 2024-09-14T06:18:02.151 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.152 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.152 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.152 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181683+0000", 2024-09-14T06:18:02.152 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036705819999999999 2024-09-14T06:18:02.152 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.153 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.153 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.153 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181705+0000", 2024-09-14T06:18:02.153 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1551000000000001e-05 2024-09-14T06:18:02.153 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.153 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.154 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.154 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181724+0000", 2024-09-14T06:18:02.154 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9123999999999999e-05 2024-09-14T06:18:02.154 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.154 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.154 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.155 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.155 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.155 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13243 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.155 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:18:02.155 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.968299721999998, 2024-09-14T06:18:02.155 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081767815999999993, 2024-09-14T06:18:02.156 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.156 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.156 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.156 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.156 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.156 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.157 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13243 2024-09-14T06:18:02.157 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.157 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.157 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.157 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.158 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:18:02.158 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.158 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.158 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.158 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:18:02.159 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.159 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.159 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.159 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:18:02.159 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1e-07 2024-09-14T06:18:02.160 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.160 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.160 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.160 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100317+0000", 2024-09-14T06:18:02.160 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.092e-06 2024-09-14T06:18:02.160 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.161 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.161 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.161 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100321+0000", 2024-09-14T06:18:02.161 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7809999999999999e-06 2024-09-14T06:18:02.161 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.161 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.162 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.162 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100324+0000", 2024-09-14T06:18:02.162 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.117e-06 2024-09-14T06:18:02.162 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.162 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.162 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.163 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171113+0000", 2024-09-14T06:18:02.163 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.070788816000000004 2024-09-14T06:18:02.163 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.163 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.163 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.163 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171202+0000", 2024-09-14T06:18:02.163 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8843000000000003e-05 2024-09-14T06:18:02.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.164 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.164 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.164 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171584+0000", 2024-09-14T06:18:02.164 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038231099999999998 2024-09-14T06:18:02.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.165 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.165 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.165 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178101+0000", 2024-09-14T06:18:02.165 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0065168329999999997 2024-09-14T06:18:02.165 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.166 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.166 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.166 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182041+0000", 2024-09-14T06:18:02.167 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039401810000000001 2024-09-14T06:18:02.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.167 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.167 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.167 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182065+0000", 2024-09-14T06:18:02.168 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3623000000000001e-05 2024-09-14T06:18:02.168 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.168 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.168 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.168 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182083+0000", 2024-09-14T06:18:02.168 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7909000000000001e-05 2024-09-14T06:18:02.169 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.169 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.169 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.169 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.169 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.170 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13244 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.170 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:18:02.170 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.968284901999994, 2024-09-14T06:18:02.170 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081837800000000002, 2024-09-14T06:18:02.170 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.171 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.171 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.171 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.171 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.171 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.171 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13244 2024-09-14T06:18:02.171 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.172 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.172 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.172 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.172 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:18:02.172 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.172 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.173 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.173 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.173 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:18:02.173 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.173 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.174 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.174 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:18:02.174 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:18:02.174 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.174 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.174 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.175 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100337+0000", 2024-09-14T06:18:02.175 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.939e-06 2024-09-14T06:18:02.175 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.175 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.175 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.175 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100341+0000", 2024-09-14T06:18:02.176 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2320000000000003e-06 2024-09-14T06:18:02.176 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.176 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.176 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.176 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100344+0000", 2024-09-14T06:18:02.176 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0759999999999999e-06 2024-09-14T06:18:02.177 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.177 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.177 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.177 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171996+0000", 2024-09-14T06:18:02.177 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.071651919999999994 2024-09-14T06:18:02.177 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.178 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.178 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.178 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172079+0000", 2024-09-14T06:18:02.178 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2579999999999999e-05 2024-09-14T06:18:02.178 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.178 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.178 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.179 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172457+0000", 2024-09-14T06:18:02.179 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037785100000000001 2024-09-14T06:18:02.179 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.179 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.179 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.179 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178104+0000", 2024-09-14T06:18:02.180 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056471730000000001 2024-09-14T06:18:02.180 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.180 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.180 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.180 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182123+0000", 2024-09-14T06:18:02.180 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040190019999999998 2024-09-14T06:18:02.181 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.181 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.181 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.181 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182147+0000", 2024-09-14T06:18:02.181 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4241000000000001e-05 2024-09-14T06:18:02.181 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.182 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.182 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.182 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182168+0000", 2024-09-14T06:18:02.182 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0492999999999998e-05 2024-09-14T06:18:02.182 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.182 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.183 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.183 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.183 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.183 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13245 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.183 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:18:02.183 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.968265395000003, 2024-09-14T06:18:02.183 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081893899000000006, 2024-09-14T06:18:02.184 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.184 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.184 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.184 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.184 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.184 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.185 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13245 2024-09-14T06:18:02.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.185 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.185 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.185 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.185 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:18:02.185 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:18:02.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100350+0000", 2024-09-14T06:18:02.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5499999999999999e-07 2024-09-14T06:18:02.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100353+0000", 2024-09-14T06:18:02.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:18:02.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100357+0000", 2024-09-14T06:18:02.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8500000000000004e-06 2024-09-14T06:18:02.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.188 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100360+0000", 2024-09-14T06:18:02.188 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8700000000000001e-06 2024-09-14T06:18:02.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.188 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.188 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.188 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172862+0000", 2024-09-14T06:18:02.188 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.072501992000000001 2024-09-14T06:18:02.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.188 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.188 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.188 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172950+0000", 2024-09-14T06:18:02.188 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8988999999999998e-05 2024-09-14T06:18:02.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.189 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.189 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.189 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173324+0000", 2024-09-14T06:18:02.189 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037366199999999997 2024-09-14T06:18:02.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.189 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.189 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.189 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178107+0000", 2024-09-14T06:18:02.189 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047830520000000003 2024-09-14T06:18:02.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.189 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.189 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.190 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182202+0000", 2024-09-14T06:18:02.190 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040949280000000003 2024-09-14T06:18:02.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.190 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.190 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182226+0000", 2024-09-14T06:18:02.190 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3535000000000001e-05 2024-09-14T06:18:02.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.190 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.190 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182243+0000", 2024-09-14T06:18:02.190 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7445e-05 2024-09-14T06:18:02.190 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.190 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.191 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.191 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13246 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.191 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:18:02.191 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.968251073000005, 2024-09-14T06:18:02.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081962276000000001, 2024-09-14T06:18:02.191 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.191 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.191 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.191 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.191 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.191 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.192 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13246 2024-09-14T06:18:02.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.192 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:18:02.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:18:02.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.193 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:18:02.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0800000000000001e-07 2024-09-14T06:18:02.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.193 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.193 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100367+0000", 2024-09-14T06:18:02.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8439999999999998e-06 2024-09-14T06:18:02.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.193 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.193 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100372+0000", 2024-09-14T06:18:02.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4009999999999996e-06 2024-09-14T06:18:02.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.194 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.194 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100378+0000", 2024-09-14T06:18:02.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7180000000000001e-06 2024-09-14T06:18:02.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.194 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.194 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173780+0000", 2024-09-14T06:18:02.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.073401991 2024-09-14T06:18:02.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.194 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.194 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173879+0000", 2024-09-14T06:18:02.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9449000000000003e-05 2024-09-14T06:18:02.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174240+0000", 2024-09-14T06:18:02.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036118100000000002 2024-09-14T06:18:02.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178109+0000", 2024-09-14T06:18:02.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038690619999999999 2024-09-14T06:18:02.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182281+0000", 2024-09-14T06:18:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041715900000000002 2024-09-14T06:18:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182306+0000", 2024-09-14T06:18:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4477e-05 2024-09-14T06:18:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182326+0000", 2024-09-14T06:18:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0254999999999998e-05 2024-09-14T06:18:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13247 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:18:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.968153896000004, 2024-09-14T06:18:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966593000000004, 2024-09-14T06:18:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13247 2024-09-14T06:18:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:18:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:18:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100462+0000", 2024-09-14T06:18:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9800000000000002e-07 2024-09-14T06:18:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100466+0000", 2024-09-14T06:18:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4039999999999997e-06 2024-09-14T06:18:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100475+0000", 2024-09-14T06:18:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3680000000000008e-06 2024-09-14T06:18:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100482+0000", 2024-09-14T06:18:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8630000000000002e-06 2024-09-14T06:18:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174669+0000", 2024-09-14T06:18:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074186849999999999 2024-09-14T06:18:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174758+0000", 2024-09-14T06:18:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9021e-05 2024-09-14T06:18:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175137+0000", 2024-09-14T06:18:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378511 2024-09-14T06:18:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178158+0000", 2024-09-14T06:18:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003020956 2024-09-14T06:18:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182386+0000", 2024-09-14T06:18:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042278840000000003 2024-09-14T06:18:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182410+0000", 2024-09-14T06:18:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4026e-05 2024-09-14T06:18:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182427+0000", 2024-09-14T06:18:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7711999999999999e-05 2024-09-14T06:18:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13248 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:18:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.968071815000002, 2024-09-14T06:18:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966527999999997, 2024-09-14T06:18:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13248 2024-09-14T06:18:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:18:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:18:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:18:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:18:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100548+0000", 2024-09-14T06:18:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:18:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100557+0000", 2024-09-14T06:18:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:18:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100563+0000", 2024-09-14T06:18:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2519999999999996e-06 2024-09-14T06:18:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175572+0000", 2024-09-14T06:18:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075008459 2024-09-14T06:18:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175656+0000", 2024-09-14T06:18:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3906999999999995e-05 2024-09-14T06:18:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176010+0000", 2024-09-14T06:18:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00035447499999999998 2024-09-14T06:18:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178304+0000", 2024-09-14T06:18:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0022941379999999998 2024-09-14T06:18:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182467+0000", 2024-09-14T06:18:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041632020000000004 2024-09-14T06:18:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182491+0000", 2024-09-14T06:18:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3733000000000001e-05 2024-09-14T06:18:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182509+0000", 2024-09-14T06:18:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8113000000000001e-05 2024-09-14T06:18:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13249 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:18:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.968036881000003, 2024-09-14T06:18:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082037225000000005, 2024-09-14T06:18:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13249 2024-09-14T06:18:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:18:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:18:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:18:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4600000000000005e-07 2024-09-14T06:18:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100584+0000", 2024-09-14T06:18:02.211 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.947e-06 2024-09-14T06:18:02.211 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.211 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.211 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.211 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100593+0000", 2024-09-14T06:18:02.211 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9760000000000001e-06 2024-09-14T06:18:02.211 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.211 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.211 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.211 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100597+0000", 2024-09-14T06:18:02.211 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9689999999999996e-06 2024-09-14T06:18:02.211 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.211 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176461+0000", 2024-09-14T06:18:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075864132000000001 2024-09-14T06:18:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176516+0000", 2024-09-14T06:18:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4416999999999998e-05 2024-09-14T06:18:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176734+0000", 2024-09-14T06:18:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021859399999999999 2024-09-14T06:18:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178308+0000", 2024-09-14T06:18:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015733660000000001 2024-09-14T06:18:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182573+0000", 2024-09-14T06:18:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042649200000000002 2024-09-14T06:18:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182597+0000", 2024-09-14T06:18:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4355e-05 2024-09-14T06:18:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182615+0000", 2024-09-14T06:18:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8003e-05 2024-09-14T06:18:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13250 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:18:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 85.967872275999994, 2024-09-14T06:18:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081983890000000004, 2024-09-14T06:18:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:02.215 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:02.215 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:02.215 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:02.215 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:02.215 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:02.215 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13250 2024-09-14T06:18:02.215 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.215 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:02.215 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.215 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:02.215 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:18:02.215 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.215 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.215 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.215 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:02.216 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:18:02.216 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:02.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.216 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.216 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:02.216 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100743+0000", 2024-09-14T06:18:02.216 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:18:02.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.216 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.216 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:02.216 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100753+0000", 2024-09-14T06:18:02.216 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9979999999999998e-06 2024-09-14T06:18:02.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.216 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.216 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:02.217 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100772+0000", 2024-09-14T06:18:02.217 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8664e-05 2024-09-14T06:18:02.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.217 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.217 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:02.217 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100779+0000", 2024-09-14T06:18:02.217 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.046e-06 2024-09-14T06:18:02.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.217 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.217 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:02.217 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177049+0000", 2024-09-14T06:18:02.217 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076270819000000004 2024-09-14T06:18:02.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.217 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.218 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:02.218 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177140+0000", 2024-09-14T06:18:02.218 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0746000000000002e-05 2024-09-14T06:18:02.218 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.218 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.218 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:02.218 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177519+0000", 2024-09-14T06:18:02.218 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378656 2024-09-14T06:18:02.218 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.218 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.218 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:02.218 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178349+0000", 2024-09-14T06:18:02.218 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00083061599999999995 2024-09-14T06:18:02.218 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.218 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.218 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:02.219 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182679+0000", 2024-09-14T06:18:02.219 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043298379999999999 2024-09-14T06:18:02.219 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.219 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.219 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:02.219 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182705+0000", 2024-09-14T06:18:02.219 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5602999999999999e-05 2024-09-14T06:18:02.219 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:02.219 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:02.219 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:02.219 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182726+0000", 2024-09-14T06:18:02.219 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1297e-05 2024-09-14T06:18:02.219 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.220 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.220 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.220 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:02.220 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:02.220 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:02.221 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:18:02.222 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:02.219+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:02.322 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:02.321+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:02.368 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-09-14T06:18:02.402 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:02.402 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:02.402 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:18:02.402 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:02.419 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:18:02.442 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:18:02.464 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:02.463+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:02.524 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:02.523+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:02.547 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:02.547 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:02.547 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:18:02.547 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:18:02.547 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:02.563 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:18:02.599 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:18:02.600 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:18:02.616 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:18:02.617 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:18:02.625 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:18:02.621+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:18:02.633 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:18:02.634 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:18:02.651 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:18:02.651 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:18:02.670 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:18:02.671 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:18:02.691 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:18:02.691 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:18:02.711 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:18:02.712 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:18:02.726 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:18:02.721+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:18:02.727 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:02.728 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:02.731 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:18:02.732 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:18:02.742 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:18:02.827 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:02.825+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:02.863 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:02.863 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:02.878 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:18:02.928 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:02.927+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:03.001 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:03.001 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:03.017 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:18:03.060 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:03.059+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:03.099 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:18:03.100 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T06:18:03.130 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:03.127+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:03.184 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:03.184 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:03.198 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:18:03.244 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:03.243+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:03.332 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:03.331+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:03.371 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:03.371 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:03.389 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:18:03.432 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:03.429+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:03.533 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:03.529+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:03.567 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:03.567 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:03.583 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:18:03.634 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:03.629+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:03.703 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:03.703 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:03.718 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:18:03.735 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:03.731+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:03.837 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:03.835+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:03.854 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:03.854 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:03.868 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:18:03.938 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:03.935+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:03.986 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:03.986 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:04.001 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:18:04.047 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:04.045+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:04.130 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:04.130 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:04.142 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:04.139+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:04.145 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:18:04.195 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T06:18:04.229 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T06:18:04.246 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:04.243+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:04.323 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:04.323 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:04.342 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:18:04.388 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:18:04.385+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:18:04.393 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:18:04.450 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:04.447+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:04.482 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:04.482 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:04.499 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:18:04.550 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:04.547+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:04.618 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:04.618 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:04.632 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:18:04.676 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:04.673+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:04.751 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:18:04.752 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:04.752 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:04.754 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:18:04.749+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:18:04.766 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:18:04.858 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:04.855+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:04.897 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:18:04.897 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:18:04.915 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:18:04.915 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:18:04.929 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:04.929 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:04.932 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:18:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:18:04.944 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:18:04.951 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:18:04.951 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:18:04.958 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:04.953+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:04.972 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:18:04.972 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:18:04.993 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:18:04.993 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:18:05.014 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:18:05.014 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:18:05.035 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:18:05.035 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:18:05.062 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:05.062 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:18:05.062 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:18:05.062 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:18:05.062 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:3916 3.16 3:6ab8b1d8:::rbd_data.1152bd92e04f.0000000000000002:head [stat,truncate 1590169] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:18:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:18:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 97.481192097000005, 2024-09-14T06:18:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.022958078999999999, 2024-09-14T06:18:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3916 2024-09-14T06:18:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:18:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579201+0000", 2024-09-14T06:18:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579203+0000", 2024-09-14T06:18:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.748e-06 2024-09-14T06:18:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579208+0000", 2024-09-14T06:18:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:18:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579223+0000", 2024-09-14T06:18:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.508e-05 2024-09-14T06:18:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579232+0000", 2024-09-14T06:18:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9719999999999998e-06 2024-09-14T06:18:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579307+0000", 2024-09-14T06:18:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5028000000000001e-05 2024-09-14T06:18:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579412+0000", 2024-09-14T06:18:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010525400000000001 2024-09-14T06:18:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 3", 2024-09-14T06:18:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.579535+0000", 2024-09-14T06:18:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000122726 2024-09-14T06:18:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.581415+0000", 2024-09-14T06:18:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018800360000000001 2024-09-14T06:18:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602017+0000", 2024-09-14T06:18:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.020602425000000001 2024-09-14T06:18:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602096+0000", 2024-09-14T06:18:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8968999999999996e-05 2024-09-14T06:18:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:27.602159+0000", 2024-09-14T06:18:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2824999999999996e-05 2024-09-14T06:18:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:7151 3.e 3:7793b17d:::rbd_data.1152f87cbac6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:18:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:18:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 93.730796229999996, 2024-09-14T06:18:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.029556783, 2024-09-14T06:18:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 7151 2024-09-14T06:18:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:18:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329597+0000", 2024-09-14T06:18:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329599+0000", 2024-09-14T06:18:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5270000000000001e-06 2024-09-14T06:18:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329610+0000", 2024-09-14T06:18:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0431000000000001e-05 2024-09-14T06:18:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329634+0000", 2024-09-14T06:18:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3617000000000001e-05 2024-09-14T06:18:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329644+0000", 2024-09-14T06:18:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.095e-05 2024-09-14T06:18:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329712+0000", 2024-09-14T06:18:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7067e-05 2024-09-14T06:18:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.329827+0000", 2024-09-14T06:18:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011498 2024-09-14T06:18:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 0", 2024-09-14T06:18:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.334271+0000", 2024-09-14T06:18:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0044445839999999997 2024-09-14T06:18:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.358967+0000", 2024-09-14T06:18:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.024696356999999999 2024-09-14T06:18:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359027+0000", 2024-09-14T06:18:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0006000000000001e-05 2024-09-14T06:18:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359099+0000", 2024-09-14T06:18:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1052999999999999e-05 2024-09-14T06:18:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359154+0000", 2024-09-14T06:18:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5210999999999997e-05 2024-09-14T06:18:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:7151 3.e e32/29 ondisk, result = 0)", 2024-09-14T06:18:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:18:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 93.710851796, 2024-09-14T06:18:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0096251340000000005, 2024-09-14T06:18:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:18:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349541+0000", 2024-09-14T06:18:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349543+0000", 2024-09-14T06:18:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:18:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349552+0000", 2024-09-14T06:18:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1039999999999996e-06 2024-09-14T06:18:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349579+0000", 2024-09-14T06:18:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6449999999999999e-05 2024-09-14T06:18:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.349593+0000", 2024-09-14T06:18:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4099e-05 2024-09-14T06:18:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359003+0000", 2024-09-14T06:18:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0094096119999999995 2024-09-14T06:18:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359014+0000", 2024-09-14T06:18:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0871e-05 2024-09-14T06:18:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:31.359167+0000", 2024-09-14T06:18:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00015300700000000001 2024-09-14T06:18:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15619 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'124, mlcod=36'119)", 2024-09-14T06:18:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:18:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 88.762538633999995, 2024-09-14T06:18:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061545389999999997, 2024-09-14T06:18:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:18:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:18:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297855+0000", 2024-09-14T06:18:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3399999999999998e-07 2024-09-14T06:18:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297861+0000", 2024-09-14T06:18:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7640000000000002e-06 2024-09-14T06:18:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297876+0000", 2024-09-14T06:18:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4881e-05 2024-09-14T06:18:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.297884+0000", 2024-09-14T06:18:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.588e-06 2024-09-14T06:18:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298294+0000", 2024-09-14T06:18:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00041011200000000002 2024-09-14T06:18:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.298317+0000", 2024-09-14T06:18:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3417999999999998e-05 2024-09-14T06:18:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303972+0000", 2024-09-14T06:18:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056548680000000004 2024-09-14T06:18:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.304009+0000", 2024-09-14T06:18:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7073999999999998e-05 2024-09-14T06:18:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:15621 3.11 e36/29 3:880eb6ac:::rbd_object_map.115223d7013d.0000000000000007:head v 36'125, mlcod=36'119)", 2024-09-14T06:18:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:18:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 88.761027616999996, 2024-09-14T06:18:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0069496169999999999, 2024-09-14T06:18:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:18:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:18:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299366+0000", 2024-09-14T06:18:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2200000000000003e-07 2024-09-14T06:18:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299372+0000", 2024-09-14T06:18:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3850000000000001e-06 2024-09-14T06:18:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299387+0000", 2024-09-14T06:18:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4905e-05 2024-09-14T06:18:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.299393+0000", 2024-09-14T06:18:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9090000000000003e-06 2024-09-14T06:18:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303591+0000", 2024-09-14T06:18:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041976640000000003 2024-09-14T06:18:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.303619+0000", 2024-09-14T06:18:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7651e-05 2024-09-14T06:18:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306287+0000", 2024-09-14T06:18:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026686689999999998 2024-09-14T06:18:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.306315+0000", 2024-09-14T06:18:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7712000000000002e-05 2024-09-14T06:18:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:24229 3.5 e47/34 3:a30fd30b:::rbd_data.1152db28e106.0000000000000001:head v 47'55, mlcod=45'53)", 2024-09-14T06:18:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:18:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 73.150382019000006, 2024-09-14T06:18:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0092924199999999992, 2024-09-14T06:18:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:18:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910011+0000", 2024-09-14T06:18:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.910013+0000", 2024-09-14T06:18:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.812e-06 2024-09-14T06:18:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915374+0000", 2024-09-14T06:18:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053609240000000004 2024-09-14T06:18:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915395+0000", 2024-09-14T06:18:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1118e-05 2024-09-14T06:18:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915406+0000", 2024-09-14T06:18:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1303e-05 2024-09-14T06:18:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915490+0000", 2024-09-14T06:18:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4017000000000006e-05 2024-09-14T06:18:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.915515+0000", 2024-09-14T06:18:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4712e-05 2024-09-14T06:18:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919240+0000", 2024-09-14T06:18:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037247690000000002 2024-09-14T06:18:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:51.919304+0000", 2024-09-14T06:18:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3764999999999997e-05 2024-09-14T06:18:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "MOSDPGPush(3.1f 55/54 [PushOp(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:head, version: 31'47, data_included: [0~98304,360448~278528], data_size: 376832, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:fc13ad45:::rbd_data.1152bd92e04f.0000000000000000:head@31'47, size: 638976, copy_subset: [(0, 638976)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 638976, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:18:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:18:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.850119074000006, 2024-09-14T06:18:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062487460000000003, 2024-09-14T06:18:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:18:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210274+0000", 2024-09-14T06:18:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210275+0000", 2024-09-14T06:18:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:18:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210752+0000", 2024-09-14T06:18:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047772100000000002 2024-09-14T06:18:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210781+0000", 2024-09-14T06:18:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8391999999999999e-05 2024-09-14T06:18:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.210805+0000", 2024-09-14T06:18:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4142999999999999e-05 2024-09-14T06:18:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.213641+0000", 2024-09-14T06:18:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002835827 2024-09-14T06:18:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.213663+0000", 2024-09-14T06:18:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.243e-05 2024-09-14T06:18:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:00.216523+0000", 2024-09-14T06:18:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028597729999999999 2024-09-14T06:18:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:50564 3.16 3:689e8220:::rbd_header.1152757de986:head [call rbd.snapshot_add in=15b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e66)", 2024-09-14T06:18:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:13.319284+0000", 2024-09-14T06:18:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 51.741108693999998, 2024-09-14T06:18:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.030881613, 2024-09-14T06:18:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50564 2024-09-14T06:18:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319284+0000", 2024-09-14T06:18:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319284+0000", 2024-09-14T06:18:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319286+0000", 2024-09-14T06:18:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.849e-06 2024-09-14T06:18:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319293+0000", 2024-09-14T06:18:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.72e-06 2024-09-14T06:18:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319313+0000", 2024-09-14T06:18:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9833000000000001e-05 2024-09-14T06:18:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319325+0000", 2024-09-14T06:18:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1924e-05 2024-09-14T06:18:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319441+0000", 2024-09-14T06:18:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000115952 2024-09-14T06:18:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op must wait for map", 2024-09-14T06:18:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319453+0000", 2024-09-14T06:18:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1810999999999999e-05 2024-09-14T06:18:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322905+0000", 2024-09-14T06:18:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034519680000000001 2024-09-14T06:18:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:18:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322917+0000", 2024-09-14T06:18:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1946999999999999e-05 2024-09-14T06:18:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.342215+0000", 2024-09-14T06:18:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0192988300000001 2024-09-14T06:18:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.342451+0000", 2024-09-14T06:18:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00023546 2024-09-14T06:18:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.342959+0000", 2024-09-14T06:18:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050845999999999997 2024-09-14T06:18:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.347472+0000", 2024-09-14T06:18:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045125310000000002 2024-09-14T06:18:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.349544+0000", 2024-09-14T06:18:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020721730000000001 2024-09-14T06:18:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.349727+0000", 2024-09-14T06:18:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000182776 2024-09-14T06:18:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.350166+0000", 2024-09-14T06:18:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000439379 2024-09-14T06:18:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:50565 3.16 3:689e8220:::rbd_header.1152757de986:head [watch reconnect cookie 140175519011472 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-14T06:18:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:13.319351+0000", 2024-09-14T06:18:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 51.741042116000003, 2024-09-14T06:18:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0352714430000001, 2024-09-14T06:18:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50565 2024-09-14T06:18:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319351+0000", 2024-09-14T06:18:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319351+0000", 2024-09-14T06:18:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319352+0000", 2024-09-14T06:18:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0500000000000003e-07 2024-09-14T06:18:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319355+0000", 2024-09-14T06:18:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3960000000000002e-06 2024-09-14T06:18:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319367+0000", 2024-09-14T06:18:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1518999999999999e-05 2024-09-14T06:18:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319373+0000", 2024-09-14T06:18:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7800000000000003e-06 2024-09-14T06:18:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319517+0000", 2024-09-14T06:18:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000143719 2024-09-14T06:18:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting_for_map not empty", 2024-09-14T06:18:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.319520+0000", 2024-09-14T06:18:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1990000000000002e-06 2024-09-14T06:18:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322963+0000", 2024-09-14T06:18:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034424920000000001 2024-09-14T06:18:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:18:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.322971+0000", 2024-09-14T06:18:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0860000000000002e-06 2024-09-14T06:18:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343565+0000", 2024-09-14T06:18:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0205935530000001 2024-09-14T06:18:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for rw locks", 2024-09-14T06:18:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343675+0000", 2024-09-14T06:18:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00011048900000000001 2024-09-14T06:18:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.350295+0000", 2024-09-14T06:18:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006620094 2024-09-14T06:18:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.350584+0000", 2024-09-14T06:18:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00028850000000000002 2024-09-14T06:18:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.354623+0000", 2024-09-14T06:18:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040389110000000001 2024-09-14T06:18:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:50566 3.16 3:689e8220:::rbd_header.1152757de986:head [watch ping cookie 140175519011472 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-14T06:18:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:13.960177+0000", 2024-09-14T06:18:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 51.100215968999997, 2024-09-14T06:18:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.39354602500000002, 2024-09-14T06:18:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 50566 2024-09-14T06:18:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960177+0000", 2024-09-14T06:18:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960177+0000", 2024-09-14T06:18:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960178+0000", 2024-09-14T06:18:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.136e-06 2024-09-14T06:18:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960186+0000", 2024-09-14T06:18:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0299999999999994e-06 2024-09-14T06:18:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960202+0000", 2024-09-14T06:18:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5238999999999999e-05 2024-09-14T06:18:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960218+0000", 2024-09-14T06:18:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.668e-05 2024-09-14T06:18:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960350+0000", 2024-09-14T06:18:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013201 2024-09-14T06:18:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:18:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:13.960368+0000", 2024-09-14T06:18:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7526000000000001e-05 2024-09-14T06:18:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343805+0000", 2024-09-14T06:18:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.38343735400000001 2024-09-14T06:18:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for rw locks", 2024-09-14T06:18:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.343907+0000", 2024-09-14T06:18:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010133599999999999 2024-09-14T06:18:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.352997+0000", 2024-09-14T06:18:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0090907409999999994 2024-09-14T06:18:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.353352+0000", 2024-09-14T06:18:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00035436000000000002 2024-09-14T06:18:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:14.353723+0000", 2024-09-14T06:18:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00037161300000000002 2024-09-14T06:18:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:51037 3.11 3:89b51a5c:::rbd_data.11522fd7a0c5.0000000000000059:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-14T06:18:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:15.038049+0000", 2024-09-14T06:18:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 50.022344627000003, 2024-09-14T06:18:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.075311116999999997, 2024-09-14T06:18:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 51037 2024-09-14T06:18:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038049+0000", 2024-09-14T06:18:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038049+0000", 2024-09-14T06:18:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038051+0000", 2024-09-14T06:18:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:18:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038065+0000", 2024-09-14T06:18:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3576e-05 2024-09-14T06:18:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038097+0000", 2024-09-14T06:18:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2566e-05 2024-09-14T06:18:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038121+0000", 2024-09-14T06:18:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3207e-05 2024-09-14T06:18:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038291+0000", 2024-09-14T06:18:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00016996900000000001 2024-09-14T06:18:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.038543+0000", 2024-09-14T06:18:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00025233999999999999 2024-09-14T06:18:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.053218+0000", 2024-09-14T06:18:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014674957000000001 2024-09-14T06:18:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113195+0000", 2024-09-14T06:18:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.059976697000000002 2024-09-14T06:18:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113212+0000", 2024-09-14T06:18:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7299000000000002e-05 2024-09-14T06:18:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113267+0000", 2024-09-14T06:18:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4632999999999997e-05 2024-09-14T06:18:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113360+0000", 2024-09-14T06:18:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3129999999999998e-05 2024-09-14T06:18:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4434.0:51037 3.11 e67/66 ondisk, result = 0)", 2024-09-14T06:18:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:15.093818+0000", 2024-09-14T06:18:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.966574835999999, 2024-09-14T06:18:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.019384384000000001, 2024-09-14T06:18:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093818+0000", 2024-09-14T06:18:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093818+0000", 2024-09-14T06:18:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093820+0000", 2024-09-14T06:18:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0169999999999999e-06 2024-09-14T06:18:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093829+0000", 2024-09-14T06:18:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6500000000000002e-06 2024-09-14T06:18:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093860+0000", 2024-09-14T06:18:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1275999999999997e-05 2024-09-14T06:18:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.093883+0000", 2024-09-14T06:18:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2464999999999999e-05 2024-09-14T06:18:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113178+0000", 2024-09-14T06:18:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.019295643000000001 2024-09-14T06:18:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113189+0000", 2024-09-14T06:18:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0285999999999999e-05 2024-09-14T06:18:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:15.113203+0000", 2024-09-14T06:18:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4047e-05 2024-09-14T06:18:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:52268 3.e e70/34 3:7013111b:::rbd_data.1152cd930a87.0000000000000001:head v 70'315, mlcod=70'313)", 2024-09-14T06:18:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:17.975434+0000", 2024-09-14T06:18:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.084959638000001, 2024-09-14T06:18:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014816765000000001, 2024-09-14T06:18:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.975434+0000", 2024-09-14T06:18:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.975434+0000", 2024-09-14T06:18:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.975436+0000", 2024-09-14T06:18:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3010000000000001e-06 2024-09-14T06:18:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982453+0000", 2024-09-14T06:18:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0070167909999999997 2024-09-14T06:18:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982489+0000", 2024-09-14T06:18:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6603000000000002e-05 2024-09-14T06:18:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982507+0000", 2024-09-14T06:18:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8074e-05 2024-09-14T06:18:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982610+0000", 2024-09-14T06:18:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010224 2024-09-14T06:18:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982639+0000", 2024-09-14T06:18:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9009999999999998e-05 2024-09-14T06:18:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.990188+0000", 2024-09-14T06:18:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0075497150000000002 2024-09-14T06:18:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.990250+0000", 2024-09-14T06:18:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2031000000000003e-05 2024-09-14T06:18:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:52269 3.1f e70/56 3:fd802c48:::rbd_data.1152cd930a87.0000000000000002:head v 70'321, mlcod=70'318)", 2024-09-14T06:18:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:17.982671+0000", 2024-09-14T06:18:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 47.077722655999999, 2024-09-14T06:18:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.016453552, 2024-09-14T06:18:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982671+0000", 2024-09-14T06:18:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982671+0000", 2024-09-14T06:18:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.982672+0000", 2024-09-14T06:18:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.246e-06 2024-09-14T06:18:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991501+0000", 2024-09-14T06:18:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0088292590000000008 2024-09-14T06:18:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991521+0000", 2024-09-14T06:18:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0211e-05 2024-09-14T06:18:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991531+0000", 2024-09-14T06:18:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0207000000000001e-05 2024-09-14T06:18:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991605+0000", 2024-09-14T06:18:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3066999999999997e-05 2024-09-14T06:18:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.991633+0000", 2024-09-14T06:18:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8136e-05 2024-09-14T06:18:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.999052+0000", 2024-09-14T06:18:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0074192320000000004 2024-09-14T06:18:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:17.999124+0000", 2024-09-14T06:18:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2193999999999994e-05 2024-09-14T06:18:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:59899 3.9 e75/68 3:96b79e96:::rbd_data.115214b4562.0000000000000004:head v 75'319, mlcod=75'317)", 2024-09-14T06:18:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:24.822693+0000", 2024-09-14T06:18:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 40.237699804999998, 2024-09-14T06:18:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014246999, 2024-09-14T06:18:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.822693+0000", 2024-09-14T06:18:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.822693+0000", 2024-09-14T06:18:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.822695+0000", 2024-09-14T06:18:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.762e-06 2024-09-14T06:18:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831397+0000", 2024-09-14T06:18:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0087016230000000003 2024-09-14T06:18:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831424+0000", 2024-09-14T06:18:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7480000000000001e-05 2024-09-14T06:18:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831435+0000", 2024-09-14T06:18:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0943e-05 2024-09-14T06:18:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831509+0000", 2024-09-14T06:18:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3596000000000005e-05 2024-09-14T06:18:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.831531+0000", 2024-09-14T06:18:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2628000000000001e-05 2024-09-14T06:18:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.836873+0000", 2024-09-14T06:18:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053417849999999999 2024-09-14T06:18:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:24.836940+0000", 2024-09-14T06:18:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7181999999999995e-05 2024-09-14T06:18:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:64148 3.2 e86/29 3:41d40d1e:::rbd_data.115267b2c56c.0000000000000000:head v 86'276, mlcod=80'274)", 2024-09-14T06:18:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:38.879703+0000", 2024-09-14T06:18:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.180690176999999, 2024-09-14T06:18:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.014921665000000001, 2024-09-14T06:18:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.879703+0000", 2024-09-14T06:18:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.879703+0000", 2024-09-14T06:18:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.879704+0000", 2024-09-14T06:18:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0219999999999999e-06 2024-09-14T06:18:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885314+0000", 2024-09-14T06:18:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056102119999999998 2024-09-14T06:18:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885331+0000", 2024-09-14T06:18:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6696000000000001e-05 2024-09-14T06:18:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885342+0000", 2024-09-14T06:18:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0886999999999999e-05 2024-09-14T06:18:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885468+0000", 2024-09-14T06:18:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012655399999999999 2024-09-14T06:18:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.885517+0000", 2024-09-14T06:18:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9057999999999997e-05 2024-09-14T06:18:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.894572+0000", 2024-09-14T06:18:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0090541130000000008 2024-09-14T06:18:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.894625+0000", 2024-09-14T06:18:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3122999999999999e-05 2024-09-14T06:18:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:64152 3.3 e86/29 3:c1c8d963:::rbd_data.115267b2c56c.0000000000000002:head v 86'180, mlcod=79'178)", 2024-09-14T06:18:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:38.884954+0000", 2024-09-14T06:18:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.175438714999999, 2024-09-14T06:18:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011982222000000001, 2024-09-14T06:18:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.884954+0000", 2024-09-14T06:18:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.884954+0000", 2024-09-14T06:18:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.884956+0000", 2024-09-14T06:18:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1179999999999999e-06 2024-09-14T06:18:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886783+0000", 2024-09-14T06:18:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018269289999999999 2024-09-14T06:18:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886795+0000", 2024-09-14T06:18:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1975e-05 2024-09-14T06:18:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886802+0000", 2024-09-14T06:18:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.824e-06 2024-09-14T06:18:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886881+0000", 2024-09-14T06:18:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8702000000000006e-05 2024-09-14T06:18:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.886902+0000", 2024-09-14T06:18:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0854000000000001e-05 2024-09-14T06:18:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.896887+0000", 2024-09-14T06:18:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0099853700000000004 2024-09-14T06:18:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.896937+0000", 2024-09-14T06:18:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9450000000000003e-05 2024-09-14T06:18:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4434.0:64155 3.1 e86/29 3:86008c25:::rbd_data.115267b2c56c.0000000000000003:head v 86'451, mlcod=78'449)", 2024-09-14T06:18:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:18:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.157337629000001, 2024-09-14T06:18:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006333038, 2024-09-14T06:18:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:18:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:18:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.903056+0000", 2024-09-14T06:18:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:18:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906826+0000", 2024-09-14T06:18:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037703260000000001 2024-09-14T06:18:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906845+0000", 2024-09-14T06:18:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8488999999999999e-05 2024-09-14T06:18:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906857+0000", 2024-09-14T06:18:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2571e-05 2024-09-14T06:18:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906966+0000", 2024-09-14T06:18:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010832000000000001 2024-09-14T06:18:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.906984+0000", 2024-09-14T06:18:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8068999999999999e-05 2024-09-14T06:18:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.909306+0000", 2024-09-14T06:18:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023221050000000001 2024-09-14T06:18:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.909389+0000", 2024-09-14T06:18:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2865e-05 2024-09-14T06:18:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(osd.5.0:1871 3.1e e99/29 3:783464bd:::rbd_data.11522fd7a0c5.000000000000005b:19 v 99'341, mlcod=95'338)", 2024-09-14T06:18:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:53.229369+0000", 2024-09-14T06:18:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 11.831024098, 2024-09-14T06:18:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0063464239999999998, 2024-09-14T06:18:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229369+0000", 2024-09-14T06:18:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229369+0000", 2024-09-14T06:18:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229370+0000", 2024-09-14T06:18:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7300000000000005e-07 2024-09-14T06:18:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229388+0000", 2024-09-14T06:18:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8006999999999999e-05 2024-09-14T06:18:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229406+0000", 2024-09-14T06:18:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7663e-05 2024-09-14T06:18:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.229417+0000", 2024-09-14T06:18:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1749999999999999e-05 2024-09-14T06:18:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230797+0000", 2024-09-14T06:18:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0013801589999999999 2024-09-14T06:18:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.233207+0000", 2024-09-14T06:18:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024092889999999998 2024-09-14T06:18:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.235660+0000", 2024-09-14T06:18:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024534370000000002 2024-09-14T06:18:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.235716+0000", 2024-09-14T06:18:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5346000000000002e-05 2024-09-14T06:18:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(osd.0.0:2754 3.e e99/34 3:70005258:::rbd_data.11522fd7a0c5.000000000000018a:19 v 99'690, mlcod=98'687)", 2024-09-14T06:18:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:53.230299+0000", 2024-09-14T06:18:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 11.830094185, 2024-09-14T06:18:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064664880000000003, 2024-09-14T06:18:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230299+0000", 2024-09-14T06:18:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230299+0000", 2024-09-14T06:18:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230299+0000", 2024-09-14T06:18:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.32e-07 2024-09-14T06:18:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230304+0000", 2024-09-14T06:18:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8729999999999998e-06 2024-09-14T06:18:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230316+0000", 2024-09-14T06:18:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2104e-05 2024-09-14T06:18:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230325+0000", 2024-09-14T06:18:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4079999999999998e-06 2024-09-14T06:18:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.230896+0000", 2024-09-14T06:18:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057151700000000001 2024-09-14T06:18:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.234280+0000", 2024-09-14T06:18:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033832060000000001 2024-09-14T06:18:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.236724+0000", 2024-09-14T06:18:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024447090000000002 2024-09-14T06:18:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:53.236765+0000", 2024-09-14T06:18:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1239000000000003e-05 2024-09-14T06:18:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.165 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:05.166 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:05.059+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:05.167 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:18:05.167 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:05.163+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:05.268 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:05.265+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:05.344 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:18:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:18:05.345 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:05.360 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:18:05.369 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:05.367+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:05.383 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-09-14T06:18:05.400 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T06:18:05.431 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T06:18:05.469 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:05.467+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:05.521 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:18:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:18:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:18:05.522 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:05.545 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:18:05.591 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:18:05.589+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:18:05.674 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:05.671+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:05.679 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:18:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:18:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:18:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:18:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 89.685968806999995, 2024-09-14T06:18:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:18:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:18:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:18:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:18:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:18:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:18:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:18:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:18:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:18:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:18:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:18:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:18:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:18:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:18:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:18:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:18:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:18:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:18:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:18:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 89.685955734999993, 2024-09-14T06:18:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:18:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:18:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:18:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:18:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:18:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:18:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:18:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:18:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:18:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:18:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:18:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:18:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:18:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:18:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:18:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:18:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:18:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:18:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:18:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 89.685938303, 2024-09-14T06:18:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:18:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:18:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:18:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:18:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:18:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:18:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:18:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:18:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:18:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:18:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:18:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:18:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:18:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:18:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:18:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:18:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:18:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:18:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:18:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 89.685917197999999, 2024-09-14T06:18:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:18:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:18:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:18:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:18:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:18:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:18:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:18:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:18:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:18:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:18:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:18:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:18:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:18:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:18:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:18:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:18:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:18:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:18:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:18:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 89.68589609, 2024-09-14T06:18:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:18:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:18:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:18:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:18:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:18:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:18:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:18:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:18:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:18:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:18:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:18:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:18:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:18:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:18:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:18:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:18:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:18:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:18:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48234 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:18:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 55.644624798000002, 2024-09-14T06:18:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051730353999999999, 2024-09-14T06:18:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48234 2024-09-14T06:18:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:18:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:18:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032937+0000", 2024-09-14T06:18:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:18:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032940+0000", 2024-09-14T06:18:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0580000000000002e-06 2024-09-14T06:18:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032946+0000", 2024-09-14T06:18:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2960000000000004e-06 2024-09-14T06:18:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032951+0000", 2024-09-14T06:18:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.938e-06 2024-09-14T06:18:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074419+0000", 2024-09-14T06:18:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041467979000000002 2024-09-14T06:18:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074490+0000", 2024-09-14T06:18:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1324999999999999e-05 2024-09-14T06:18:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074859+0000", 2024-09-14T06:18:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036914899999999999 2024-09-14T06:18:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078464+0000", 2024-09-14T06:18:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036043170000000001 2024-09-14T06:18:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084547+0000", 2024-09-14T06:18:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006083097 2024-09-14T06:18:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084614+0000", 2024-09-14T06:18:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7509999999999996e-05 2024-09-14T06:18:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084667+0000", 2024-09-14T06:18:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:18:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48237 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:18:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 55.644545053999998, 2024-09-14T06:18:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051772830999999998, 2024-09-14T06:18:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48237 2024-09-14T06:18:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:18:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:18:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:18:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0499999999999999e-07 2024-09-14T06:18:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033018+0000", 2024-09-14T06:18:05.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0949999999999998e-06 2024-09-14T06:18:05.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033023+0000", 2024-09-14T06:18:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1620000000000001e-06 2024-09-14T06:18:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033026+0000", 2024-09-14T06:18:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0120000000000001e-06 2024-09-14T06:18:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075349+0000", 2024-09-14T06:18:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042323373999999997 2024-09-14T06:18:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075394+0000", 2024-09-14T06:18:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4861000000000003e-05 2024-09-14T06:18:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075578+0000", 2024-09-14T06:18:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00018414899999999999 2024-09-14T06:18:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078894+0000", 2024-09-14T06:18:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033158480000000001 2024-09-14T06:18:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084677+0000", 2024-09-14T06:18:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0057832120000000002 2024-09-14T06:18:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084752+0000", 2024-09-14T06:18:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5123000000000006e-05 2024-09-14T06:18:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084789+0000", 2024-09-14T06:18:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6690000000000003e-05 2024-09-14T06:18:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48243 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:18:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 55.644463928999997, 2024-09-14T06:18:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051780221000000001, 2024-09-14T06:18:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48243 2024-09-14T06:18:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:18:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:18:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:18:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:18:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033100+0000", 2024-09-14T06:18:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:18:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033108+0000", 2024-09-14T06:18:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1760000000000004e-06 2024-09-14T06:18:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033111+0000", 2024-09-14T06:18:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0790000000000001e-06 2024-09-14T06:18:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075903+0000", 2024-09-14T06:18:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042791942999999999 2024-09-14T06:18:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075987+0000", 2024-09-14T06:18:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4467000000000003e-05 2024-09-14T06:18:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.076259+0000", 2024-09-14T06:18:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027201900000000003 2024-09-14T06:18:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078947+0000", 2024-09-14T06:18:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026875879999999999 2024-09-14T06:18:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084797+0000", 2024-09-14T06:18:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0058500380000000001 2024-09-14T06:18:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084843+0000", 2024-09-14T06:18:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6298999999999998e-05 2024-09-14T06:18:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084877+0000", 2024-09-14T06:18:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4378999999999997e-05 2024-09-14T06:18:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48287 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:18:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 55.644424129000001, 2024-09-14T06:18:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051924706000000001, 2024-09-14T06:18:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48287 2024-09-14T06:18:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:18:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:18:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033138+0000", 2024-09-14T06:18:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6599999999999996e-07 2024-09-14T06:18:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033141+0000", 2024-09-14T06:18:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0879999999999999e-06 2024-09-14T06:18:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033147+0000", 2024-09-14T06:18:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2249999999999997e-06 2024-09-14T06:18:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033153+0000", 2024-09-14T06:18:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.066e-06 2024-09-14T06:18:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077790+0000", 2024-09-14T06:18:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.044637255000000001 2024-09-14T06:18:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077845+0000", 2024-09-14T06:18:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4410999999999998e-05 2024-09-14T06:18:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078062+0000", 2024-09-14T06:18:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021755599999999999 2024-09-14T06:18:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.080991+0000", 2024-09-14T06:18:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029282969999999998 2024-09-14T06:18:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084885+0000", 2024-09-14T06:18:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003894495 2024-09-14T06:18:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085020+0000", 2024-09-14T06:18:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013500799999999999 2024-09-14T06:18:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085062+0000", 2024-09-14T06:18:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1638999999999999e-05 2024-09-14T06:18:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48288 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:18:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 55.644400429999997, 2024-09-14T06:18:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051994018000000003, 2024-09-14T06:18:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48288 2024-09-14T06:18:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:18:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:18:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:18:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.89e-07 2024-09-14T06:18:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033164+0000", 2024-09-14T06:18:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.779e-06 2024-09-14T06:18:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033171+0000", 2024-09-14T06:18:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6279999999999999e-06 2024-09-14T06:18:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033176+0000", 2024-09-14T06:18:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1610000000000002e-06 2024-09-14T06:18:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078991+0000", 2024-09-14T06:18:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045815348999999998 2024-09-14T06:18:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079086+0000", 2024-09-14T06:18:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5282e-05 2024-09-14T06:18:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079397+0000", 2024-09-14T06:18:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031055999999999998 2024-09-14T06:18:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084432+0000", 2024-09-14T06:18:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050355599999999997 2024-09-14T06:18:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085073+0000", 2024-09-14T06:18:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064015199999999995 2024-09-14T06:18:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085120+0000", 2024-09-14T06:18:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7107000000000001e-05 2024-09-14T06:18:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085155+0000", 2024-09-14T06:18:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4950999999999998e-05 2024-09-14T06:18:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48416 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:18:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 55.642160017999998, 2024-09-14T06:18:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053690144000000002, 2024-09-14T06:18:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48416 2024-09-14T06:18:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:18:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:18:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035402+0000", 2024-09-14T06:18:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7999999999999995e-07 2024-09-14T06:18:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035405+0000", 2024-09-14T06:18:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3960000000000002e-06 2024-09-14T06:18:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035413+0000", 2024-09-14T06:18:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2209999999999997e-06 2024-09-14T06:18:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035420+0000", 2024-09-14T06:18:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1539999999999996e-06 2024-09-14T06:18:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081021+0000", 2024-09-14T06:18:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045600389999999998 2024-09-14T06:18:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081091+0000", 2024-09-14T06:18:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0325000000000002e-05 2024-09-14T06:18:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081434+0000", 2024-09-14T06:18:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000342519 2024-09-14T06:18:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085254+0000", 2024-09-14T06:18:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038201609999999999 2024-09-14T06:18:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088977+0000", 2024-09-14T06:18:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003722805 2024-09-14T06:18:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089048+0000", 2024-09-14T06:18:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1266999999999994e-05 2024-09-14T06:18:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089091+0000", 2024-09-14T06:18:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3325999999999998e-05 2024-09-14T06:18:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48417 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:18:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 55.642131349000003, 2024-09-14T06:18:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053805032000000003, 2024-09-14T06:18:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48417 2024-09-14T06:18:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:18:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:18:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:18:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9500000000000003e-07 2024-09-14T06:18:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035433+0000", 2024-09-14T06:18:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.796e-06 2024-09-14T06:18:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035442+0000", 2024-09-14T06:18:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7120000000000006e-06 2024-09-14T06:18:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035448+0000", 2024-09-14T06:18:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.015e-06 2024-09-14T06:18:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081855+0000", 2024-09-14T06:18:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.046407634000000003 2024-09-14T06:18:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081938+0000", 2024-09-14T06:18:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2687000000000003e-05 2024-09-14T06:18:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082313+0000", 2024-09-14T06:18:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000374872 2024-09-14T06:18:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087616+0000", 2024-09-14T06:18:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053032670000000004 2024-09-14T06:18:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089140+0000", 2024-09-14T06:18:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0015234350000000001 2024-09-14T06:18:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089184+0000", 2024-09-14T06:18:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4804e-05 2024-09-14T06:18:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089235+0000", 2024-09-14T06:18:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0315000000000002e-05 2024-09-14T06:18:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48418 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:18:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 55.64208721, 2024-09-14T06:18:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053936425000000003, 2024-09-14T06:18:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48418 2024-09-14T06:18:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:18:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:18:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035475+0000", 2024-09-14T06:18:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7000000000000003e-07 2024-09-14T06:18:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035479+0000", 2024-09-14T06:18:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2350000000000001e-06 2024-09-14T06:18:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035488+0000", 2024-09-14T06:18:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8820000000000007e-06 2024-09-14T06:18:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035494+0000", 2024-09-14T06:18:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3060000000000003e-06 2024-09-14T06:18:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082826+0000", 2024-09-14T06:18:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.047332008000000002 2024-09-14T06:18:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082883+0000", 2024-09-14T06:18:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7124999999999999e-05 2024-09-14T06:18:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.083385+0000", 2024-09-14T06:18:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050134099999999998 2024-09-14T06:18:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088888+0000", 2024-09-14T06:18:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005503127 2024-09-14T06:18:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089294+0000", 2024-09-14T06:18:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040669700000000003 2024-09-14T06:18:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089348+0000", 2024-09-14T06:18:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3729e-05 2024-09-14T06:18:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089410+0000", 2024-09-14T06:18:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2105000000000003e-05 2024-09-14T06:18:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48419 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:18:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 55.64200958, 2024-09-14T06:18:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054007545999999997, 2024-09-14T06:18:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48419 2024-09-14T06:18:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:18:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:18:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035552+0000", 2024-09-14T06:18:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4099999999999998e-07 2024-09-14T06:18:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035557+0000", 2024-09-14T06:18:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1819999999999997e-06 2024-09-14T06:18:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035566+0000", 2024-09-14T06:18:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1120000000000002e-06 2024-09-14T06:18:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035572+0000", 2024-09-14T06:18:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0430000000000004e-06 2024-09-14T06:18:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085304+0000", 2024-09-14T06:18:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049731707 2024-09-14T06:18:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085371+0000", 2024-09-14T06:18:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6771000000000002e-05 2024-09-14T06:18:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085601+0000", 2024-09-14T06:18:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022968000000000001 2024-09-14T06:18:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088893+0000", 2024-09-14T06:18:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032927780000000001 2024-09-14T06:18:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089467+0000", 2024-09-14T06:18:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057330399999999996 2024-09-14T06:18:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089520+0000", 2024-09-14T06:18:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3133000000000001e-05 2024-09-14T06:18:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089559+0000", 2024-09-14T06:18:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9195e-05 2024-09-14T06:18:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48420 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:18:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 55.641920349999999, 2024-09-14T06:18:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056401932000000002, 2024-09-14T06:18:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48420 2024-09-14T06:18:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:18:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:18:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035642+0000", 2024-09-14T06:18:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1519999999999999e-06 2024-09-14T06:18:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035647+0000", 2024-09-14T06:18:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2299999999999999e-06 2024-09-14T06:18:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035660+0000", 2024-09-14T06:18:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3166999999999999e-05 2024-09-14T06:18:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035666+0000", 2024-09-14T06:18:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1720000000000004e-06 2024-09-14T06:18:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086104+0000", 2024-09-14T06:18:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050437956999999999 2024-09-14T06:18:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086164+0000", 2024-09-14T06:18:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9592999999999997e-05 2024-09-14T06:18:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086402+0000", 2024-09-14T06:18:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000238314 2024-09-14T06:18:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089103+0000", 2024-09-14T06:18:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027011600000000002 2024-09-14T06:18:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091962+0000", 2024-09-14T06:18:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028581259999999999 2024-09-14T06:18:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092005+0000", 2024-09-14T06:18:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3111000000000001e-05 2024-09-14T06:18:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092043+0000", 2024-09-14T06:18:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7950000000000001e-05 2024-09-14T06:18:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48424 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:18:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 55.641656404999999, 2024-09-14T06:18:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056273173000000003, 2024-09-14T06:18:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48424 2024-09-14T06:18:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:18:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:18:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:18:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2900000000000003e-07 2024-09-14T06:18:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035909+0000", 2024-09-14T06:18:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6490000000000001e-06 2024-09-14T06:18:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035918+0000", 2024-09-14T06:18:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5089999999999999e-06 2024-09-14T06:18:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035924+0000", 2024-09-14T06:18:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7089999999999999e-06 2024-09-14T06:18:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087662+0000", 2024-09-14T06:18:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737621999999997 2024-09-14T06:18:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087744+0000", 2024-09-14T06:18:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2174000000000003e-05 2024-09-14T06:18:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088106+0000", 2024-09-14T06:18:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000361947 2024-09-14T06:18:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092053+0000", 2024-09-14T06:18:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0039473080000000001 2024-09-14T06:18:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092087+0000", 2024-09-14T06:18:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4153000000000003e-05 2024-09-14T06:18:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092138+0000", 2024-09-14T06:18:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0804999999999999e-05 2024-09-14T06:18:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092178+0000", 2024-09-14T06:18:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9668e-05 2024-09-14T06:18:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48508 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:18:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 55.636581323999998, 2024-09-14T06:18:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054906017000000001, 2024-09-14T06:18:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48508 2024-09-14T06:18:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:18:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:18:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040981+0000", 2024-09-14T06:18:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7700000000000004e-07 2024-09-14T06:18:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040984+0000", 2024-09-14T06:18:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1480000000000002e-06 2024-09-14T06:18:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040991+0000", 2024-09-14T06:18:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4580000000000004e-06 2024-09-14T06:18:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040997+0000", 2024-09-14T06:18:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7799999999999997e-06 2024-09-14T06:18:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089601+0000", 2024-09-14T06:18:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048604264000000001 2024-09-14T06:18:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089708+0000", 2024-09-14T06:18:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010681200000000001 2024-09-14T06:18:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090384+0000", 2024-09-14T06:18:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00067557499999999998 2024-09-14T06:18:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093879+0000", 2024-09-14T06:18:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034954679999999998 2024-09-14T06:18:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095764+0000", 2024-09-14T06:18:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018848070000000001 2024-09-14T06:18:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095837+0000", 2024-09-14T06:18:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3433e-05 2024-09-14T06:18:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095886+0000", 2024-09-14T06:18:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8495000000000002e-05 2024-09-14T06:18:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48537 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:18:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 55.635217554, 2024-09-14T06:18:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053681833999999998, 2024-09-14T06:18:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48537 2024-09-14T06:18:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:18:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:18:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:18:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2299999999999999e-07 2024-09-14T06:18:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042350+0000", 2024-09-14T06:18:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5690000000000004e-06 2024-09-14T06:18:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042361+0000", 2024-09-14T06:18:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1337e-05 2024-09-14T06:18:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042369+0000", 2024-09-14T06:18:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8359999999999992e-06 2024-09-14T06:18:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090914+0000", 2024-09-14T06:18:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048544945999999999 2024-09-14T06:18:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090995+0000", 2024-09-14T06:18:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0548000000000002e-05 2024-09-14T06:18:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091508+0000", 2024-09-14T06:18:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000513723 2024-09-14T06:18:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093900+0000", 2024-09-14T06:18:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023920429999999999 2024-09-14T06:18:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095944+0000", 2024-09-14T06:18:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020437379999999998 2024-09-14T06:18:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095987+0000", 2024-09-14T06:18:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2521999999999997e-05 2024-09-14T06:18:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.096025+0000", 2024-09-14T06:18:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8649000000000002e-05 2024-09-14T06:18:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:63810 3.1d 3.c887d99d (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-14T06:18:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:18:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 32.097459667999999, 2024-09-14T06:18:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.61346978900000004, 2024-09-14T06:18:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:18:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 63810 2024-09-14T06:18:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:18:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:18:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580104+0000", 2024-09-14T06:18:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4650000000000001e-06 2024-09-14T06:18:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580112+0000", 2024-09-14T06:18:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9219999999999997e-06 2024-09-14T06:18:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580130+0000", 2024-09-14T06:18:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8128999999999999e-05 2024-09-14T06:18:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580140+0000", 2024-09-14T06:18:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0285000000000001e-05 2024-09-14T06:18:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580201+0000", 2024-09-14T06:18:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0517999999999999e-05 2024-09-14T06:18:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:18:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580209+0000", 2024-09-14T06:18:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6430000000000001e-06 2024-09-14T06:18:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:34.193554+0000", 2024-09-14T06:18:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.61334482899999998 2024-09-14T06:18:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:34.193571+0000", 2024-09-14T06:18:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6997999999999999e-05 2024-09-14T06:18:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:63832 3.1d 3:b99be113:::rbd_object_map.11525ae33f05:head [read 0~52] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T06:18:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:18:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 28.439463281999998, 2024-09-14T06:18:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.99066972200000003, 2024-09-14T06:18:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 63832 2024-09-14T06:18:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:18:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:18:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238100+0000", 2024-09-14T06:18:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0849999999999999e-06 2024-09-14T06:18:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238108+0000", 2024-09-14T06:18:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.089e-06 2024-09-14T06:18:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238123+0000", 2024-09-14T06:18:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5418e-05 2024-09-14T06:18:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238136+0000", 2024-09-14T06:18:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2204e-05 2024-09-14T06:18:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238913+0000", 2024-09-14T06:18:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00077720399999999998 2024-09-14T06:18:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:18:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238934+0000", 2024-09-14T06:18:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1311999999999999e-05 2024-09-14T06:18:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228134+0000", 2024-09-14T06:18:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.98919991200000001 2024-09-14T06:18:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228261+0000", 2024-09-14T06:18:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000126742 2024-09-14T06:18:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228768+0000", 2024-09-14T06:18:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050675600000000003 2024-09-14T06:18:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:05.804 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:05.805 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:18:05.806 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:05.803+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:05.907 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:05.903+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:05.937 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:05.937 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:05.937 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:18:05.937 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:05.951 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:18:06.007 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:06.005+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:06.110 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:06.107+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:06.121 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:06.122 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:06.122 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:18:06.122 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:18:06.122 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:06.136 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:18:06.210 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:06.205+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:06.304 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:06.304 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:18:06.304 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:18:06.304 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:18:06.304 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.304 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13229 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.304 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:18:06.305 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.201672287999997, 2024-09-14T06:18:06.305 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081798868999999996, 2024-09-14T06:18:06.305 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.305 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.305 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.305 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.305 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.305 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.305 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13229 2024-09-14T06:18:06.305 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.305 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.305 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.305 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.305 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:18:06.306 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.306 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.306 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.306 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.306 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:18:06.306 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.306 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.306 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.306 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.306 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098710+0000", 2024-09-14T06:18:06.306 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8199999999999999e-07 2024-09-14T06:18:06.306 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.306 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.307 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.307 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098714+0000", 2024-09-14T06:18:06.307 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:18:06.307 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.307 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.307 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.307 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098727+0000", 2024-09-14T06:18:06.307 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2520999999999999e-05 2024-09-14T06:18:06.307 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.307 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.307 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.307 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098749+0000", 2024-09-14T06:18:06.307 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2773999999999999e-05 2024-09-14T06:18:06.307 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.308 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.308 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.308 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160234+0000", 2024-09-14T06:18:06.308 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.061484430999999999 2024-09-14T06:18:06.308 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.308 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.308 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.308 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160281+0000", 2024-09-14T06:18:06.308 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7437e-05 2024-09-14T06:18:06.308 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.308 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.308 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.308 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160569+0000", 2024-09-14T06:18:06.309 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028788700000000002 2024-09-14T06:18:06.309 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.309 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.309 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.309 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163121+0000", 2024-09-14T06:18:06.309 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025514090000000001 2024-09-14T06:18:06.309 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.309 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.309 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.309 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180410+0000", 2024-09-14T06:18:06.309 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.017289098999999999 2024-09-14T06:18:06.309 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.309 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.309 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.310 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180472+0000", 2024-09-14T06:18:06.310 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2547999999999998e-05 2024-09-14T06:18:06.310 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.310 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.310 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.310 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180508+0000", 2024-09-14T06:18:06.310 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5534000000000002e-05 2024-09-14T06:18:06.310 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.310 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.310 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.310 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.310 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.310 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13230 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.311 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:18:06.311 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.201591085999993, 2024-09-14T06:18:06.311 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081851558000000005, 2024-09-14T06:18:06.311 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.311 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.311 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.311 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.311 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.311 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.311 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13230 2024-09-14T06:18:06.311 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.311 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.311 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.312 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.312 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:18:06.312 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.312 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.312 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.312 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.312 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:18:06.312 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.312 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.312 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.312 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.312 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098791+0000", 2024-09-14T06:18:06.312 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8800000000000001e-07 2024-09-14T06:18:06.313 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.313 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.313 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.313 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098795+0000", 2024-09-14T06:18:06.313 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.391e-06 2024-09-14T06:18:06.313 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.313 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.313 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.313 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098804+0000", 2024-09-14T06:18:06.313 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3440000000000001e-06 2024-09-14T06:18:06.313 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.313 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.313 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.313 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098810+0000", 2024-09-14T06:18:06.314 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2339999999999999e-06 2024-09-14T06:18:06.314 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.314 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.314 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.314 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160930+0000", 2024-09-14T06:18:06.314 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062120018999999999 2024-09-14T06:18:06.314 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.314 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.314 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.314 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161017+0000", 2024-09-14T06:18:06.314 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6868999999999998e-05 2024-09-14T06:18:06.314 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.314 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.314 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.314 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161300+0000", 2024-09-14T06:18:06.315 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028349499999999998 2024-09-14T06:18:06.315 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.315 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.315 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.315 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164612+0000", 2024-09-14T06:18:06.315 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033119680000000002 2024-09-14T06:18:06.315 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.315 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.315 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.315 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180559+0000", 2024-09-14T06:18:06.315 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015946612999999998 2024-09-14T06:18:06.315 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.315 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.316 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.316 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180605+0000", 2024-09-14T06:18:06.316 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5890000000000003e-05 2024-09-14T06:18:06.316 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.316 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.316 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.316 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180642+0000", 2024-09-14T06:18:06.316 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:18:06.316 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.316 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.316 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.316 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.316 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.316 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13231 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.317 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:18:06.317 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.201328672000002, 2024-09-14T06:18:06.317 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081750685000000003, 2024-09-14T06:18:06.317 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.317 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.317 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.317 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.317 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.317 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.317 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13231 2024-09-14T06:18:06.317 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.317 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.317 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.317 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.318 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:18:06.318 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.318 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.318 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.318 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.318 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:18:06.318 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.318 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.318 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.318 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.319 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099053+0000", 2024-09-14T06:18:06.319 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:18:06.319 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.319 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.319 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.319 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099059+0000", 2024-09-14T06:18:06.319 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1789999999999999e-06 2024-09-14T06:18:06.319 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.319 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.319 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.319 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099072+0000", 2024-09-14T06:18:06.319 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3179999999999999e-05 2024-09-14T06:18:06.319 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.320 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.320 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.320 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099080+0000", 2024-09-14T06:18:06.320 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9030000000000004e-06 2024-09-14T06:18:06.320 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.320 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.320 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.320 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161681+0000", 2024-09-14T06:18:06.320 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062601641999999999 2024-09-14T06:18:06.320 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.320 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.320 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.320 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161766+0000", 2024-09-14T06:18:06.321 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4909999999999998e-05 2024-09-14T06:18:06.321 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.321 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.321 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.321 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162109+0000", 2024-09-14T06:18:06.321 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00034264899999999999 2024-09-14T06:18:06.321 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.321 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.321 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.321 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164615+0000", 2024-09-14T06:18:06.321 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025061979999999998 2024-09-14T06:18:06.322 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.322 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.322 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.322 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180718+0000", 2024-09-14T06:18:06.322 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016103240000000001 2024-09-14T06:18:06.322 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.322 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.322 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.322 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180768+0000", 2024-09-14T06:18:06.322 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9781999999999999e-05 2024-09-14T06:18:06.322 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.322 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.322 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.323 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180803+0000", 2024-09-14T06:18:06.323 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5021999999999998e-05 2024-09-14T06:18:06.323 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.323 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.323 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.323 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.323 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.323 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13232 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.323 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:18:06.323 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.201286898000006, 2024-09-14T06:18:06.323 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081861981, 2024-09-14T06:18:06.323 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.323 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.324 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.324 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.324 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.324 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.324 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13232 2024-09-14T06:18:06.324 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.324 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.324 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.324 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.324 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:18:06.324 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.324 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.325 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.325 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.325 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:18:06.325 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.325 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.325 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.325 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.325 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099095+0000", 2024-09-14T06:18:06.325 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:18:06.325 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.325 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.325 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.325 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099098+0000", 2024-09-14T06:18:06.326 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5420000000000001e-06 2024-09-14T06:18:06.326 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.326 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.326 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.326 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099106+0000", 2024-09-14T06:18:06.326 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.824e-06 2024-09-14T06:18:06.326 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.326 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.326 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.326 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099112+0000", 2024-09-14T06:18:06.326 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9789999999999998e-06 2024-09-14T06:18:06.326 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.326 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.327 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.327 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162576+0000", 2024-09-14T06:18:06.327 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.063463723999999999 2024-09-14T06:18:06.327 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.327 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.327 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.327 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162627+0000", 2024-09-14T06:18:06.327 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1530000000000003e-05 2024-09-14T06:18:06.327 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.327 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.327 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.327 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162854+0000", 2024-09-14T06:18:06.327 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000226159 2024-09-14T06:18:06.327 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.328 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.328 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.328 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166090+0000", 2024-09-14T06:18:06.328 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032362319999999999 2024-09-14T06:18:06.328 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.328 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.328 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.328 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180877+0000", 2024-09-14T06:18:06.328 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014786726 2024-09-14T06:18:06.328 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.328 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.328 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.328 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180920+0000", 2024-09-14T06:18:06.329 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3427000000000003e-05 2024-09-14T06:18:06.329 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.329 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.329 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.329 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180956+0000", 2024-09-14T06:18:06.329 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6195999999999997e-05 2024-09-14T06:18:06.329 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.329 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.329 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.329 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.329 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.329 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13233 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.329 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:18:06.330 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.201261634999994, 2024-09-14T06:18:06.330 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081924860000000002, 2024-09-14T06:18:06.330 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.330 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.330 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.330 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.330 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.330 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.330 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13233 2024-09-14T06:18:06.331 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.331 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.331 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.331 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.331 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:18:06.331 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.331 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.331 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.331 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.331 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:18:06.331 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.331 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.331 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.332 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.332 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099120+0000", 2024-09-14T06:18:06.332 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:18:06.332 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.332 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.332 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.332 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099122+0000", 2024-09-14T06:18:06.332 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:18:06.332 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.332 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.332 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.332 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099126+0000", 2024-09-14T06:18:06.332 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2540000000000003e-06 2024-09-14T06:18:06.333 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.333 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.333 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.333 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099130+0000", 2024-09-14T06:18:06.333 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4060000000000001e-06 2024-09-14T06:18:06.333 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.333 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.333 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.333 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163169+0000", 2024-09-14T06:18:06.333 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064039415000000002 2024-09-14T06:18:06.333 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.333 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.333 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.334 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163260+0000", 2024-09-14T06:18:06.334 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1038000000000005e-05 2024-09-14T06:18:06.334 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.334 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.334 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.334 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163640+0000", 2024-09-14T06:18:06.334 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037954699999999999 2024-09-14T06:18:06.334 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.334 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.334 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.334 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166093+0000", 2024-09-14T06:18:06.335 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024533229999999999 2024-09-14T06:18:06.335 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.335 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.335 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.335 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180993+0000", 2024-09-14T06:18:06.335 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01489997 2024-09-14T06:18:06.335 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.335 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.335 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.336 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181020+0000", 2024-09-14T06:18:06.336 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6806999999999999e-05 2024-09-14T06:18:06.336 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.336 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.336 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.337 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181044+0000", 2024-09-14T06:18:06.337 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4405000000000001e-05 2024-09-14T06:18:06.337 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.337 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.337 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.337 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.337 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.338 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13234 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.338 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:18:06.338 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.201219429999995, 2024-09-14T06:18:06.338 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082015597999999995, 2024-09-14T06:18:06.338 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.338 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.339 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.339 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.339 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.339 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.339 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13234 2024-09-14T06:18:06.339 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.340 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.340 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.340 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.340 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:18:06.340 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.340 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.341 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.341 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.341 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:18:06.341 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.341 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.341 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.341 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.342 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:18:06.342 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0999999999999998e-07 2024-09-14T06:18:06.342 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.342 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.342 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.342 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099166+0000", 2024-09-14T06:18:06.342 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:18:06.343 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.343 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.343 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.343 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099171+0000", 2024-09-14T06:18:06.343 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6949999999999997e-06 2024-09-14T06:18:06.343 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.343 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.343 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.344 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099175+0000", 2024-09-14T06:18:06.344 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9029999999999997e-06 2024-09-14T06:18:06.344 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.344 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.344 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.345 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164063+0000", 2024-09-14T06:18:06.345 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064887608999999999 2024-09-14T06:18:06.345 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.345 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.345 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.345 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164111+0000", 2024-09-14T06:18:06.346 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7648999999999997e-05 2024-09-14T06:18:06.346 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.346 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.346 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.346 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164351+0000", 2024-09-14T06:18:06.346 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024072899999999999 2024-09-14T06:18:06.346 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.347 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.347 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.347 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167561+0000", 2024-09-14T06:18:06.347 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032094179999999999 2024-09-14T06:18:06.347 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.347 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.347 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.348 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181093+0000", 2024-09-14T06:18:06.348 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013532245 2024-09-14T06:18:06.348 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.348 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.348 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.348 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181139+0000", 2024-09-14T06:18:06.348 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6483999999999997e-05 2024-09-14T06:18:06.349 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.349 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.349 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.349 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181177+0000", 2024-09-14T06:18:06.349 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7861000000000002e-05 2024-09-14T06:18:06.349 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.350 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.350 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.350 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.350 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.350 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13235 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.350 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:18:06.350 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.201112155999994, 2024-09-14T06:18:06.351 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081988088000000001, 2024-09-14T06:18:06.351 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.351 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.351 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.351 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.351 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.351 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.351 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13235 2024-09-14T06:18:06.352 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.352 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.352 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.352 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.352 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:18:06.352 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.353 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.353 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.353 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.353 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:18:06.353 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.353 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.353 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.354 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.354 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099270+0000", 2024-09-14T06:18:06.354 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7400000000000003e-07 2024-09-14T06:18:06.354 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.354 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.354 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.354 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099276+0000", 2024-09-14T06:18:06.355 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:18:06.355 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.355 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.355 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.355 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099284+0000", 2024-09-14T06:18:06.355 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7979999999999995e-06 2024-09-14T06:18:06.355 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.356 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.356 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.356 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099289+0000", 2024-09-14T06:18:06.356 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:18:06.356 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.356 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.356 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.357 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164664+0000", 2024-09-14T06:18:06.357 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065374901999999999 2024-09-14T06:18:06.357 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.357 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.357 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.357 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164755+0000", 2024-09-14T06:18:06.357 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0790000000000003e-05 2024-09-14T06:18:06.358 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.358 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.358 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.358 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165143+0000", 2024-09-14T06:18:06.358 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000387904 2024-09-14T06:18:06.359 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.359 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.359 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.359 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167564+0000", 2024-09-14T06:18:06.359 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024208989999999998 2024-09-14T06:18:06.359 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.359 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.360 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.360 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181212+0000", 2024-09-14T06:18:06.360 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013648664 2024-09-14T06:18:06.360 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.360 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.360 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.361 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181237+0000", 2024-09-14T06:18:06.361 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5177999999999998e-05 2024-09-14T06:18:06.361 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.361 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.361 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.361 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181257+0000", 2024-09-14T06:18:06.361 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9660000000000002e-05 2024-09-14T06:18:06.362 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.362 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.362 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.362 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.362 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.362 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13236 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.362 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:18:06.362 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.201005336999998, 2024-09-14T06:18:06.363 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082010939000000005, 2024-09-14T06:18:06.363 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.363 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.363 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.363 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.363 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.363 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.364 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13236 2024-09-14T06:18:06.364 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.364 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.364 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.364 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.364 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:18:06.365 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.365 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.365 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.365 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.366 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:18:06.366 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.366 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.366 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.366 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.366 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:18:06.367 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.61e-07 2024-09-14T06:18:06.367 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.367 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.367 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.367 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099382+0000", 2024-09-14T06:18:06.367 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3589999999999999e-06 2024-09-14T06:18:06.368 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.368 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.368 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.368 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099388+0000", 2024-09-14T06:18:06.368 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9619999999999998e-06 2024-09-14T06:18:06.368 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.369 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.369 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.369 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099393+0000", 2024-09-14T06:18:06.369 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3410000000000002e-06 2024-09-14T06:18:06.369 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.370 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.370 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.370 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165553+0000", 2024-09-14T06:18:06.370 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066159604999999996 2024-09-14T06:18:06.370 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.371 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.371 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.371 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165603+0000", 2024-09-14T06:18:06.371 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0427999999999999e-05 2024-09-14T06:18:06.372 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.372 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.372 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.372 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165827+0000", 2024-09-14T06:18:06.372 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022358299999999999 2024-09-14T06:18:06.372 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.373 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.373 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.373 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177005+0000", 2024-09-14T06:18:06.373 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011178044 2024-09-14T06:18:06.373 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.373 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.373 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.374 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181302+0000", 2024-09-14T06:18:06.374 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042970669999999999 2024-09-14T06:18:06.374 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.374 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.374 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.374 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181349+0000", 2024-09-14T06:18:06.374 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7287999999999999e-05 2024-09-14T06:18:06.375 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.375 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.375 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.375 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181387+0000", 2024-09-14T06:18:06.375 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7601000000000001e-05 2024-09-14T06:18:06.375 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.375 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.376 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.376 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.376 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.376 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13237 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.376 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:18:06.376 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.200847190000005, 2024-09-14T06:18:06.376 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081927299999999995, 2024-09-14T06:18:06.377 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.377 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.377 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.377 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.377 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.377 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.378 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13237 2024-09-14T06:18:06.378 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.378 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.378 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.378 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.378 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:18:06.379 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.379 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.379 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.379 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.380 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:18:06.380 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.380 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.380 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.380 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.380 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099535+0000", 2024-09-14T06:18:06.380 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:18:06.381 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.381 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.382 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.382 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099538+0000", 2024-09-14T06:18:06.382 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:18:06.382 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.382 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.382 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.382 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099545+0000", 2024-09-14T06:18:06.382 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:18:06.382 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.382 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.382 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.382 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099551+0000", 2024-09-14T06:18:06.382 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1770000000000003e-06 2024-09-14T06:18:06.383 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.383 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.383 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.383 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166131+0000", 2024-09-14T06:18:06.383 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066580499000000001 2024-09-14T06:18:06.383 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.383 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.383 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.383 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166221+0000", 2024-09-14T06:18:06.383 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0000000000000006e-05 2024-09-14T06:18:06.383 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.383 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.383 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.383 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166622+0000", 2024-09-14T06:18:06.384 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040029799999999999 2024-09-14T06:18:06.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.384 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177008+0000", 2024-09-14T06:18:06.384 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010386772000000001 2024-09-14T06:18:06.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.384 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181420+0000", 2024-09-14T06:18:06.384 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044119190000000003 2024-09-14T06:18:06.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.384 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.384 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.385 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181443+0000", 2024-09-14T06:18:06.385 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2603999999999999e-05 2024-09-14T06:18:06.385 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.385 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.385 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.385 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181461+0000", 2024-09-14T06:18:06.385 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8405000000000001e-05 2024-09-14T06:18:06.385 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.385 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.385 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.385 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.385 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.385 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13238 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.386 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:18:06.386 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.200810881999999, 2024-09-14T06:18:06.386 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082014927000000001, 2024-09-14T06:18:06.386 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.386 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.386 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.386 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.386 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.386 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.386 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13238 2024-09-14T06:18:06.386 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.386 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.386 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.387 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.387 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:18:06.387 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.387 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.387 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.387 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.387 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:18:06.387 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.387 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.387 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.387 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.387 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099571+0000", 2024-09-14T06:18:06.387 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9699999999999996e-07 2024-09-14T06:18:06.387 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.388 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.388 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.388 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099577+0000", 2024-09-14T06:18:06.388 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2890000000000003e-06 2024-09-14T06:18:06.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.388 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.388 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.388 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099587+0000", 2024-09-14T06:18:06.388 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0002e-05 2024-09-14T06:18:06.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.388 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.388 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.389 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099593+0000", 2024-09-14T06:18:06.389 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4539999999999998e-06 2024-09-14T06:18:06.389 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.389 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.389 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.389 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167024+0000", 2024-09-14T06:18:06.389 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067431480000000002 2024-09-14T06:18:06.389 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.389 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.389 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.389 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167074+0000", 2024-09-14T06:18:06.389 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9996e-05 2024-09-14T06:18:06.389 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.389 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.390 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167288+0000", 2024-09-14T06:18:06.390 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021410900000000001 2024-09-14T06:18:06.390 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.390 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.390 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177997+0000", 2024-09-14T06:18:06.390 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010708756 2024-09-14T06:18:06.390 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.390 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.390 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.390 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181508+0000", 2024-09-14T06:18:06.391 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003511363 2024-09-14T06:18:06.391 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.391 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.391 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.391 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181552+0000", 2024-09-14T06:18:06.391 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3609999999999998e-05 2024-09-14T06:18:06.391 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.391 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.391 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.391 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181585+0000", 2024-09-14T06:18:06.391 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3271e-05 2024-09-14T06:18:06.391 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.391 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.391 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.392 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.392 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.392 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13239 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.392 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:18:06.392 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.200719063999998, 2024-09-14T06:18:06.392 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081998772999999997, 2024-09-14T06:18:06.392 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.392 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.392 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.392 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.392 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.392 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.393 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13239 2024-09-14T06:18:06.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.393 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.393 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.393 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:18:06.393 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.393 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.393 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.393 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:18:06.393 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.394 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.394 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.394 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099663+0000", 2024-09-14T06:18:06.394 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6100000000000001e-07 2024-09-14T06:18:06.394 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.394 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.394 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.394 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099666+0000", 2024-09-14T06:18:06.394 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.056e-06 2024-09-14T06:18:06.394 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.394 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.394 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.394 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099672+0000", 2024-09-14T06:18:06.394 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8120000000000002e-06 2024-09-14T06:18:06.395 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.395 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.395 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.395 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099676+0000", 2024-09-14T06:18:06.395 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:18:06.395 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.395 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.395 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.395 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167602+0000", 2024-09-14T06:18:06.395 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067925867000000001 2024-09-14T06:18:06.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.396 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.396 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.396 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167693+0000", 2024-09-14T06:18:06.396 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0736000000000007e-05 2024-09-14T06:18:06.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.396 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.396 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.396 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168065+0000", 2024-09-14T06:18:06.396 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037245399999999998 2024-09-14T06:18:06.396 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.396 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.396 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.397 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178003+0000", 2024-09-14T06:18:06.397 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0099372360000000003 2024-09-14T06:18:06.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.397 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.397 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181621+0000", 2024-09-14T06:18:06.397 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036178489999999998 2024-09-14T06:18:06.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.397 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.397 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181643+0000", 2024-09-14T06:18:06.397 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2155e-05 2024-09-14T06:18:06.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.397 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.398 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.398 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181661+0000", 2024-09-14T06:18:06.398 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8147000000000001e-05 2024-09-14T06:18:06.398 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.398 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.398 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.398 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.398 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.398 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13240 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.398 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:18:06.398 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.200599971000003, 2024-09-14T06:18:06.398 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081942607000000001, 2024-09-14T06:18:06.398 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.398 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.398 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.399 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.399 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.399 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.399 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13240 2024-09-14T06:18:06.399 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.399 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.399 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.399 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.399 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:18:06.399 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.399 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.399 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.399 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.399 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:18:06.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.400 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.400 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.400 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099782+0000", 2024-09-14T06:18:06.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3900000000000004e-07 2024-09-14T06:18:06.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.400 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.400 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.400 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099787+0000", 2024-09-14T06:18:06.400 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6189999999999999e-06 2024-09-14T06:18:06.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.400 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.401 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.401 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099793+0000", 2024-09-14T06:18:06.401 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.998e-06 2024-09-14T06:18:06.401 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.401 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.401 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.401 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099797+0000", 2024-09-14T06:18:06.401 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8449999999999996e-06 2024-09-14T06:18:06.401 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.401 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.401 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.401 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168494+0000", 2024-09-14T06:18:06.401 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.068696500999999993 2024-09-14T06:18:06.402 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.402 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.402 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.402 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168590+0000", 2024-09-14T06:18:06.402 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6733999999999999e-05 2024-09-14T06:18:06.402 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.402 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.402 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.402 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168973+0000", 2024-09-14T06:18:06.402 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038229399999999998 2024-09-14T06:18:06.402 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.402 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.402 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.403 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178013+0000", 2024-09-14T06:18:06.403 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0090397199999999994 2024-09-14T06:18:06.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.403 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.403 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.403 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181683+0000", 2024-09-14T06:18:06.403 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036705819999999999 2024-09-14T06:18:06.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.403 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.403 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.404 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181705+0000", 2024-09-14T06:18:06.404 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1551000000000001e-05 2024-09-14T06:18:06.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.404 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.404 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181724+0000", 2024-09-14T06:18:06.404 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9123999999999999e-05 2024-09-14T06:18:06.404 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.404 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.404 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.404 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.404 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13243 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.404 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:18:06.405 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.200066241000002, 2024-09-14T06:18:06.405 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081767815999999993, 2024-09-14T06:18:06.405 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.405 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.405 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.405 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.405 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.405 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.405 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13243 2024-09-14T06:18:06.405 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.405 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.405 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.405 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.405 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:18:06.405 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.406 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.406 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.406 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.406 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:18:06.406 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.406 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.406 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.406 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.406 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:18:06.406 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1e-07 2024-09-14T06:18:06.406 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.406 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.406 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.406 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100317+0000", 2024-09-14T06:18:06.406 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.092e-06 2024-09-14T06:18:06.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.407 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.407 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.407 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100321+0000", 2024-09-14T06:18:06.407 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7809999999999999e-06 2024-09-14T06:18:06.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.407 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.407 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.407 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100324+0000", 2024-09-14T06:18:06.407 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.117e-06 2024-09-14T06:18:06.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.407 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.407 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.407 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171113+0000", 2024-09-14T06:18:06.407 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.070788816000000004 2024-09-14T06:18:06.408 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.408 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.408 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.408 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171202+0000", 2024-09-14T06:18:06.408 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8843000000000003e-05 2024-09-14T06:18:06.408 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.408 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.408 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.408 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171584+0000", 2024-09-14T06:18:06.408 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038231099999999998 2024-09-14T06:18:06.408 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.408 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.408 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.408 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178101+0000", 2024-09-14T06:18:06.409 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0065168329999999997 2024-09-14T06:18:06.409 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.409 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.409 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.409 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182041+0000", 2024-09-14T06:18:06.409 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039401810000000001 2024-09-14T06:18:06.409 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.409 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.409 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.409 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182065+0000", 2024-09-14T06:18:06.409 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3623000000000001e-05 2024-09-14T06:18:06.409 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.409 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.409 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.410 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182083+0000", 2024-09-14T06:18:06.410 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7909000000000001e-05 2024-09-14T06:18:06.410 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.410 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.410 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.410 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.410 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.410 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13244 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.410 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:18:06.410 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.200051420999998, 2024-09-14T06:18:06.410 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081837800000000002, 2024-09-14T06:18:06.410 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.410 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.410 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.410 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.411 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.411 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.411 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13244 2024-09-14T06:18:06.411 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.411 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.411 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.411 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.411 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:18:06.411 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.411 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.411 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.411 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.411 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:18:06.411 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.411 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.412 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.412 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.412 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:18:06.412 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:18:06.412 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.412 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.412 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.412 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100337+0000", 2024-09-14T06:18:06.412 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.939e-06 2024-09-14T06:18:06.412 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.412 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.412 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.412 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100341+0000", 2024-09-14T06:18:06.412 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2320000000000003e-06 2024-09-14T06:18:06.413 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.413 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.413 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.413 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100344+0000", 2024-09-14T06:18:06.413 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0759999999999999e-06 2024-09-14T06:18:06.413 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.413 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.413 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.413 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171996+0000", 2024-09-14T06:18:06.413 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.071651919999999994 2024-09-14T06:18:06.413 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.413 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.413 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.413 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172079+0000", 2024-09-14T06:18:06.414 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2579999999999999e-05 2024-09-14T06:18:06.414 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.414 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.414 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.414 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172457+0000", 2024-09-14T06:18:06.414 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037785100000000001 2024-09-14T06:18:06.414 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.414 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.414 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.414 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178104+0000", 2024-09-14T06:18:06.414 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056471730000000001 2024-09-14T06:18:06.414 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.414 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.414 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.415 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182123+0000", 2024-09-14T06:18:06.415 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040190019999999998 2024-09-14T06:18:06.415 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.415 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.415 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.415 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182147+0000", 2024-09-14T06:18:06.415 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4241000000000001e-05 2024-09-14T06:18:06.415 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.415 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.415 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.415 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182168+0000", 2024-09-14T06:18:06.415 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0492999999999998e-05 2024-09-14T06:18:06.415 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.415 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.416 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.416 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.416 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.416 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13245 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.416 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:18:06.416 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.200031913999993, 2024-09-14T06:18:06.416 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081893899000000006, 2024-09-14T06:18:06.416 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.416 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.416 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.416 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.416 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.416 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.417 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13245 2024-09-14T06:18:06.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.417 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.417 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.417 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.417 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:18:06.417 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.417 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.417 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.417 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:18:06.417 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.417 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.417 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.418 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100350+0000", 2024-09-14T06:18:06.418 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5499999999999999e-07 2024-09-14T06:18:06.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.418 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.418 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.418 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100353+0000", 2024-09-14T06:18:06.418 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:18:06.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.418 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.418 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.418 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100357+0000", 2024-09-14T06:18:06.418 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8500000000000004e-06 2024-09-14T06:18:06.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.418 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.419 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.419 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100360+0000", 2024-09-14T06:18:06.419 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8700000000000001e-06 2024-09-14T06:18:06.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.419 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.419 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.419 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172862+0000", 2024-09-14T06:18:06.419 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.072501992000000001 2024-09-14T06:18:06.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.419 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.419 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.419 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172950+0000", 2024-09-14T06:18:06.419 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8988999999999998e-05 2024-09-14T06:18:06.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.420 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.420 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.420 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173324+0000", 2024-09-14T06:18:06.420 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037366199999999997 2024-09-14T06:18:06.420 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.420 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.420 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.420 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178107+0000", 2024-09-14T06:18:06.420 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047830520000000003 2024-09-14T06:18:06.420 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.420 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.420 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.420 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182202+0000", 2024-09-14T06:18:06.421 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040949280000000003 2024-09-14T06:18:06.421 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.421 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.421 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.421 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182226+0000", 2024-09-14T06:18:06.421 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3535000000000001e-05 2024-09-14T06:18:06.421 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.421 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.421 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.421 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182243+0000", 2024-09-14T06:18:06.421 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7445e-05 2024-09-14T06:18:06.421 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.421 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.421 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.422 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.422 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.422 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13246 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.422 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:18:06.422 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.200017591999995, 2024-09-14T06:18:06.422 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081962276000000001, 2024-09-14T06:18:06.422 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.422 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.422 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.422 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.422 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.422 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.422 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13246 2024-09-14T06:18:06.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.423 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.423 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.423 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.423 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:18:06.423 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.423 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.423 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.423 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:18:06.423 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.423 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.424 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.424 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:18:06.424 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0800000000000001e-07 2024-09-14T06:18:06.424 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.424 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.424 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.424 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100367+0000", 2024-09-14T06:18:06.424 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8439999999999998e-06 2024-09-14T06:18:06.424 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.424 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.424 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.424 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100372+0000", 2024-09-14T06:18:06.424 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4009999999999996e-06 2024-09-14T06:18:06.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.425 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.425 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.425 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100378+0000", 2024-09-14T06:18:06.425 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7180000000000001e-06 2024-09-14T06:18:06.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.425 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.425 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.425 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173780+0000", 2024-09-14T06:18:06.425 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.073401991 2024-09-14T06:18:06.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.425 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.425 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.425 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173879+0000", 2024-09-14T06:18:06.425 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9449000000000003e-05 2024-09-14T06:18:06.426 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.426 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.426 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.426 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174240+0000", 2024-09-14T06:18:06.426 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036118100000000002 2024-09-14T06:18:06.426 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.426 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.426 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.426 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178109+0000", 2024-09-14T06:18:06.426 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038690619999999999 2024-09-14T06:18:06.426 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.426 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.426 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.426 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182281+0000", 2024-09-14T06:18:06.426 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041715900000000002 2024-09-14T06:18:06.427 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.427 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.427 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.427 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182306+0000", 2024-09-14T06:18:06.427 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4477e-05 2024-09-14T06:18:06.427 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.427 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.427 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.427 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182326+0000", 2024-09-14T06:18:06.427 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0254999999999998e-05 2024-09-14T06:18:06.427 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.427 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.427 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.427 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.428 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.428 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13247 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.428 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:18:06.428 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.199920414999994, 2024-09-14T06:18:06.428 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966593000000004, 2024-09-14T06:18:06.428 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.428 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.428 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.428 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.428 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.428 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.428 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13247 2024-09-14T06:18:06.428 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.429 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.429 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.429 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.429 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:18:06.429 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.429 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.429 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.429 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.429 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:18:06.429 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.429 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.429 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.429 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.430 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100462+0000", 2024-09-14T06:18:06.430 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9800000000000002e-07 2024-09-14T06:18:06.430 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.430 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.430 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.430 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100466+0000", 2024-09-14T06:18:06.430 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4039999999999997e-06 2024-09-14T06:18:06.430 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.430 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.430 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.430 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100475+0000", 2024-09-14T06:18:06.430 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3680000000000008e-06 2024-09-14T06:18:06.430 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.430 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.431 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.431 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100482+0000", 2024-09-14T06:18:06.431 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8630000000000002e-06 2024-09-14T06:18:06.431 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.431 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.431 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.431 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174669+0000", 2024-09-14T06:18:06.431 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074186849999999999 2024-09-14T06:18:06.431 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.431 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.431 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.431 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174758+0000", 2024-09-14T06:18:06.431 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9021e-05 2024-09-14T06:18:06.432 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.432 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.432 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.432 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175137+0000", 2024-09-14T06:18:06.432 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378511 2024-09-14T06:18:06.432 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.432 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.432 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.432 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178158+0000", 2024-09-14T06:18:06.432 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003020956 2024-09-14T06:18:06.432 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.432 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.432 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.433 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182386+0000", 2024-09-14T06:18:06.433 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042278840000000003 2024-09-14T06:18:06.433 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.433 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.433 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.433 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182410+0000", 2024-09-14T06:18:06.433 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4026e-05 2024-09-14T06:18:06.433 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.433 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.433 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.433 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182427+0000", 2024-09-14T06:18:06.433 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7711999999999999e-05 2024-09-14T06:18:06.433 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.433 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.434 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.434 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.434 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.434 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13248 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.434 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:18:06.434 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.199838334000006, 2024-09-14T06:18:06.434 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966527999999997, 2024-09-14T06:18:06.434 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.434 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.434 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.434 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.434 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.434 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.435 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13248 2024-09-14T06:18:06.435 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.435 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.435 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.435 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.435 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:18:06.435 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.435 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.435 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.435 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.435 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:18:06.435 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.435 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.436 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.436 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.436 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:18:06.436 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:18:06.436 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.436 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.436 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.436 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100548+0000", 2024-09-14T06:18:06.436 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:18:06.436 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.436 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.436 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.436 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100557+0000", 2024-09-14T06:18:06.437 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:18:06.437 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.437 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.437 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.437 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100563+0000", 2024-09-14T06:18:06.437 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2519999999999996e-06 2024-09-14T06:18:06.437 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.437 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.437 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.437 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175572+0000", 2024-09-14T06:18:06.437 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075008459 2024-09-14T06:18:06.437 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.437 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.438 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.438 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175656+0000", 2024-09-14T06:18:06.438 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3906999999999995e-05 2024-09-14T06:18:06.438 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.438 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.438 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.438 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176010+0000", 2024-09-14T06:18:06.438 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00035447499999999998 2024-09-14T06:18:06.438 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.438 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.438 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.438 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178304+0000", 2024-09-14T06:18:06.438 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0022941379999999998 2024-09-14T06:18:06.438 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.439 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.439 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.439 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182467+0000", 2024-09-14T06:18:06.439 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041632020000000004 2024-09-14T06:18:06.439 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.439 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.439 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.439 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182491+0000", 2024-09-14T06:18:06.439 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3733000000000001e-05 2024-09-14T06:18:06.439 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.439 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.439 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.440 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182509+0000", 2024-09-14T06:18:06.440 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8113000000000001e-05 2024-09-14T06:18:06.440 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.440 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.440 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.440 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.440 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.440 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13249 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.440 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:18:06.440 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.199803399999993, 2024-09-14T06:18:06.440 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082037225000000005, 2024-09-14T06:18:06.440 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.440 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.440 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.441 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.441 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.441 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.441 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13249 2024-09-14T06:18:06.441 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.441 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.441 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.441 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.441 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:18:06.441 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.441 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.441 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.441 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.441 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:18:06.441 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.442 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.442 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.442 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.442 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:18:06.442 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4600000000000005e-07 2024-09-14T06:18:06.442 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.442 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.442 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.442 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100584+0000", 2024-09-14T06:18:06.442 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.947e-06 2024-09-14T06:18:06.442 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.442 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.442 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.442 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100593+0000", 2024-09-14T06:18:06.443 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9760000000000001e-06 2024-09-14T06:18:06.443 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.443 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.443 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.443 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100597+0000", 2024-09-14T06:18:06.443 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9689999999999996e-06 2024-09-14T06:18:06.443 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.443 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.443 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.443 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176461+0000", 2024-09-14T06:18:06.443 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075864132000000001 2024-09-14T06:18:06.443 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.443 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.444 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.444 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176516+0000", 2024-09-14T06:18:06.444 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4416999999999998e-05 2024-09-14T06:18:06.444 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.444 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.444 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.444 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176734+0000", 2024-09-14T06:18:06.444 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021859399999999999 2024-09-14T06:18:06.444 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.444 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.444 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.444 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178308+0000", 2024-09-14T06:18:06.444 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015733660000000001 2024-09-14T06:18:06.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.445 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.445 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182573+0000", 2024-09-14T06:18:06.445 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042649200000000002 2024-09-14T06:18:06.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.445 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.445 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182597+0000", 2024-09-14T06:18:06.445 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4355e-05 2024-09-14T06:18:06.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.445 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.445 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.445 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182615+0000", 2024-09-14T06:18:06.446 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8003e-05 2024-09-14T06:18:06.446 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.446 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.446 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.446 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.446 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.446 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13250 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:06.446 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:18:06.446 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 90.199638794999998, 2024-09-14T06:18:06.446 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081983890000000004, 2024-09-14T06:18:06.446 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.446 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.446 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.446 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.446 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.447 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.447 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13250 2024-09-14T06:18:06.447 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.447 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.447 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.447 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.447 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:18:06.447 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.447 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.447 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.447 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.447 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:18:06.447 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.448 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.448 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.448 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100743+0000", 2024-09-14T06:18:06.448 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:18:06.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.448 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.448 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.448 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100753+0000", 2024-09-14T06:18:06.448 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9979999999999998e-06 2024-09-14T06:18:06.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.448 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.448 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.449 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100772+0000", 2024-09-14T06:18:06.449 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8664e-05 2024-09-14T06:18:06.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.449 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.449 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100779+0000", 2024-09-14T06:18:06.449 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.046e-06 2024-09-14T06:18:06.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.449 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.449 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177049+0000", 2024-09-14T06:18:06.449 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076270819000000004 2024-09-14T06:18:06.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.449 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.450 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.450 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177140+0000", 2024-09-14T06:18:06.450 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0746000000000002e-05 2024-09-14T06:18:06.450 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.450 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.450 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:06.450 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177519+0000", 2024-09-14T06:18:06.450 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378656 2024-09-14T06:18:06.450 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.450 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.450 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.450 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178349+0000", 2024-09-14T06:18:06.450 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00083061599999999995 2024-09-14T06:18:06.451 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.451 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.451 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.451 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182679+0000", 2024-09-14T06:18:06.451 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043298379999999999 2024-09-14T06:18:06.451 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.451 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.451 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.451 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182705+0000", 2024-09-14T06:18:06.451 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5602999999999999e-05 2024-09-14T06:18:06.451 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.451 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.451 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.451 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182726+0000", 2024-09-14T06:18:06.452 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1297e-05 2024-09-14T06:18:06.452 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.452 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.452 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.452 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.452 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.452 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:06.454 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T06:18:06.454 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:18:06.455 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T06:18:06.455 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:18:06.449+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:18:06.554 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:06.549+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:06.596 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:06.596 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:06.596 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:18:06.596 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:06.610 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:18:06.656 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:06.655+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:06.732 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:06.732 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:06.732 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:18:06.732 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:18:06.732 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:06.746 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:18:06.756 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:06.753+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:06.857 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:06.853+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:06.913 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:06.914 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:18:06.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:18:06.914 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:18:06.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.914 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:30547 3.1f 3:f84dfb53:::rbd_data.11529eb229bd.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write 16226533~50642] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T06:18:06.914 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:18:06.914 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 67.721125291999996, 2024-09-14T06:18:06.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0079770020000001, 2024-09-14T06:18:06.914 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.914 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.914 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.914 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.915 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.915 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.915 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 30547 2024-09-14T06:18:06.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.915 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.915 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.915 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:18:06.915 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.915 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.915 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:18:06.915 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189811+0000", 2024-09-14T06:18:06.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2060000000000001e-06 2024-09-14T06:18:06.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189871+0000", 2024-09-14T06:18:06.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9862999999999999e-05 2024-09-14T06:18:06.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189899+0000", 2024-09-14T06:18:06.917 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7849000000000001e-05 2024-09-14T06:18:06.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.917 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.917 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.917 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189915+0000", 2024-09-14T06:18:06.917 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:18:06.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.917 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.917 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.917 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190024+0000", 2024-09-14T06:18:06.917 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000109202 2024-09-14T06:18:06.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.917 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.918 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:18:06.918 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190041+0000", 2024-09-14T06:18:06.918 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7665000000000001e-05 2024-09-14T06:18:06.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.918 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.918 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.918 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188512+0000", 2024-09-14T06:18:06.918 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.99847100700000002 2024-09-14T06:18:06.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.918 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.918 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for missing object", 2024-09-14T06:18:06.919 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188646+0000", 2024-09-14T06:18:06.919 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000133986 2024-09-14T06:18:06.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.919 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.919 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.919 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193097+0000", 2024-09-14T06:18:06.919 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044507059999999996 2024-09-14T06:18:06.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.920 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.920 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193183+0000", 2024-09-14T06:18:06.920 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5496999999999998e-05 2024-09-14T06:18:06.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.920 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:18:06.920 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193307+0000", 2024-09-14T06:18:06.920 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000124611 2024-09-14T06:18:06.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.920 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.921 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.194318+0000", 2024-09-14T06:18:06.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0010106 2024-09-14T06:18:06.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.921 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.921 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197670+0000", 2024-09-14T06:18:06.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033523170000000001 2024-09-14T06:18:06.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.921 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.922 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197712+0000", 2024-09-14T06:18:06.922 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1681000000000003e-05 2024-09-14T06:18:06.922 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.922 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.922 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.922 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197786+0000", 2024-09-14T06:18:06.922 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4148999999999998e-05 2024-09-14T06:18:06.922 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.922 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.922 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.922 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.922 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.922 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32022 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:06.923 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:18:06.923 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 63.042745081, 2024-09-14T06:18:06.923 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095598104000000003, 2024-09-14T06:18:06.923 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.923 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.923 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:06.923 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.923 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.923 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.923 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32022 2024-09-14T06:18:06.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.923 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.923 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.923 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.924 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:18:06.924 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.924 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.924 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.924 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:18:06.924 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.924 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.924 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.924 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868190+0000", 2024-09-14T06:18:06.924 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6000000000000003e-07 2024-09-14T06:18:06.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868243+0000", 2024-09-14T06:18:06.925 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3443999999999999e-05 2024-09-14T06:18:06.925 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868260+0000", 2024-09-14T06:18:06.926 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6799e-05 2024-09-14T06:18:06.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.926 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.926 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868271+0000", 2024-09-14T06:18:06.926 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0523000000000001e-05 2024-09-14T06:18:06.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.926 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.926 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963502+0000", 2024-09-14T06:18:06.926 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095231466000000001 2024-09-14T06:18:06.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.926 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.927 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963636+0000", 2024-09-14T06:18:06.927 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013388699999999999 2024-09-14T06:18:06.927 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.927 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.927 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.927 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963787+0000", 2024-09-14T06:18:06.927 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000151225 2024-09-14T06:18:06.927 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.927 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.927 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.927 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.928 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.928 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32023 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5816320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:06.928 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:18:06.928 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 63.042649955000002, 2024-09-14T06:18:06.928 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095879753999999998, 2024-09-14T06:18:06.928 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.928 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.928 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:06.928 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.928 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.928 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.928 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32023 2024-09-14T06:18:06.928 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.929 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:18:06.929 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.929 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:18:06.929 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.929 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868285+0000", 2024-09-14T06:18:06.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6299999999999993e-07 2024-09-14T06:18:06.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.930 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868288+0000", 2024-09-14T06:18:06.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.18e-06 2024-09-14T06:18:06.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.930 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868298+0000", 2024-09-14T06:18:06.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0029e-05 2024-09-14T06:18:06.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.931 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.931 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.931 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868307+0000", 2024-09-14T06:18:06.931 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7229999999999993e-06 2024-09-14T06:18:06.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.931 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.931 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.931 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963871+0000", 2024-09-14T06:18:06.931 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095563876000000006 2024-09-14T06:18:06.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.931 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.931 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.931 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964015+0000", 2024-09-14T06:18:06.932 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143573 2024-09-14T06:18:06.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.932 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.932 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.932 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964164+0000", 2024-09-14T06:18:06.932 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014940999999999999 2024-09-14T06:18:06.932 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.932 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.932 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.932 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.932 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32024 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:06.932 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:18:06.932 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 63.042616811000002, 2024-09-14T06:18:06.932 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096230231999999999, 2024-09-14T06:18:06.933 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.933 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.933 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:06.933 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.933 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.933 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.933 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32024 2024-09-14T06:18:06.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.933 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.933 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:18:06.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:18:06.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868318+0000", 2024-09-14T06:18:06.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1699999999999997e-07 2024-09-14T06:18:06.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.935 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868321+0000", 2024-09-14T06:18:06.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6769999999999999e-06 2024-09-14T06:18:06.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.935 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.935 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.935 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868334+0000", 2024-09-14T06:18:06.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2928999999999999e-05 2024-09-14T06:18:06.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.935 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.935 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.935 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868341+0000", 2024-09-14T06:18:06.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9800000000000001e-06 2024-09-14T06:18:06.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.935 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.935 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.936 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964258+0000", 2024-09-14T06:18:06.936 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095917099000000006 2024-09-14T06:18:06.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.936 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.936 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.936 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964392+0000", 2024-09-14T06:18:06.936 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013396999999999999 2024-09-14T06:18:06.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.936 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.936 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.936 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964548+0000", 2024-09-14T06:18:06.936 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015585999999999999 2024-09-14T06:18:06.936 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.936 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.937 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.937 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32025 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:06.937 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:18:06.937 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 63.042584155, 2024-09-14T06:18:06.937 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096576591000000003, 2024-09-14T06:18:06.937 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.937 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.937 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:06.937 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.937 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.937 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.938 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32025 2024-09-14T06:18:06.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.938 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:18:06.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:18:06.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868351+0000", 2024-09-14T06:18:06.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1600000000000001e-07 2024-09-14T06:18:06.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868353+0000", 2024-09-14T06:18:06.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.627e-06 2024-09-14T06:18:06.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868364+0000", 2024-09-14T06:18:06.940 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0893000000000001e-05 2024-09-14T06:18:06.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.940 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.940 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.940 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868381+0000", 2024-09-14T06:18:06.940 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6769000000000002e-05 2024-09-14T06:18:06.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.940 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.940 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.940 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964632+0000", 2024-09-14T06:18:06.940 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096250589999999997 2024-09-14T06:18:06.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.940 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964772+0000", 2024-09-14T06:18:06.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014045299999999999 2024-09-14T06:18:06.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964927+0000", 2024-09-14T06:18:06.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015454299999999999 2024-09-14T06:18:06.941 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.941 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.941 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.943 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32026 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:06.943 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:18:06.943 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 63.042543203999998, 2024-09-14T06:18:06.944 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096901622000000007, 2024-09-14T06:18:06.944 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.944 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.944 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:06.944 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.944 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.944 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.944 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32026 2024-09-14T06:18:06.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.944 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.944 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.944 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:18:06.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:18:06.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868392+0000", 2024-09-14T06:18:06.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0699999999999996e-07 2024-09-14T06:18:06.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868395+0000", 2024-09-14T06:18:06.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9859999999999999e-06 2024-09-14T06:18:06.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868405+0000", 2024-09-14T06:18:06.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9229999999999997e-06 2024-09-14T06:18:06.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868412+0000", 2024-09-14T06:18:06.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9670000000000004e-06 2024-09-14T06:18:06.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965010+0000", 2024-09-14T06:18:06.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096598562999999998 2024-09-14T06:18:06.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965143+0000", 2024-09-14T06:18:06.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013247700000000001 2024-09-14T06:18:06.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965293+0000", 2024-09-14T06:18:06.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000149899 2024-09-14T06:18:06.947 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.947 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.948 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.948 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.948 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32027 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:06.948 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:18:06.948 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 63.042474867000003, 2024-09-14T06:18:06.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097211306999999997, 2024-09-14T06:18:06.948 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.948 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.948 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:06.948 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.948 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.948 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.949 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32027 2024-09-14T06:18:06.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.949 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.949 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.949 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:18:06.949 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.949 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.949 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:18:06.949 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868461+0000", 2024-09-14T06:18:06.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.296e-06 2024-09-14T06:18:06.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868465+0000", 2024-09-14T06:18:06.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9570000000000004e-06 2024-09-14T06:18:06.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868476+0000", 2024-09-14T06:18:06.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1365999999999999e-05 2024-09-14T06:18:06.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868483+0000", 2024-09-14T06:18:06.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2529999999999998e-06 2024-09-14T06:18:06.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965375+0000", 2024-09-14T06:18:06.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096891973000000006 2024-09-14T06:18:06.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.952 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965518+0000", 2024-09-14T06:18:06.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143017 2024-09-14T06:18:06.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.952 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.952 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965671+0000", 2024-09-14T06:18:06.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015244499999999999 2024-09-14T06:18:06.952 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.952 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.952 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.952 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32028 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:06.953 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:18:06.953 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 63.042440038999999, 2024-09-14T06:18:06.953 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097841480999999994, 2024-09-14T06:18:06.953 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.953 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.953 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:06.953 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.953 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.953 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.953 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32028 2024-09-14T06:18:06.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.953 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.953 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.953 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:18:06.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:18:06.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868495+0000", 2024-09-14T06:18:06.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6400000000000001e-07 2024-09-14T06:18:06.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868498+0000", 2024-09-14T06:18:06.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7760000000000002e-06 2024-09-14T06:18:06.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868507+0000", 2024-09-14T06:18:06.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1530000000000003e-06 2024-09-14T06:18:06.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868516+0000", 2024-09-14T06:18:06.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8929999999999994e-06 2024-09-14T06:18:06.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965753+0000", 2024-09-14T06:18:06.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097236970000000006 2024-09-14T06:18:06.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965890+0000", 2024-09-14T06:18:06.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013677599999999999 2024-09-14T06:18:06.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966336+0000", 2024-09-14T06:18:06.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00044614900000000001 2024-09-14T06:18:06.956 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.956 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.956 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.957 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32029 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:06.957 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:18:06.957 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 63.042408964000003, 2024-09-14T06:18:06.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098577779000000004, 2024-09-14T06:18:06.957 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.957 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.957 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:06.957 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.957 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.957 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.957 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32029 2024-09-14T06:18:06.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.957 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:18:06.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.958 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:18:06.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.958 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868526+0000", 2024-09-14T06:18:06.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9300000000000006e-07 2024-09-14T06:18:06.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868529+0000", 2024-09-14T06:18:06.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5969999999999999e-06 2024-09-14T06:18:06.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868538+0000", 2024-09-14T06:18:06.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1700000000000003e-06 2024-09-14T06:18:06.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868560+0000", 2024-09-14T06:18:06.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1596000000000001e-05 2024-09-14T06:18:06.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.960 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.960 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966423+0000", 2024-09-14T06:18:06.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097863396000000005 2024-09-14T06:18:06.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.960 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.960 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966578+0000", 2024-09-14T06:18:06.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000154792 2024-09-14T06:18:06.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.960 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.960 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967103+0000", 2024-09-14T06:18:06.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00052523499999999998 2024-09-14T06:18:06.960 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.961 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.961 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.961 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.961 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.961 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32030 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:06.961 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:18:06.961 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 63.042364046000003, 2024-09-14T06:18:06.961 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099324030999999993, 2024-09-14T06:18:06.961 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.961 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.961 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:06.961 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.961 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.961 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.961 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32030 2024-09-14T06:18:06.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.962 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.962 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.962 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.962 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:18:06.962 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.962 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.962 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.962 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:18:06.962 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.962 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.962 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.962 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868571+0000", 2024-09-14T06:18:06.962 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9699999999999995e-07 2024-09-14T06:18:06.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868574+0000", 2024-09-14T06:18:06.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9529999999999999e-06 2024-09-14T06:18:06.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868584+0000", 2024-09-14T06:18:06.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7860000000000008e-06 2024-09-14T06:18:06.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868591+0000", 2024-09-14T06:18:06.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0500000000000003e-06 2024-09-14T06:18:06.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967191+0000", 2024-09-14T06:18:06.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098600035000000003 2024-09-14T06:18:06.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967333+0000", 2024-09-14T06:18:06.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014190599999999999 2024-09-14T06:18:06.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967894+0000", 2024-09-14T06:18:06.965 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000561504 2024-09-14T06:18:06.965 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.965 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.965 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.965 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.965 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32031 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:06.965 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:18:06.965 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 63.042334484000001, 2024-09-14T06:18:06.965 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.10010279, 2024-09-14T06:18:06.965 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.965 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.965 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:06.966 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.966 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.966 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.966 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32031 2024-09-14T06:18:06.966 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.966 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.966 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.966 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.966 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:18:06.966 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.966 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.966 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.966 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.966 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:18:06.966 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.967 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.967 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.967 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:18:06.967 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:18:06.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.967 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.967 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.967 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868603+0000", 2024-09-14T06:18:06.967 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.599e-06 2024-09-14T06:18:06.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.967 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.967 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.968 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868612+0000", 2024-09-14T06:18:06.968 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.986e-06 2024-09-14T06:18:06.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.968 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.968 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.968 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868619+0000", 2024-09-14T06:18:06.968 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8900000000000001e-06 2024-09-14T06:18:06.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.968 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.968 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.968 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967983+0000", 2024-09-14T06:18:06.968 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099363612000000004 2024-09-14T06:18:06.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.968 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.969 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.969 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968140+0000", 2024-09-14T06:18:06.969 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015788600000000001 2024-09-14T06:18:06.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.969 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.969 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968703+0000", 2024-09-14T06:18:06.969 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00056208300000000001 2024-09-14T06:18:06.969 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.969 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.969 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.969 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32032 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6258688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:06.969 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:18:06.970 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 63.042306009000001, 2024-09-14T06:18:06.970 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100488088, 2024-09-14T06:18:06.970 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.970 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.970 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:06.970 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.970 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.970 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.970 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32032 2024-09-14T06:18:06.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.970 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.970 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.970 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.971 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:18:06.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.971 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.971 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.971 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:18:06.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.971 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.971 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.971 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868629+0000", 2024-09-14T06:18:06.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:18:06.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.972 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.972 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.972 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868631+0000", 2024-09-14T06:18:06.972 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4099999999999998e-06 2024-09-14T06:18:06.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.972 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.972 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.972 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868640+0000", 2024-09-14T06:18:06.972 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0939999999999997e-06 2024-09-14T06:18:06.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.972 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.972 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.972 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868647+0000", 2024-09-14T06:18:06.972 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9460000000000001e-06 2024-09-14T06:18:06.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.973 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968797+0000", 2024-09-14T06:18:06.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100149867 2024-09-14T06:18:06.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.973 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968937+0000", 2024-09-14T06:18:06.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013981399999999999 2024-09-14T06:18:06.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.973 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969116+0000", 2024-09-14T06:18:06.974 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017927399999999999 2024-09-14T06:18:06.974 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.974 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.974 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.974 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.974 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32044 3.1e 3:7e6619d4:::rbd_data.115274ed0aa8.0000000000000068:head [set-alloc-hint object_size 1048576 write_size 1048576,write 835584~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:06.974 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:18:06.974 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 63.034616925999998, 2024-09-14T06:18:06.974 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096039367, 2024-09-14T06:18:06.974 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.974 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.974 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:06.975 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.975 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.975 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.975 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32044 2024-09-14T06:18:06.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.975 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.975 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.975 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.975 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:18:06.975 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.975 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.975 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.975 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:18:06.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.976 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876320+0000", 2024-09-14T06:18:06.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8270000000000002e-06 2024-09-14T06:18:06.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.976 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876397+0000", 2024-09-14T06:18:06.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6985999999999998e-05 2024-09-14T06:18:06.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.976 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.977 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876431+0000", 2024-09-14T06:18:06.977 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3719e-05 2024-09-14T06:18:06.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.977 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.977 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876452+0000", 2024-09-14T06:18:06.977 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0891999999999999e-05 2024-09-14T06:18:06.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.977 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.977 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969186+0000", 2024-09-14T06:18:06.977 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.092734681999999999 2024-09-14T06:18:06.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.978 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.978 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969550+0000", 2024-09-14T06:18:06.978 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036402199999999998 2024-09-14T06:18:06.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.978 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.978 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:18:06.978 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969987+0000", 2024-09-14T06:18:06.978 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000436759 2024-09-14T06:18:06.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.978 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.978 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:06.978 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971569+0000", 2024-09-14T06:18:06.978 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001581821 2024-09-14T06:18:06.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:06.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971947+0000", 2024-09-14T06:18:06.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378446 2024-09-14T06:18:06.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:06.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972146+0000", 2024-09-14T06:18:06.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019819 2024-09-14T06:18:06.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972357+0000", 2024-09-14T06:18:06.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021102299999999999 2024-09-14T06:18:06.979 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.980 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.980 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.980 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.980 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40252 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 827392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:06.980 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:18:06.980 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.19398082, 2024-09-14T06:18:06.980 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095633474999999996, 2024-09-14T06:18:06.980 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.980 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.980 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:06.980 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.980 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.981 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.981 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40252 2024-09-14T06:18:06.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.981 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.981 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.981 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.981 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:18:06.981 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.981 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.981 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.981 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:18:06.981 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.982 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717376+0000", 2024-09-14T06:18:06.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000422216 2024-09-14T06:18:06.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.982 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717386+0000", 2024-09-14T06:18:06.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0396e-05 2024-09-14T06:18:06.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.982 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717395+0000", 2024-09-14T06:18:06.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7819999999999999e-06 2024-09-14T06:18:06.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.983 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717400+0000", 2024-09-14T06:18:06.983 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1599999999999997e-06 2024-09-14T06:18:06.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.983 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.811922+0000", 2024-09-14T06:18:06.983 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.094522018999999999 2024-09-14T06:18:06.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.983 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812007+0000", 2024-09-14T06:18:06.984 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5328999999999996e-05 2024-09-14T06:18:06.984 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.984 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.984 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.984 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812587+0000", 2024-09-14T06:18:06.984 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000579573 2024-09-14T06:18:06.984 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.984 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.984 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.984 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.984 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.984 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40254 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:06.985 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:18:06.985 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.193522647000002, 2024-09-14T06:18:06.985 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095562731999999997, 2024-09-14T06:18:06.985 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.985 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.985 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:06.985 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.985 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.985 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.985 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40254 2024-09-14T06:18:06.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.985 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.985 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.985 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.985 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:18:06.986 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.986 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.986 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.986 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.986 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:18:06.986 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.986 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.986 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.986 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.986 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:18:06.986 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1699999999999999e-07 2024-09-14T06:18:06.986 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.986 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.986 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.986 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717414+0000", 2024-09-14T06:18:06.987 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:18:06.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.987 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.987 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.987 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717419+0000", 2024-09-14T06:18:06.987 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2839999999999997e-06 2024-09-14T06:18:06.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.987 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.987 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.987 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717424+0000", 2024-09-14T06:18:06.987 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1549999999999998e-06 2024-09-14T06:18:06.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.987 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.987 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.987 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812619+0000", 2024-09-14T06:18:06.988 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095194940000000006 2024-09-14T06:18:06.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.988 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.988 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.988 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812717+0000", 2024-09-14T06:18:06.988 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7647999999999997e-05 2024-09-14T06:18:06.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.988 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.988 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.988 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812974+0000", 2024-09-14T06:18:06.988 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025752099999999998 2024-09-14T06:18:06.988 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.989 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.989 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.989 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.989 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.989 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40256 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 843776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:06.989 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:18:06.989 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.193500043, 2024-09-14T06:18:06.989 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095932066999999996, 2024-09-14T06:18:06.989 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.989 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.989 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:06.989 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.989 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.990 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.990 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40256 2024-09-14T06:18:06.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.990 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.990 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.990 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.990 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:18:06.990 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.990 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.990 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.990 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:18:06.990 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.991 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.991 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.991 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717628+0000", 2024-09-14T06:18:06.991 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019339499999999999 2024-09-14T06:18:06.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.991 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.991 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.991 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717632+0000", 2024-09-14T06:18:06.991 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2230000000000001e-06 2024-09-14T06:18:06.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.991 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.991 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.991 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717641+0000", 2024-09-14T06:18:06.991 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6309999999999998e-06 2024-09-14T06:18:06.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.992 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.992 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.992 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717656+0000", 2024-09-14T06:18:06.992 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4724e-05 2024-09-14T06:18:06.992 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.992 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.992 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.992 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813006+0000", 2024-09-14T06:18:06.992 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095349801999999997 2024-09-14T06:18:06.992 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.992 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.992 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.992 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813107+0000", 2024-09-14T06:18:06.993 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000100973 2024-09-14T06:18:06.993 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.993 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.993 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.993 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813366+0000", 2024-09-14T06:18:06.993 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025931899999999999 2024-09-14T06:18:06.993 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.993 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.993 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.993 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.993 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.993 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40258 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 851968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:06.994 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:18:06.994 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.193263942999998, 2024-09-14T06:18:06.994 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096062543, 2024-09-14T06:18:06.994 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.994 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.994 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:06.994 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.994 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.994 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.994 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40258 2024-09-14T06:18:06.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.994 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.994 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:18:06.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:18:06.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:06.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717671+0000", 2024-09-14T06:18:06.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8299999999999997e-07 2024-09-14T06:18:06.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.996 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:06.996 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717674+0000", 2024-09-14T06:18:06.996 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7080000000000002e-06 2024-09-14T06:18:06.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.996 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.996 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:06.996 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717681+0000", 2024-09-14T06:18:06.996 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7889999999999995e-06 2024-09-14T06:18:06.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.996 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.996 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:06.996 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717687+0000", 2024-09-14T06:18:06.996 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7529999999999998e-06 2024-09-14T06:18:06.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.997 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.997 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:06.997 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813408+0000", 2024-09-14T06:18:06.997 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095720451999999998 2024-09-14T06:18:06.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.997 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.997 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:06.997 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813499+0000", 2024-09-14T06:18:06.997 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0933000000000005e-05 2024-09-14T06:18:06.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.997 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.997 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:06.997 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813733+0000", 2024-09-14T06:18:06.997 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00023432499999999999 2024-09-14T06:18:06.998 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.998 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:06.998 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:06.998 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.998 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.998 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40260 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 860160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:06.998 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:18:06.998 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.193238094000002, 2024-09-14T06:18:06.998 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096337381999999999, 2024-09-14T06:18:06.998 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:06.998 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:06.998 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:06.998 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:06.999 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:06.999 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:06.999 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40260 2024-09-14T06:18:06.999 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.999 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:06.999 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.999 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:06.999 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:18:06.999 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:06.999 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:06.999 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:06.999 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:06.999 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:18:06.999 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:07.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.000 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:07.000 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718008+0000", 2024-09-14T06:18:07.000 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00031192699999999998 2024-09-14T06:18:07.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.000 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:07.000 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718013+0000", 2024-09-14T06:18:07.000 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1830000000000002e-06 2024-09-14T06:18:07.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.000 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:07.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718036+0000", 2024-09-14T06:18:07.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2721000000000002e-05 2024-09-14T06:18:07.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:07.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718053+0000", 2024-09-14T06:18:07.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6736e-05 2024-09-14T06:18:07.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:07.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813767+0000", 2024-09-14T06:18:07.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095714658999999994 2024-09-14T06:18:07.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.002 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:07.002 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813816+0000", 2024-09-14T06:18:07.002 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8848999999999999e-05 2024-09-14T06:18:07.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.002 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:07.002 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814034+0000", 2024-09-14T06:18:07.002 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000217307 2024-09-14T06:18:07.002 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:07.002 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:07.002 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:07.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.003 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40262 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 868352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:07.003 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:18:07.003 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.192838410999997, 2024-09-14T06:18:07.003 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096354165000000006, 2024-09-14T06:18:07.003 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:07.003 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:07.003 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:07.003 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:07.003 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:07.003 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:07.003 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40262 2024-09-14T06:18:07.003 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.003 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:07.003 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.004 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:07.004 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:18:07.004 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:07.004 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.004 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.004 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:07.004 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:18:07.004 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:07.004 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.004 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.004 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:07.004 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718281+0000", 2024-09-14T06:18:07.004 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000185069 2024-09-14T06:18:07.004 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.005 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.005 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:07.005 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718286+0000", 2024-09-14T06:18:07.005 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.118e-06 2024-09-14T06:18:07.005 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.005 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.005 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:07.005 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718295+0000", 2024-09-14T06:18:07.005 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0429999999999996e-06 2024-09-14T06:18:07.005 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.005 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.005 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:07.005 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718302+0000", 2024-09-14T06:18:07.005 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2790000000000001e-06 2024-09-14T06:18:07.006 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.006 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.006 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:07.006 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814076+0000", 2024-09-14T06:18:07.006 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095774076999999999 2024-09-14T06:18:07.006 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.006 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.006 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:07.006 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814191+0000", 2024-09-14T06:18:07.006 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00011500700000000001 2024-09-14T06:18:07.006 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.006 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.006 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:07.006 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814450+0000", 2024-09-14T06:18:07.006 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025857199999999999 2024-09-14T06:18:07.007 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:07.007 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:07.007 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:07.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.007 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.007 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40264 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 876544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:07.007 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:18:07.007 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 57.192481342999997, 2024-09-14T06:18:07.007 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096372478999999997, 2024-09-14T06:18:07.007 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:07.007 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:07.007 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:07.007 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:07.007 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:07.008 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:07.008 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40264 2024-09-14T06:18:07.008 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.008 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:07.008 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.008 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:07.008 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:18:07.008 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:07.008 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.008 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.008 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:07.008 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:18:07.008 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:07.008 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.008 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.009 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:07.009 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718793+0000", 2024-09-14T06:18:07.009 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00033991099999999998 2024-09-14T06:18:07.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.009 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.009 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:07.009 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718805+0000", 2024-09-14T06:18:07.009 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2514000000000001e-05 2024-09-14T06:18:07.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.009 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.009 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:07.009 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718815+0000", 2024-09-14T06:18:07.009 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9590000000000007e-06 2024-09-14T06:18:07.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.010 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.010 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:07.010 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718822+0000", 2024-09-14T06:18:07.010 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2030000000000003e-06 2024-09-14T06:18:07.010 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.010 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.010 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:07.010 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814490+0000", 2024-09-14T06:18:07.010 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095667346 2024-09-14T06:18:07.010 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.010 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.010 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:07.010 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814577+0000", 2024-09-14T06:18:07.010 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6920000000000001e-05 2024-09-14T06:18:07.011 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:07.011 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:07.011 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:07.011 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814825+0000", 2024-09-14T06:18:07.011 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000248626 2024-09-14T06:18:07.011 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:07.011 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:07.011 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:07.011 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:07.011 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:07.011 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:07.013 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:18:07.013 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:07.009+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:07.053 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:18:07.114 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:07.111+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:07.139 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:07.139 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:07.139 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:18:07.139 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:07.153 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:18:07.212 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:18:07.213 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:18:07.215 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:07.209+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:07.229 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:18:07.229 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:18:07.245 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:18:07.245 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:18:07.262 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:18:07.262 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:18:07.282 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:18:07.283 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:18:07.302 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:18:07.302 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:18:07.315 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:07.313+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:07.323 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:07.323 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:07.323 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:18:07.323 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:18:07.337 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:18:07.345 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:18:07.345 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:18:07.417 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:07.413+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:07.418 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:18:07.421 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T06:18:07.452 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T06:18:07.470 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:07.471 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:07.486 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:18:07.543 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:07.541+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:07.617 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:07.617 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:07.618 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:07.613+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:07.632 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:18:07.719 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:07.717+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:07.754 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:07.754 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:07.768 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:18:07.821 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:07.817+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:07.891 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:07.891 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:07.905 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:18:07.952 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:07.949+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:08.023 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:08.019+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:08.037 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:08.037 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:08.051 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:18:08.124 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:08.123+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:08.224 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:08.225 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:08.225 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:08.223+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:08.239 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:18:08.326 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:08.321+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:08.361 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:08.361 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:08.375 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:18:08.427 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:08.423+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:08.429 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T06:18:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-09-14T06:18:08.459 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T06:18:08.559 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:18:08.557+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:18:08.567 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:08.567 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:08.584 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:18:08.628 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:08.627+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:08.729 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:08.725+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:08.757 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:08.757 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:08.772 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:18:08.867 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:08.865+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:08.931 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:08.927+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:08.944 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:08.944 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:08.959 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:18:09.048 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:09.045+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:09.131 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:09.131 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:09.133 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:18:09.129+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:18:09.145 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:18:09.201 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T06:18:09.234 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:09.229+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:09.238 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T06:18:09.310 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:09.310 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:09.324 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:18:09.365 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:18:09.372 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:18:09.369+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:18:09.436 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:09.435+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:09.490 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:09.490 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:09.506 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:18:09.533 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:18:09.533 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:18:09.537 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:09.535+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:09.552 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:18:09.552 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:18:09.570 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:18:09.571 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:18:09.587 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:18:09.588 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:18:09.608 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:18:09.608 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:18:09.626 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:09.626 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:18:09.626 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-14T06:18:09.626 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:09.628 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:18:09.629 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:18:09.637 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:09.635+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:09.641 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:18:09.650 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:18:09.650 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:18:09.671 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:18:09.671 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:18:09.738 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:09.735+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:09.768 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:18:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:18:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:18:09.769 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:09.782 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:18:09.839 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:09.833+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:09.912 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:18:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:18:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:18:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:18:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 93.917013462, 2024-09-14T06:18:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:18:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:18:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:18:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:18:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:18:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:18:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:18:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:18:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:18:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:18:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:18:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:18:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:18:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:18:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:18:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:18:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:18:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:18:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:18:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 93.917000389999998, 2024-09-14T06:18:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:18:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:18:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:18:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:18:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:18:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:18:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:18:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:18:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:18:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:18:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:18:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:18:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:18:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:18:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:18:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:18:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:18:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:18:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:18:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 93.916982958000006, 2024-09-14T06:18:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:18:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:18:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:18:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:18:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:18:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:18:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.924 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:09.924 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:18:09.924 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:18:09.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.924 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.924 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:09.924 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:18:09.924 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:18:09.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.924 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.924 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:09.924 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:18:09.924 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:18:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:18:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:18:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:18:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:18:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:18:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:18:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:18:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 93.916961853000004, 2024-09-14T06:18:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:18:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:18:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:18:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:18:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:18:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:18:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:18:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:18:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:18:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:18:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:18:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:18:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:18:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:18:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:18:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:18:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:18:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:18:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:18:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 93.916940745000005, 2024-09-14T06:18:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:18:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:18:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:18:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:18:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:18:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:18:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:18:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:18:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:18:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:18:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:18:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:18:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:18:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:18:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:18:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:18:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:18:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:18:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48234 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:18:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 59.875669453, 2024-09-14T06:18:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051730353999999999, 2024-09-14T06:18:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48234 2024-09-14T06:18:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:18:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:18:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032937+0000", 2024-09-14T06:18:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:18:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032940+0000", 2024-09-14T06:18:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0580000000000002e-06 2024-09-14T06:18:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032946+0000", 2024-09-14T06:18:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2960000000000004e-06 2024-09-14T06:18:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032951+0000", 2024-09-14T06:18:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.938e-06 2024-09-14T06:18:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074419+0000", 2024-09-14T06:18:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041467979000000002 2024-09-14T06:18:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074490+0000", 2024-09-14T06:18:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1324999999999999e-05 2024-09-14T06:18:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074859+0000", 2024-09-14T06:18:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036914899999999999 2024-09-14T06:18:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078464+0000", 2024-09-14T06:18:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036043170000000001 2024-09-14T06:18:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084547+0000", 2024-09-14T06:18:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006083097 2024-09-14T06:18:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084614+0000", 2024-09-14T06:18:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7509999999999996e-05 2024-09-14T06:18:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084667+0000", 2024-09-14T06:18:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:18:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48237 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:18:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 59.875589709000003, 2024-09-14T06:18:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051772830999999998, 2024-09-14T06:18:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48237 2024-09-14T06:18:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:18:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:18:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:18:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0499999999999999e-07 2024-09-14T06:18:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033018+0000", 2024-09-14T06:18:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0949999999999998e-06 2024-09-14T06:18:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033023+0000", 2024-09-14T06:18:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1620000000000001e-06 2024-09-14T06:18:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033026+0000", 2024-09-14T06:18:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0120000000000001e-06 2024-09-14T06:18:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075349+0000", 2024-09-14T06:18:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042323373999999997 2024-09-14T06:18:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075394+0000", 2024-09-14T06:18:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4861000000000003e-05 2024-09-14T06:18:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075578+0000", 2024-09-14T06:18:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00018414899999999999 2024-09-14T06:18:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078894+0000", 2024-09-14T06:18:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033158480000000001 2024-09-14T06:18:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084677+0000", 2024-09-14T06:18:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0057832120000000002 2024-09-14T06:18:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084752+0000", 2024-09-14T06:18:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5123000000000006e-05 2024-09-14T06:18:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084789+0000", 2024-09-14T06:18:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6690000000000003e-05 2024-09-14T06:18:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48243 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:18:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 59.875508584000002, 2024-09-14T06:18:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051780221000000001, 2024-09-14T06:18:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48243 2024-09-14T06:18:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:18:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:18:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:18:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:18:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033100+0000", 2024-09-14T06:18:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:18:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033108+0000", 2024-09-14T06:18:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1760000000000004e-06 2024-09-14T06:18:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033111+0000", 2024-09-14T06:18:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0790000000000001e-06 2024-09-14T06:18:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075903+0000", 2024-09-14T06:18:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042791942999999999 2024-09-14T06:18:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075987+0000", 2024-09-14T06:18:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4467000000000003e-05 2024-09-14T06:18:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.076259+0000", 2024-09-14T06:18:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027201900000000003 2024-09-14T06:18:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078947+0000", 2024-09-14T06:18:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026875879999999999 2024-09-14T06:18:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084797+0000", 2024-09-14T06:18:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0058500380000000001 2024-09-14T06:18:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084843+0000", 2024-09-14T06:18:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6298999999999998e-05 2024-09-14T06:18:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084877+0000", 2024-09-14T06:18:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4378999999999997e-05 2024-09-14T06:18:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48287 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:18:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 59.875468783999999, 2024-09-14T06:18:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051924706000000001, 2024-09-14T06:18:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48287 2024-09-14T06:18:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:18:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:18:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033138+0000", 2024-09-14T06:18:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6599999999999996e-07 2024-09-14T06:18:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033141+0000", 2024-09-14T06:18:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0879999999999999e-06 2024-09-14T06:18:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033147+0000", 2024-09-14T06:18:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2249999999999997e-06 2024-09-14T06:18:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033153+0000", 2024-09-14T06:18:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.066e-06 2024-09-14T06:18:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077790+0000", 2024-09-14T06:18:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.044637255000000001 2024-09-14T06:18:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077845+0000", 2024-09-14T06:18:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4410999999999998e-05 2024-09-14T06:18:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078062+0000", 2024-09-14T06:18:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021755599999999999 2024-09-14T06:18:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.080991+0000", 2024-09-14T06:18:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029282969999999998 2024-09-14T06:18:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084885+0000", 2024-09-14T06:18:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003894495 2024-09-14T06:18:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085020+0000", 2024-09-14T06:18:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013500799999999999 2024-09-14T06:18:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085062+0000", 2024-09-14T06:18:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1638999999999999e-05 2024-09-14T06:18:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48288 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:18:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 59.875445085000003, 2024-09-14T06:18:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051994018000000003, 2024-09-14T06:18:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48288 2024-09-14T06:18:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:18:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:18:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:18:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.89e-07 2024-09-14T06:18:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033164+0000", 2024-09-14T06:18:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.779e-06 2024-09-14T06:18:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033171+0000", 2024-09-14T06:18:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6279999999999999e-06 2024-09-14T06:18:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033176+0000", 2024-09-14T06:18:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1610000000000002e-06 2024-09-14T06:18:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078991+0000", 2024-09-14T06:18:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045815348999999998 2024-09-14T06:18:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079086+0000", 2024-09-14T06:18:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5282e-05 2024-09-14T06:18:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079397+0000", 2024-09-14T06:18:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031055999999999998 2024-09-14T06:18:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084432+0000", 2024-09-14T06:18:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050355599999999997 2024-09-14T06:18:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085073+0000", 2024-09-14T06:18:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064015199999999995 2024-09-14T06:18:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085120+0000", 2024-09-14T06:18:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7107000000000001e-05 2024-09-14T06:18:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085155+0000", 2024-09-14T06:18:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4950999999999998e-05 2024-09-14T06:18:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48416 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:18:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 59.873204673000004, 2024-09-14T06:18:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053690144000000002, 2024-09-14T06:18:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48416 2024-09-14T06:18:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:18:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:18:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035402+0000", 2024-09-14T06:18:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7999999999999995e-07 2024-09-14T06:18:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035405+0000", 2024-09-14T06:18:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3960000000000002e-06 2024-09-14T06:18:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035413+0000", 2024-09-14T06:18:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2209999999999997e-06 2024-09-14T06:18:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035420+0000", 2024-09-14T06:18:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1539999999999996e-06 2024-09-14T06:18:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081021+0000", 2024-09-14T06:18:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045600389999999998 2024-09-14T06:18:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081091+0000", 2024-09-14T06:18:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0325000000000002e-05 2024-09-14T06:18:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081434+0000", 2024-09-14T06:18:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000342519 2024-09-14T06:18:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085254+0000", 2024-09-14T06:18:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038201609999999999 2024-09-14T06:18:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088977+0000", 2024-09-14T06:18:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003722805 2024-09-14T06:18:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089048+0000", 2024-09-14T06:18:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1266999999999994e-05 2024-09-14T06:18:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089091+0000", 2024-09-14T06:18:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3325999999999998e-05 2024-09-14T06:18:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48417 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:18:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 59.873176004000001, 2024-09-14T06:18:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053805032000000003, 2024-09-14T06:18:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48417 2024-09-14T06:18:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:18:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:18:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:18:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9500000000000003e-07 2024-09-14T06:18:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035433+0000", 2024-09-14T06:18:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.796e-06 2024-09-14T06:18:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035442+0000", 2024-09-14T06:18:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7120000000000006e-06 2024-09-14T06:18:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035448+0000", 2024-09-14T06:18:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.015e-06 2024-09-14T06:18:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081855+0000", 2024-09-14T06:18:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.046407634000000003 2024-09-14T06:18:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081938+0000", 2024-09-14T06:18:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2687000000000003e-05 2024-09-14T06:18:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082313+0000", 2024-09-14T06:18:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000374872 2024-09-14T06:18:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087616+0000", 2024-09-14T06:18:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053032670000000004 2024-09-14T06:18:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089140+0000", 2024-09-14T06:18:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0015234350000000001 2024-09-14T06:18:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089184+0000", 2024-09-14T06:18:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4804e-05 2024-09-14T06:18:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089235+0000", 2024-09-14T06:18:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0315000000000002e-05 2024-09-14T06:18:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48418 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:18:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 59.873131864999998, 2024-09-14T06:18:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053936425000000003, 2024-09-14T06:18:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48418 2024-09-14T06:18:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:18:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:18:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035475+0000", 2024-09-14T06:18:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7000000000000003e-07 2024-09-14T06:18:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035479+0000", 2024-09-14T06:18:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2350000000000001e-06 2024-09-14T06:18:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035488+0000", 2024-09-14T06:18:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8820000000000007e-06 2024-09-14T06:18:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035494+0000", 2024-09-14T06:18:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3060000000000003e-06 2024-09-14T06:18:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082826+0000", 2024-09-14T06:18:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.047332008000000002 2024-09-14T06:18:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082883+0000", 2024-09-14T06:18:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7124999999999999e-05 2024-09-14T06:18:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.083385+0000", 2024-09-14T06:18:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050134099999999998 2024-09-14T06:18:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088888+0000", 2024-09-14T06:18:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005503127 2024-09-14T06:18:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089294+0000", 2024-09-14T06:18:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040669700000000003 2024-09-14T06:18:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089348+0000", 2024-09-14T06:18:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3729e-05 2024-09-14T06:18:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089410+0000", 2024-09-14T06:18:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2105000000000003e-05 2024-09-14T06:18:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48419 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:18:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 59.873054234999998, 2024-09-14T06:18:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054007545999999997, 2024-09-14T06:18:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48419 2024-09-14T06:18:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:18:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:18:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035552+0000", 2024-09-14T06:18:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4099999999999998e-07 2024-09-14T06:18:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035557+0000", 2024-09-14T06:18:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1819999999999997e-06 2024-09-14T06:18:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035566+0000", 2024-09-14T06:18:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1120000000000002e-06 2024-09-14T06:18:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035572+0000", 2024-09-14T06:18:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0430000000000004e-06 2024-09-14T06:18:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085304+0000", 2024-09-14T06:18:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049731707 2024-09-14T06:18:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085371+0000", 2024-09-14T06:18:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6771000000000002e-05 2024-09-14T06:18:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085601+0000", 2024-09-14T06:18:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022968000000000001 2024-09-14T06:18:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088893+0000", 2024-09-14T06:18:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032927780000000001 2024-09-14T06:18:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089467+0000", 2024-09-14T06:18:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057330399999999996 2024-09-14T06:18:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089520+0000", 2024-09-14T06:18:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3133000000000001e-05 2024-09-14T06:18:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089559+0000", 2024-09-14T06:18:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9195e-05 2024-09-14T06:18:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48420 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:18:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 59.872965004999998, 2024-09-14T06:18:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056401932000000002, 2024-09-14T06:18:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48420 2024-09-14T06:18:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:18:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:18:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035642+0000", 2024-09-14T06:18:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1519999999999999e-06 2024-09-14T06:18:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035647+0000", 2024-09-14T06:18:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2299999999999999e-06 2024-09-14T06:18:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035660+0000", 2024-09-14T06:18:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3166999999999999e-05 2024-09-14T06:18:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035666+0000", 2024-09-14T06:18:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1720000000000004e-06 2024-09-14T06:18:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086104+0000", 2024-09-14T06:18:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050437956999999999 2024-09-14T06:18:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086164+0000", 2024-09-14T06:18:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9592999999999997e-05 2024-09-14T06:18:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086402+0000", 2024-09-14T06:18:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000238314 2024-09-14T06:18:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089103+0000", 2024-09-14T06:18:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027011600000000002 2024-09-14T06:18:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091962+0000", 2024-09-14T06:18:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028581259999999999 2024-09-14T06:18:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092005+0000", 2024-09-14T06:18:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3111000000000001e-05 2024-09-14T06:18:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092043+0000", 2024-09-14T06:18:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7950000000000001e-05 2024-09-14T06:18:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48424 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:18:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 59.872701059999997, 2024-09-14T06:18:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056273173000000003, 2024-09-14T06:18:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48424 2024-09-14T06:18:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:18:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:18:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:18:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2900000000000003e-07 2024-09-14T06:18:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035909+0000", 2024-09-14T06:18:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6490000000000001e-06 2024-09-14T06:18:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035918+0000", 2024-09-14T06:18:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5089999999999999e-06 2024-09-14T06:18:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035924+0000", 2024-09-14T06:18:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7089999999999999e-06 2024-09-14T06:18:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087662+0000", 2024-09-14T06:18:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737621999999997 2024-09-14T06:18:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087744+0000", 2024-09-14T06:18:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2174000000000003e-05 2024-09-14T06:18:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088106+0000", 2024-09-14T06:18:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000361947 2024-09-14T06:18:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092053+0000", 2024-09-14T06:18:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0039473080000000001 2024-09-14T06:18:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092087+0000", 2024-09-14T06:18:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4153000000000003e-05 2024-09-14T06:18:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092138+0000", 2024-09-14T06:18:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0804999999999999e-05 2024-09-14T06:18:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092178+0000", 2024-09-14T06:18:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9668e-05 2024-09-14T06:18:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48508 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:18:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 59.867625979000003, 2024-09-14T06:18:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054906017000000001, 2024-09-14T06:18:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48508 2024-09-14T06:18:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:18:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:18:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040981+0000", 2024-09-14T06:18:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7700000000000004e-07 2024-09-14T06:18:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040984+0000", 2024-09-14T06:18:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1480000000000002e-06 2024-09-14T06:18:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040991+0000", 2024-09-14T06:18:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4580000000000004e-06 2024-09-14T06:18:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040997+0000", 2024-09-14T06:18:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7799999999999997e-06 2024-09-14T06:18:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089601+0000", 2024-09-14T06:18:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048604264000000001 2024-09-14T06:18:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089708+0000", 2024-09-14T06:18:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010681200000000001 2024-09-14T06:18:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090384+0000", 2024-09-14T06:18:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00067557499999999998 2024-09-14T06:18:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093879+0000", 2024-09-14T06:18:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034954679999999998 2024-09-14T06:18:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095764+0000", 2024-09-14T06:18:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018848070000000001 2024-09-14T06:18:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095837+0000", 2024-09-14T06:18:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3433e-05 2024-09-14T06:18:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095886+0000", 2024-09-14T06:18:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8495000000000002e-05 2024-09-14T06:18:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48537 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:18:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 59.866262208999999, 2024-09-14T06:18:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053681833999999998, 2024-09-14T06:18:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48537 2024-09-14T06:18:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:18:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:18:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:18:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2299999999999999e-07 2024-09-14T06:18:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042350+0000", 2024-09-14T06:18:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5690000000000004e-06 2024-09-14T06:18:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042361+0000", 2024-09-14T06:18:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1337e-05 2024-09-14T06:18:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042369+0000", 2024-09-14T06:18:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8359999999999992e-06 2024-09-14T06:18:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090914+0000", 2024-09-14T06:18:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048544945999999999 2024-09-14T06:18:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090995+0000", 2024-09-14T06:18:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0548000000000002e-05 2024-09-14T06:18:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091508+0000", 2024-09-14T06:18:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000513723 2024-09-14T06:18:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093900+0000", 2024-09-14T06:18:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023920429999999999 2024-09-14T06:18:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095944+0000", 2024-09-14T06:18:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020437379999999998 2024-09-14T06:18:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095987+0000", 2024-09-14T06:18:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2521999999999997e-05 2024-09-14T06:18:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.096025+0000", 2024-09-14T06:18:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8649000000000002e-05 2024-09-14T06:18:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:63810 3.1d 3.c887d99d (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-14T06:18:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:18:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.328504322999997, 2024-09-14T06:18:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.61346978900000004, 2024-09-14T06:18:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:18:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 63810 2024-09-14T06:18:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:18:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:18:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580104+0000", 2024-09-14T06:18:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4650000000000001e-06 2024-09-14T06:18:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580112+0000", 2024-09-14T06:18:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9219999999999997e-06 2024-09-14T06:18:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580130+0000", 2024-09-14T06:18:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8128999999999999e-05 2024-09-14T06:18:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580140+0000", 2024-09-14T06:18:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0285000000000001e-05 2024-09-14T06:18:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580201+0000", 2024-09-14T06:18:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0517999999999999e-05 2024-09-14T06:18:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:18:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580209+0000", 2024-09-14T06:18:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6430000000000001e-06 2024-09-14T06:18:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:34.193554+0000", 2024-09-14T06:18:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.61334482899999998 2024-09-14T06:18:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:34.193571+0000", 2024-09-14T06:18:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6997999999999999e-05 2024-09-14T06:18:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:63832 3.1d 3:b99be113:::rbd_object_map.11525ae33f05:head [read 0~52] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T06:18:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:18:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 32.670507936999996, 2024-09-14T06:18:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.99066972200000003, 2024-09-14T06:18:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 63832 2024-09-14T06:18:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:18:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:18:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238100+0000", 2024-09-14T06:18:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0849999999999999e-06 2024-09-14T06:18:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238108+0000", 2024-09-14T06:18:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.089e-06 2024-09-14T06:18:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238123+0000", 2024-09-14T06:18:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5418e-05 2024-09-14T06:18:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238136+0000", 2024-09-14T06:18:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2204e-05 2024-09-14T06:18:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238913+0000", 2024-09-14T06:18:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00077720399999999998 2024-09-14T06:18:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:18:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238934+0000", 2024-09-14T06:18:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1311999999999999e-05 2024-09-14T06:18:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228134+0000", 2024-09-14T06:18:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.98919991200000001 2024-09-14T06:18:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228261+0000", 2024-09-14T06:18:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000126742 2024-09-14T06:18:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228768+0000", 2024-09-14T06:18:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050675600000000003 2024-09-14T06:18:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:10.026 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:10.026 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:10.026 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:10.027 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:18:10.028 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:10.025+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:10.129 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:10.125+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:10.159 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:10.159 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:10.159 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:18:10.159 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:10.173 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:18:10.211 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T06:18:10.229 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:10.227+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:10.238 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T06:18:10.302 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:10.302 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:10.302 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:18:10.302 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:18:10.302 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:10.320 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:18:10.329 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:10.325+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:10.430 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:10.427+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:10.475 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:18:10.492 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:10.492 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:18:10.493 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:18:10.493 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:18:10.494 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.494 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13229 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.494 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:18:10.494 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.390230400999997, 2024-09-14T06:18:10.494 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081798868999999996, 2024-09-14T06:18:10.494 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.494 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.494 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.494 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.494 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.494 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.494 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13229 2024-09-14T06:18:10.494 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.495 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:18:10.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:18:10.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098710+0000", 2024-09-14T06:18:10.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8199999999999999e-07 2024-09-14T06:18:10.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.496 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.496 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098714+0000", 2024-09-14T06:18:10.496 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:18:10.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.496 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.496 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098727+0000", 2024-09-14T06:18:10.496 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2520999999999999e-05 2024-09-14T06:18:10.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.496 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.496 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098749+0000", 2024-09-14T06:18:10.497 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2773999999999999e-05 2024-09-14T06:18:10.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.497 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.497 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160234+0000", 2024-09-14T06:18:10.497 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.061484430999999999 2024-09-14T06:18:10.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.497 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160281+0000", 2024-09-14T06:18:10.498 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7437e-05 2024-09-14T06:18:10.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.498 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160569+0000", 2024-09-14T06:18:10.498 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028788700000000002 2024-09-14T06:18:10.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.498 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163121+0000", 2024-09-14T06:18:10.498 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025514090000000001 2024-09-14T06:18:10.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.498 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.499 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180410+0000", 2024-09-14T06:18:10.499 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.017289098999999999 2024-09-14T06:18:10.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.499 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.499 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180472+0000", 2024-09-14T06:18:10.499 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2547999999999998e-05 2024-09-14T06:18:10.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.499 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.499 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180508+0000", 2024-09-14T06:18:10.499 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5534000000000002e-05 2024-09-14T06:18:10.499 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.499 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.500 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.500 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13230 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.500 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:18:10.500 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.390149199000007, 2024-09-14T06:18:10.500 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081851558000000005, 2024-09-14T06:18:10.500 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.500 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.500 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.500 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.500 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.500 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.500 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13230 2024-09-14T06:18:10.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.501 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.501 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.501 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:18:10.501 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.501 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.501 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:18:10.501 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098791+0000", 2024-09-14T06:18:10.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8800000000000001e-07 2024-09-14T06:18:10.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098795+0000", 2024-09-14T06:18:10.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.391e-06 2024-09-14T06:18:10.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098804+0000", 2024-09-14T06:18:10.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3440000000000001e-06 2024-09-14T06:18:10.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.503 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.503 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098810+0000", 2024-09-14T06:18:10.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2339999999999999e-06 2024-09-14T06:18:10.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.503 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.503 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160930+0000", 2024-09-14T06:18:10.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062120018999999999 2024-09-14T06:18:10.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.503 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.503 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161017+0000", 2024-09-14T06:18:10.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6868999999999998e-05 2024-09-14T06:18:10.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.504 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.504 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.504 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161300+0000", 2024-09-14T06:18:10.504 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028349499999999998 2024-09-14T06:18:10.504 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.504 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.504 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.504 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164612+0000", 2024-09-14T06:18:10.504 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033119680000000002 2024-09-14T06:18:10.504 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.504 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.504 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.504 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180559+0000", 2024-09-14T06:18:10.504 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015946612999999998 2024-09-14T06:18:10.505 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.505 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.505 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.505 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180605+0000", 2024-09-14T06:18:10.505 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5890000000000003e-05 2024-09-14T06:18:10.505 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.505 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.505 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.505 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180642+0000", 2024-09-14T06:18:10.505 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:18:10.505 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.505 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.505 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.505 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.506 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.506 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13231 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.506 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:18:10.506 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.389886785000002, 2024-09-14T06:18:10.506 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081750685000000003, 2024-09-14T06:18:10.506 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.506 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.506 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.506 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.506 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.506 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.506 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13231 2024-09-14T06:18:10.506 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.506 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.507 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.507 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.507 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:18:10.507 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.507 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.507 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.507 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:18:10.507 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.507 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.507 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.507 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099053+0000", 2024-09-14T06:18:10.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:18:10.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099059+0000", 2024-09-14T06:18:10.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1789999999999999e-06 2024-09-14T06:18:10.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099072+0000", 2024-09-14T06:18:10.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3179999999999999e-05 2024-09-14T06:18:10.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.509 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099080+0000", 2024-09-14T06:18:10.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9030000000000004e-06 2024-09-14T06:18:10.509 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.509 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.509 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161681+0000", 2024-09-14T06:18:10.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062601641999999999 2024-09-14T06:18:10.509 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.509 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.509 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161766+0000", 2024-09-14T06:18:10.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4909999999999998e-05 2024-09-14T06:18:10.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.510 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162109+0000", 2024-09-14T06:18:10.510 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00034264899999999999 2024-09-14T06:18:10.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.510 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164615+0000", 2024-09-14T06:18:10.510 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025061979999999998 2024-09-14T06:18:10.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.510 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.510 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180718+0000", 2024-09-14T06:18:10.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016103240000000001 2024-09-14T06:18:10.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.511 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.511 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180768+0000", 2024-09-14T06:18:10.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9781999999999999e-05 2024-09-14T06:18:10.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.511 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.511 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180803+0000", 2024-09-14T06:18:10.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5021999999999998e-05 2024-09-14T06:18:10.511 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.511 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.511 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.512 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13232 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.512 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:18:10.512 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.389845011000006, 2024-09-14T06:18:10.512 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081861981, 2024-09-14T06:18:10.512 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.512 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.512 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.512 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.512 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.512 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.513 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13232 2024-09-14T06:18:10.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.513 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.513 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.513 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:18:10.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.513 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.513 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:18:10.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.514 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.514 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099095+0000", 2024-09-14T06:18:10.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:18:10.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.514 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.514 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099098+0000", 2024-09-14T06:18:10.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5420000000000001e-06 2024-09-14T06:18:10.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.514 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.514 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099106+0000", 2024-09-14T06:18:10.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.824e-06 2024-09-14T06:18:10.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099112+0000", 2024-09-14T06:18:10.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9789999999999998e-06 2024-09-14T06:18:10.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162576+0000", 2024-09-14T06:18:10.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.063463723999999999 2024-09-14T06:18:10.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162627+0000", 2024-09-14T06:18:10.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1530000000000003e-05 2024-09-14T06:18:10.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.516 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.516 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162854+0000", 2024-09-14T06:18:10.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000226159 2024-09-14T06:18:10.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.516 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.516 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166090+0000", 2024-09-14T06:18:10.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032362319999999999 2024-09-14T06:18:10.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.516 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.516 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180877+0000", 2024-09-14T06:18:10.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014786726 2024-09-14T06:18:10.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.517 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.517 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.517 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180920+0000", 2024-09-14T06:18:10.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3427000000000003e-05 2024-09-14T06:18:10.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.517 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.517 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.517 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180956+0000", 2024-09-14T06:18:10.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6195999999999997e-05 2024-09-14T06:18:10.517 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.517 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.517 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.518 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13233 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.518 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:18:10.518 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.389819747999994, 2024-09-14T06:18:10.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081924860000000002, 2024-09-14T06:18:10.518 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.518 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.518 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.518 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.518 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.518 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.518 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13233 2024-09-14T06:18:10.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.519 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:18:10.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:18:10.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.520 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.520 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099120+0000", 2024-09-14T06:18:10.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:18:10.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.520 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.520 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.520 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099122+0000", 2024-09-14T06:18:10.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:18:10.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.520 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.520 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.520 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099126+0000", 2024-09-14T06:18:10.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2540000000000003e-06 2024-09-14T06:18:10.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099130+0000", 2024-09-14T06:18:10.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4060000000000001e-06 2024-09-14T06:18:10.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163169+0000", 2024-09-14T06:18:10.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064039415000000002 2024-09-14T06:18:10.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163260+0000", 2024-09-14T06:18:10.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1038000000000005e-05 2024-09-14T06:18:10.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163640+0000", 2024-09-14T06:18:10.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037954699999999999 2024-09-14T06:18:10.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166093+0000", 2024-09-14T06:18:10.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024533229999999999 2024-09-14T06:18:10.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180993+0000", 2024-09-14T06:18:10.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01489997 2024-09-14T06:18:10.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.523 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.523 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181020+0000", 2024-09-14T06:18:10.523 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6806999999999999e-05 2024-09-14T06:18:10.523 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.523 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.523 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181044+0000", 2024-09-14T06:18:10.523 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4405000000000001e-05 2024-09-14T06:18:10.523 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.523 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.523 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.523 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.523 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13234 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.524 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:18:10.524 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.389777542999994, 2024-09-14T06:18:10.524 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082015597999999995, 2024-09-14T06:18:10.524 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.524 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.524 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.524 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.524 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.524 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.524 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13234 2024-09-14T06:18:10.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.524 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.524 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:18:10.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:18:10.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:18:10.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0999999999999998e-07 2024-09-14T06:18:10.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099166+0000", 2024-09-14T06:18:10.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:18:10.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099171+0000", 2024-09-14T06:18:10.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6949999999999997e-06 2024-09-14T06:18:10.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099175+0000", 2024-09-14T06:18:10.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9029999999999997e-06 2024-09-14T06:18:10.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.527 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164063+0000", 2024-09-14T06:18:10.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064887608999999999 2024-09-14T06:18:10.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.527 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164111+0000", 2024-09-14T06:18:10.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7648999999999997e-05 2024-09-14T06:18:10.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.527 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164351+0000", 2024-09-14T06:18:10.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024072899999999999 2024-09-14T06:18:10.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167561+0000", 2024-09-14T06:18:10.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032094179999999999 2024-09-14T06:18:10.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181093+0000", 2024-09-14T06:18:10.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013532245 2024-09-14T06:18:10.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181139+0000", 2024-09-14T06:18:10.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6483999999999997e-05 2024-09-14T06:18:10.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.529 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.529 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181177+0000", 2024-09-14T06:18:10.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7861000000000002e-05 2024-09-14T06:18:10.529 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.529 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.529 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.529 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13235 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.529 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:18:10.529 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.389670269000007, 2024-09-14T06:18:10.530 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081988088000000001, 2024-09-14T06:18:10.530 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.530 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.530 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.530 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.530 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.530 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.530 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13235 2024-09-14T06:18:10.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.530 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.530 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.530 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.530 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:18:10.530 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:18:10.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099270+0000", 2024-09-14T06:18:10.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7400000000000003e-07 2024-09-14T06:18:10.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099276+0000", 2024-09-14T06:18:10.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:18:10.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099284+0000", 2024-09-14T06:18:10.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7979999999999995e-06 2024-09-14T06:18:10.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099289+0000", 2024-09-14T06:18:10.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:18:10.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.533 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164664+0000", 2024-09-14T06:18:10.533 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065374901999999999 2024-09-14T06:18:10.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.533 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.533 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.533 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164755+0000", 2024-09-14T06:18:10.533 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0790000000000003e-05 2024-09-14T06:18:10.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.533 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.533 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.533 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165143+0000", 2024-09-14T06:18:10.533 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000387904 2024-09-14T06:18:10.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.533 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.534 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.534 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167564+0000", 2024-09-14T06:18:10.534 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024208989999999998 2024-09-14T06:18:10.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.534 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.534 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181212+0000", 2024-09-14T06:18:10.534 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013648664 2024-09-14T06:18:10.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.534 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.534 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181237+0000", 2024-09-14T06:18:10.534 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5177999999999998e-05 2024-09-14T06:18:10.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.535 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181257+0000", 2024-09-14T06:18:10.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9660000000000002e-05 2024-09-14T06:18:10.535 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.535 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.535 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.535 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13236 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.535 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:18:10.535 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.389563449999997, 2024-09-14T06:18:10.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082010939000000005, 2024-09-14T06:18:10.536 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.536 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.536 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.536 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.536 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.536 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.536 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13236 2024-09-14T06:18:10.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.536 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:18:10.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:18:10.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:18:10.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.61e-07 2024-09-14T06:18:10.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099382+0000", 2024-09-14T06:18:10.538 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3589999999999999e-06 2024-09-14T06:18:10.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.538 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.538 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099388+0000", 2024-09-14T06:18:10.538 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9619999999999998e-06 2024-09-14T06:18:10.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.538 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.538 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099393+0000", 2024-09-14T06:18:10.538 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3410000000000002e-06 2024-09-14T06:18:10.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.539 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.539 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165553+0000", 2024-09-14T06:18:10.539 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066159604999999996 2024-09-14T06:18:10.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.539 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.539 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.539 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165603+0000", 2024-09-14T06:18:10.539 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0427999999999999e-05 2024-09-14T06:18:10.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.539 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.539 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.539 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165827+0000", 2024-09-14T06:18:10.539 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022358299999999999 2024-09-14T06:18:10.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177005+0000", 2024-09-14T06:18:10.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011178044 2024-09-14T06:18:10.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181302+0000", 2024-09-14T06:18:10.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042970669999999999 2024-09-14T06:18:10.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.540 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181349+0000", 2024-09-14T06:18:10.540 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7287999999999999e-05 2024-09-14T06:18:10.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.541 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.541 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.541 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181387+0000", 2024-09-14T06:18:10.541 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7601000000000001e-05 2024-09-14T06:18:10.541 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.541 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.541 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.541 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.541 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13237 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.541 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:18:10.541 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.389405303000004, 2024-09-14T06:18:10.541 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081927299999999995, 2024-09-14T06:18:10.541 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.542 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.542 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.542 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.542 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.542 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.542 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13237 2024-09-14T06:18:10.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.542 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.542 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.542 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:18:10.542 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.542 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:18:10.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099535+0000", 2024-09-14T06:18:10.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:18:10.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099538+0000", 2024-09-14T06:18:10.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:18:10.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.544 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099545+0000", 2024-09-14T06:18:10.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:18:10.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.544 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.544 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099551+0000", 2024-09-14T06:18:10.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1770000000000003e-06 2024-09-14T06:18:10.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.544 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.544 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166131+0000", 2024-09-14T06:18:10.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066580499000000001 2024-09-14T06:18:10.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.545 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.545 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166221+0000", 2024-09-14T06:18:10.545 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0000000000000006e-05 2024-09-14T06:18:10.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.545 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.545 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166622+0000", 2024-09-14T06:18:10.545 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040029799999999999 2024-09-14T06:18:10.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.545 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.545 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177008+0000", 2024-09-14T06:18:10.545 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010386772000000001 2024-09-14T06:18:10.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.546 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.546 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181420+0000", 2024-09-14T06:18:10.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044119190000000003 2024-09-14T06:18:10.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.546 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.546 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181443+0000", 2024-09-14T06:18:10.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2603999999999999e-05 2024-09-14T06:18:10.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.546 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.546 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181461+0000", 2024-09-14T06:18:10.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8405000000000001e-05 2024-09-14T06:18:10.547 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.547 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.547 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.547 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.547 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.547 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13238 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.547 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:18:10.547 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.389368994999998, 2024-09-14T06:18:10.547 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082014927000000001, 2024-09-14T06:18:10.547 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.547 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.547 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.547 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.547 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.548 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.548 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13238 2024-09-14T06:18:10.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.548 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.548 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.548 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.548 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:18:10.548 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.548 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.548 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.548 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:18:10.548 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099571+0000", 2024-09-14T06:18:10.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9699999999999996e-07 2024-09-14T06:18:10.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099577+0000", 2024-09-14T06:18:10.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2890000000000003e-06 2024-09-14T06:18:10.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099587+0000", 2024-09-14T06:18:10.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0002e-05 2024-09-14T06:18:10.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099593+0000", 2024-09-14T06:18:10.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4539999999999998e-06 2024-09-14T06:18:10.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167024+0000", 2024-09-14T06:18:10.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067431480000000002 2024-09-14T06:18:10.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167074+0000", 2024-09-14T06:18:10.551 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9996e-05 2024-09-14T06:18:10.551 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.551 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.551 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.551 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167288+0000", 2024-09-14T06:18:10.551 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021410900000000001 2024-09-14T06:18:10.551 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.551 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.551 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.551 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177997+0000", 2024-09-14T06:18:10.551 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010708756 2024-09-14T06:18:10.551 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.551 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.551 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.552 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181508+0000", 2024-09-14T06:18:10.552 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003511363 2024-09-14T06:18:10.552 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.552 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.552 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.552 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181552+0000", 2024-09-14T06:18:10.552 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3609999999999998e-05 2024-09-14T06:18:10.552 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.552 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.552 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.552 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181585+0000", 2024-09-14T06:18:10.552 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3271e-05 2024-09-14T06:18:10.552 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.552 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.553 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.553 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.553 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.553 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13239 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.553 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:18:10.553 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.389277176999997, 2024-09-14T06:18:10.553 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081998772999999997, 2024-09-14T06:18:10.553 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.553 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.553 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.553 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.553 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.553 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.553 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13239 2024-09-14T06:18:10.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.554 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:18:10.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:18:10.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099663+0000", 2024-09-14T06:18:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6100000000000001e-07 2024-09-14T06:18:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099666+0000", 2024-09-14T06:18:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.056e-06 2024-09-14T06:18:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099672+0000", 2024-09-14T06:18:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8120000000000002e-06 2024-09-14T06:18:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.555 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099676+0000", 2024-09-14T06:18:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:18:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167602+0000", 2024-09-14T06:18:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067925867000000001 2024-09-14T06:18:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167693+0000", 2024-09-14T06:18:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0736000000000007e-05 2024-09-14T06:18:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168065+0000", 2024-09-14T06:18:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037245399999999998 2024-09-14T06:18:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178003+0000", 2024-09-14T06:18:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0099372360000000003 2024-09-14T06:18:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181621+0000", 2024-09-14T06:18:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036178489999999998 2024-09-14T06:18:10.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181643+0000", 2024-09-14T06:18:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2155e-05 2024-09-14T06:18:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181661+0000", 2024-09-14T06:18:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8147000000000001e-05 2024-09-14T06:18:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.558 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13240 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:18:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.389158084000002, 2024-09-14T06:18:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081942607000000001, 2024-09-14T06:18:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13240 2024-09-14T06:18:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.559 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:18:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:18:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099782+0000", 2024-09-14T06:18:10.560 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3900000000000004e-07 2024-09-14T06:18:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099787+0000", 2024-09-14T06:18:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6189999999999999e-06 2024-09-14T06:18:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099793+0000", 2024-09-14T06:18:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.998e-06 2024-09-14T06:18:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.561 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099797+0000", 2024-09-14T06:18:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8449999999999996e-06 2024-09-14T06:18:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168494+0000", 2024-09-14T06:18:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.068696500999999993 2024-09-14T06:18:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168590+0000", 2024-09-14T06:18:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6733999999999999e-05 2024-09-14T06:18:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.562 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168973+0000", 2024-09-14T06:18:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038229399999999998 2024-09-14T06:18:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178013+0000", 2024-09-14T06:18:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0090397199999999994 2024-09-14T06:18:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181683+0000", 2024-09-14T06:18:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036705819999999999 2024-09-14T06:18:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.563 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181705+0000", 2024-09-14T06:18:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1551000000000001e-05 2024-09-14T06:18:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181724+0000", 2024-09-14T06:18:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9123999999999999e-05 2024-09-14T06:18:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.564 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13243 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:18:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.388624354000001, 2024-09-14T06:18:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081767815999999993, 2024-09-14T06:18:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13243 2024-09-14T06:18:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:18:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:18:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:18:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1e-07 2024-09-14T06:18:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.566 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100317+0000", 2024-09-14T06:18:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.092e-06 2024-09-14T06:18:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100321+0000", 2024-09-14T06:18:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7809999999999999e-06 2024-09-14T06:18:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100324+0000", 2024-09-14T06:18:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.117e-06 2024-09-14T06:18:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.567 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171113+0000", 2024-09-14T06:18:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.070788816000000004 2024-09-14T06:18:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171202+0000", 2024-09-14T06:18:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8843000000000003e-05 2024-09-14T06:18:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171584+0000", 2024-09-14T06:18:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038231099999999998 2024-09-14T06:18:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.568 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178101+0000", 2024-09-14T06:18:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0065168329999999997 2024-09-14T06:18:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182041+0000", 2024-09-14T06:18:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039401810000000001 2024-09-14T06:18:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182065+0000", 2024-09-14T06:18:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3623000000000001e-05 2024-09-14T06:18:10.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182083+0000", 2024-09-14T06:18:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7909000000000001e-05 2024-09-14T06:18:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13244 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:18:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.388609533999997, 2024-09-14T06:18:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081837800000000002, 2024-09-14T06:18:10.570 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13244 2024-09-14T06:18:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:18:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:18:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:18:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:18:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100337+0000", 2024-09-14T06:18:10.572 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.939e-06 2024-09-14T06:18:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100341+0000", 2024-09-14T06:18:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2320000000000003e-06 2024-09-14T06:18:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100344+0000", 2024-09-14T06:18:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0759999999999999e-06 2024-09-14T06:18:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.573 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171996+0000", 2024-09-14T06:18:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.071651919999999994 2024-09-14T06:18:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172079+0000", 2024-09-14T06:18:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2579999999999999e-05 2024-09-14T06:18:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172457+0000", 2024-09-14T06:18:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037785100000000001 2024-09-14T06:18:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.574 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178104+0000", 2024-09-14T06:18:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056471730000000001 2024-09-14T06:18:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182123+0000", 2024-09-14T06:18:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040190019999999998 2024-09-14T06:18:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182147+0000", 2024-09-14T06:18:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4241000000000001e-05 2024-09-14T06:18:10.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182168+0000", 2024-09-14T06:18:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0492999999999998e-05 2024-09-14T06:18:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13245 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:18:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.388590027000006, 2024-09-14T06:18:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081893899000000006, 2024-09-14T06:18:10.576 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13245 2024-09-14T06:18:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:18:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.577 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:18:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100350+0000", 2024-09-14T06:18:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5499999999999999e-07 2024-09-14T06:18:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100353+0000", 2024-09-14T06:18:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:18:10.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100357+0000", 2024-09-14T06:18:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8500000000000004e-06 2024-09-14T06:18:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100360+0000", 2024-09-14T06:18:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8700000000000001e-06 2024-09-14T06:18:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172862+0000", 2024-09-14T06:18:10.579 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.072501992000000001 2024-09-14T06:18:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172950+0000", 2024-09-14T06:18:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8988999999999998e-05 2024-09-14T06:18:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173324+0000", 2024-09-14T06:18:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037366199999999997 2024-09-14T06:18:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.580 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178107+0000", 2024-09-14T06:18:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047830520000000003 2024-09-14T06:18:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182202+0000", 2024-09-14T06:18:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040949280000000003 2024-09-14T06:18:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182226+0000", 2024-09-14T06:18:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3535000000000001e-05 2024-09-14T06:18:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182243+0000", 2024-09-14T06:18:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7445e-05 2024-09-14T06:18:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13246 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:18:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.388575704999994, 2024-09-14T06:18:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081962276000000001, 2024-09-14T06:18:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.582 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13246 2024-09-14T06:18:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:18:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.583 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:18:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:18:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0800000000000001e-07 2024-09-14T06:18:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100367+0000", 2024-09-14T06:18:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8439999999999998e-06 2024-09-14T06:18:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.584 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100372+0000", 2024-09-14T06:18:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4009999999999996e-06 2024-09-14T06:18:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100378+0000", 2024-09-14T06:18:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7180000000000001e-06 2024-09-14T06:18:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.585 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173780+0000", 2024-09-14T06:18:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.073401991 2024-09-14T06:18:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173879+0000", 2024-09-14T06:18:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9449000000000003e-05 2024-09-14T06:18:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.588 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174240+0000", 2024-09-14T06:18:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036118100000000002 2024-09-14T06:18:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178109+0000", 2024-09-14T06:18:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038690619999999999 2024-09-14T06:18:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182281+0000", 2024-09-14T06:18:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041715900000000002 2024-09-14T06:18:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.589 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182306+0000", 2024-09-14T06:18:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4477e-05 2024-09-14T06:18:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182326+0000", 2024-09-14T06:18:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0254999999999998e-05 2024-09-14T06:18:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13247 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:18:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.388478527999993, 2024-09-14T06:18:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966593000000004, 2024-09-14T06:18:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13247 2024-09-14T06:18:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:18:10.591 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:18:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100462+0000", 2024-09-14T06:18:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9800000000000002e-07 2024-09-14T06:18:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.592 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100466+0000", 2024-09-14T06:18:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4039999999999997e-06 2024-09-14T06:18:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100475+0000", 2024-09-14T06:18:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3680000000000008e-06 2024-09-14T06:18:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100482+0000", 2024-09-14T06:18:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8630000000000002e-06 2024-09-14T06:18:10.593 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174669+0000", 2024-09-14T06:18:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074186849999999999 2024-09-14T06:18:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174758+0000", 2024-09-14T06:18:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9021e-05 2024-09-14T06:18:10.594 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175137+0000", 2024-09-14T06:18:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378511 2024-09-14T06:18:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178158+0000", 2024-09-14T06:18:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003020956 2024-09-14T06:18:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.595 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182386+0000", 2024-09-14T06:18:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042278840000000003 2024-09-14T06:18:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182410+0000", 2024-09-14T06:18:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4026e-05 2024-09-14T06:18:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182427+0000", 2024-09-14T06:18:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7711999999999999e-05 2024-09-14T06:18:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.596 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13248 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:18:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.388396447000005, 2024-09-14T06:18:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966527999999997, 2024-09-14T06:18:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.597 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13248 2024-09-14T06:18:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:18:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:18:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:18:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:18:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100548+0000", 2024-09-14T06:18:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:18:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100557+0000", 2024-09-14T06:18:10.599 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:18:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100563+0000", 2024-09-14T06:18:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2519999999999996e-06 2024-09-14T06:18:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175572+0000", 2024-09-14T06:18:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075008459 2024-09-14T06:18:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.600 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175656+0000", 2024-09-14T06:18:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3906999999999995e-05 2024-09-14T06:18:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176010+0000", 2024-09-14T06:18:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00035447499999999998 2024-09-14T06:18:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.601 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178304+0000", 2024-09-14T06:18:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0022941379999999998 2024-09-14T06:18:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182467+0000", 2024-09-14T06:18:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041632020000000004 2024-09-14T06:18:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182491+0000", 2024-09-14T06:18:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3733000000000001e-05 2024-09-14T06:18:10.602 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182509+0000", 2024-09-14T06:18:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8113000000000001e-05 2024-09-14T06:18:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13249 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.603 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:18:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.388361513000007, 2024-09-14T06:18:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082037225000000005, 2024-09-14T06:18:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13249 2024-09-14T06:18:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.604 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:18:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:18:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:18:10.605 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4600000000000005e-07 2024-09-14T06:18:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100584+0000", 2024-09-14T06:18:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.947e-06 2024-09-14T06:18:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100593+0000", 2024-09-14T06:18:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9760000000000001e-06 2024-09-14T06:18:10.606 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100597+0000", 2024-09-14T06:18:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9689999999999996e-06 2024-09-14T06:18:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176461+0000", 2024-09-14T06:18:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075864132000000001 2024-09-14T06:18:10.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176516+0000", 2024-09-14T06:18:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4416999999999998e-05 2024-09-14T06:18:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176734+0000", 2024-09-14T06:18:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021859399999999999 2024-09-14T06:18:10.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178308+0000", 2024-09-14T06:18:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015733660000000001 2024-09-14T06:18:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182573+0000", 2024-09-14T06:18:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042649200000000002 2024-09-14T06:18:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.609 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182597+0000", 2024-09-14T06:18:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4355e-05 2024-09-14T06:18:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182615+0000", 2024-09-14T06:18:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8003e-05 2024-09-14T06:18:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.610 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13250 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:18:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 94.388196907999998, 2024-09-14T06:18:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081983890000000004, 2024-09-14T06:18:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13250 2024-09-14T06:18:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.611 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:18:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:18:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.612 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100743+0000", 2024-09-14T06:18:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:18:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100753+0000", 2024-09-14T06:18:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9979999999999998e-06 2024-09-14T06:18:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100772+0000", 2024-09-14T06:18:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8664e-05 2024-09-14T06:18:10.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.614 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.614 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:10.614 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100779+0000", 2024-09-14T06:18:10.614 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.046e-06 2024-09-14T06:18:10.614 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.614 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.614 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:10.614 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177049+0000", 2024-09-14T06:18:10.614 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076270819000000004 2024-09-14T06:18:10.614 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177140+0000", 2024-09-14T06:18:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0746000000000002e-05 2024-09-14T06:18:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177519+0000", 2024-09-14T06:18:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378656 2024-09-14T06:18:10.615 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178349+0000", 2024-09-14T06:18:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00083061599999999995 2024-09-14T06:18:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182679+0000", 2024-09-14T06:18:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043298379999999999 2024-09-14T06:18:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.616 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182705+0000", 2024-09-14T06:18:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5602999999999999e-05 2024-09-14T06:18:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182726+0000", 2024-09-14T06:18:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1297e-05 2024-09-14T06:18:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.617 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.618 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:10.618 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:10.618 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:10.619 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:18:10.620 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:10.615+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:10.720 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:10.717+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:10.791 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:10.792 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:10.792 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:18:10.792 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:10.806 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:18:10.852 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:10.851+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:10.922 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:10.922 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:10.922 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:18:10.922 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:18:10.922 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:10.922 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:10.919+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:10.936 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:18:11.023 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:11.021+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:11.110 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:11.110 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:18:11.110 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:18:11.110 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:18:11.110 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.110 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:30547 3.1f 3:f84dfb53:::rbd_data.11529eb229bd.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write 16226533~50642] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T06:18:11.110 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:18:11.110 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 71.917325778999995, 2024-09-14T06:18:11.111 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0079770020000001, 2024-09-14T06:18:11.111 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.111 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.111 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:11.111 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.111 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.111 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.111 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 30547 2024-09-14T06:18:11.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.111 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.111 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.111 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.112 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:18:11.112 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.112 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.112 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.112 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.112 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:18:11.112 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.112 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.112 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.112 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.112 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189811+0000", 2024-09-14T06:18:11.113 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2060000000000001e-06 2024-09-14T06:18:11.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.113 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.113 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.113 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189871+0000", 2024-09-14T06:18:11.113 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9862999999999999e-05 2024-09-14T06:18:11.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.113 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.113 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.113 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189899+0000", 2024-09-14T06:18:11.113 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7849000000000001e-05 2024-09-14T06:18:11.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.113 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.114 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.114 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189915+0000", 2024-09-14T06:18:11.114 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:18:11.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.114 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.114 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190024+0000", 2024-09-14T06:18:11.114 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000109202 2024-09-14T06:18:11.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.114 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:18:11.114 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190041+0000", 2024-09-14T06:18:11.115 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7665000000000001e-05 2024-09-14T06:18:11.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.115 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.115 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.115 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188512+0000", 2024-09-14T06:18:11.115 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.99847100700000002 2024-09-14T06:18:11.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.115 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.115 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for missing object", 2024-09-14T06:18:11.115 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188646+0000", 2024-09-14T06:18:11.115 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000133986 2024-09-14T06:18:11.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.116 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.116 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.116 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193097+0000", 2024-09-14T06:18:11.116 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044507059999999996 2024-09-14T06:18:11.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.116 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.116 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.116 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193183+0000", 2024-09-14T06:18:11.116 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5496999999999998e-05 2024-09-14T06:18:11.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.117 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.117 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:18:11.117 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193307+0000", 2024-09-14T06:18:11.117 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000124611 2024-09-14T06:18:11.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.117 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.117 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:11.117 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.194318+0000", 2024-09-14T06:18:11.117 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0010106 2024-09-14T06:18:11.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.117 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.117 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:11.117 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197670+0000", 2024-09-14T06:18:11.117 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033523170000000001 2024-09-14T06:18:11.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:11.118 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197712+0000", 2024-09-14T06:18:11.118 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1681000000000003e-05 2024-09-14T06:18:11.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.118 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197786+0000", 2024-09-14T06:18:11.118 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4148999999999998e-05 2024-09-14T06:18:11.118 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.118 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.118 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.119 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32022 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:11.119 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:18:11.119 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 67.238945568000005, 2024-09-14T06:18:11.119 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095598104000000003, 2024-09-14T06:18:11.119 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.119 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.119 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.119 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.119 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.119 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.119 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32022 2024-09-14T06:18:11.119 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.119 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.119 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.120 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.120 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:18:11.120 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.120 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.120 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.120 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.120 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:18:11.120 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.120 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.120 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.120 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.120 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868190+0000", 2024-09-14T06:18:11.120 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6000000000000003e-07 2024-09-14T06:18:11.120 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.121 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.121 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868243+0000", 2024-09-14T06:18:11.121 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3443999999999999e-05 2024-09-14T06:18:11.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.121 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.121 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868260+0000", 2024-09-14T06:18:11.121 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6799e-05 2024-09-14T06:18:11.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.121 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.121 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868271+0000", 2024-09-14T06:18:11.121 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0523000000000001e-05 2024-09-14T06:18:11.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.122 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.122 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.122 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963502+0000", 2024-09-14T06:18:11.122 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095231466000000001 2024-09-14T06:18:11.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.122 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.122 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.122 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963636+0000", 2024-09-14T06:18:11.122 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013388699999999999 2024-09-14T06:18:11.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.122 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.122 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.122 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963787+0000", 2024-09-14T06:18:11.122 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000151225 2024-09-14T06:18:11.123 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.123 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.123 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.123 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.123 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.123 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32023 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5816320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:11.123 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:18:11.123 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 67.238850442, 2024-09-14T06:18:11.123 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095879753999999998, 2024-09-14T06:18:11.123 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.123 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.123 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.123 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.123 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.123 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.123 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32023 2024-09-14T06:18:11.124 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.124 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.124 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.124 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.124 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:18:11.124 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.124 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.124 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.124 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.124 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:18:11.124 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.124 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.124 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.124 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.125 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868285+0000", 2024-09-14T06:18:11.125 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6299999999999993e-07 2024-09-14T06:18:11.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.125 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.125 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.125 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868288+0000", 2024-09-14T06:18:11.125 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.18e-06 2024-09-14T06:18:11.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.125 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.125 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.125 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868298+0000", 2024-09-14T06:18:11.125 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0029e-05 2024-09-14T06:18:11.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.125 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.126 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.126 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868307+0000", 2024-09-14T06:18:11.126 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7229999999999993e-06 2024-09-14T06:18:11.126 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.126 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.126 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.126 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963871+0000", 2024-09-14T06:18:11.126 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095563876000000006 2024-09-14T06:18:11.126 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.126 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.126 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.126 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964015+0000", 2024-09-14T06:18:11.126 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143573 2024-09-14T06:18:11.126 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.126 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.127 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.127 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964164+0000", 2024-09-14T06:18:11.127 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014940999999999999 2024-09-14T06:18:11.127 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.127 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.127 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.127 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.127 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.127 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32024 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:11.127 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:18:11.127 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 67.238817298000001, 2024-09-14T06:18:11.127 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096230231999999999, 2024-09-14T06:18:11.127 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.127 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.128 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.128 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.128 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.128 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.128 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32024 2024-09-14T06:18:11.128 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.128 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.128 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.128 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.128 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:18:11.128 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.128 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.128 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.128 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.128 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:18:11.129 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.129 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.129 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.129 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.129 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868318+0000", 2024-09-14T06:18:11.129 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1699999999999997e-07 2024-09-14T06:18:11.129 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.129 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.129 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.129 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868321+0000", 2024-09-14T06:18:11.129 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6769999999999999e-06 2024-09-14T06:18:11.129 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.129 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.129 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.129 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868334+0000", 2024-09-14T06:18:11.130 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2928999999999999e-05 2024-09-14T06:18:11.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.130 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.130 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.130 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868341+0000", 2024-09-14T06:18:11.130 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9800000000000001e-06 2024-09-14T06:18:11.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.130 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.130 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.130 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964258+0000", 2024-09-14T06:18:11.130 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095917099000000006 2024-09-14T06:18:11.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.130 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.130 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.131 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964392+0000", 2024-09-14T06:18:11.131 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013396999999999999 2024-09-14T06:18:11.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.131 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.131 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.131 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964548+0000", 2024-09-14T06:18:11.131 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015585999999999999 2024-09-14T06:18:11.131 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.131 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.131 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.131 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.131 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32025 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:11.132 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:18:11.132 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 67.238784641999999, 2024-09-14T06:18:11.132 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096576591000000003, 2024-09-14T06:18:11.132 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.132 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.132 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.132 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.132 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.132 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.132 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32025 2024-09-14T06:18:11.132 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.133 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.133 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.133 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.133 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:18:11.133 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.133 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.133 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.133 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.134 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:18:11.134 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.134 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.134 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.134 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868351+0000", 2024-09-14T06:18:11.134 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1600000000000001e-07 2024-09-14T06:18:11.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.134 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.134 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.134 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868353+0000", 2024-09-14T06:18:11.134 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.627e-06 2024-09-14T06:18:11.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.135 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.135 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.135 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868364+0000", 2024-09-14T06:18:11.135 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0893000000000001e-05 2024-09-14T06:18:11.135 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.135 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.135 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.135 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868381+0000", 2024-09-14T06:18:11.135 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6769000000000002e-05 2024-09-14T06:18:11.135 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.135 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.135 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.135 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964632+0000", 2024-09-14T06:18:11.135 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096250589999999997 2024-09-14T06:18:11.136 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.136 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.136 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.136 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964772+0000", 2024-09-14T06:18:11.136 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014045299999999999 2024-09-14T06:18:11.136 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.136 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.136 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.136 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964927+0000", 2024-09-14T06:18:11.136 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015454299999999999 2024-09-14T06:18:11.136 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.136 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.136 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.136 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.137 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.137 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32026 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:11.137 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:18:11.137 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 67.238743690999996, 2024-09-14T06:18:11.137 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096901622000000007, 2024-09-14T06:18:11.137 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.137 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.137 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.137 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.137 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.137 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.137 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32026 2024-09-14T06:18:11.137 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.137 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.138 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.138 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.138 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:18:11.138 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.138 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.138 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.138 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:18:11.138 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.138 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.138 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.138 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868392+0000", 2024-09-14T06:18:11.138 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0699999999999996e-07 2024-09-14T06:18:11.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.139 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.139 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.139 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868395+0000", 2024-09-14T06:18:11.139 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9859999999999999e-06 2024-09-14T06:18:11.139 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.139 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.139 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.139 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868405+0000", 2024-09-14T06:18:11.139 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9229999999999997e-06 2024-09-14T06:18:11.139 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.139 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.139 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.139 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868412+0000", 2024-09-14T06:18:11.140 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9670000000000004e-06 2024-09-14T06:18:11.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.140 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.140 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.140 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965010+0000", 2024-09-14T06:18:11.140 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096598562999999998 2024-09-14T06:18:11.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.140 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.140 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.140 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965143+0000", 2024-09-14T06:18:11.140 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013247700000000001 2024-09-14T06:18:11.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.140 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.141 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.141 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965293+0000", 2024-09-14T06:18:11.141 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000149899 2024-09-14T06:18:11.141 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.141 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.141 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.141 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.141 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.141 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32027 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:11.141 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:18:11.141 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 67.238675353999994, 2024-09-14T06:18:11.141 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097211306999999997, 2024-09-14T06:18:11.141 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.141 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.142 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.142 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.142 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.142 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.142 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32027 2024-09-14T06:18:11.142 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.142 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.142 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.142 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.142 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:18:11.142 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.142 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.142 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.142 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.143 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:18:11.143 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.143 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.143 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.143 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868461+0000", 2024-09-14T06:18:11.143 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.296e-06 2024-09-14T06:18:11.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.143 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.143 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.143 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868465+0000", 2024-09-14T06:18:11.143 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9570000000000004e-06 2024-09-14T06:18:11.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.143 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.144 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.144 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868476+0000", 2024-09-14T06:18:11.144 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1365999999999999e-05 2024-09-14T06:18:11.144 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.144 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.144 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.144 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868483+0000", 2024-09-14T06:18:11.144 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2529999999999998e-06 2024-09-14T06:18:11.144 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.144 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.144 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.144 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965375+0000", 2024-09-14T06:18:11.144 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096891973000000006 2024-09-14T06:18:11.144 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.145 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.145 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.145 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965518+0000", 2024-09-14T06:18:11.145 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143017 2024-09-14T06:18:11.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.145 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.145 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.145 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965671+0000", 2024-09-14T06:18:11.145 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015244499999999999 2024-09-14T06:18:11.145 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.145 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.145 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.145 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.146 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32028 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:11.146 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:18:11.146 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 67.238640525999998, 2024-09-14T06:18:11.146 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097841480999999994, 2024-09-14T06:18:11.146 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.146 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.146 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.146 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.146 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.146 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.146 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32028 2024-09-14T06:18:11.146 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.146 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.146 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.147 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.147 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:18:11.147 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.147 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.147 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.147 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:18:11.147 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.147 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.147 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.147 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868495+0000", 2024-09-14T06:18:11.147 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6400000000000001e-07 2024-09-14T06:18:11.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.147 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.148 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.148 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868498+0000", 2024-09-14T06:18:11.148 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7760000000000002e-06 2024-09-14T06:18:11.148 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.148 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.148 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.148 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868507+0000", 2024-09-14T06:18:11.148 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1530000000000003e-06 2024-09-14T06:18:11.148 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.148 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.148 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.148 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868516+0000", 2024-09-14T06:18:11.148 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8929999999999994e-06 2024-09-14T06:18:11.148 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.149 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.149 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.149 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965753+0000", 2024-09-14T06:18:11.149 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097236970000000006 2024-09-14T06:18:11.149 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.149 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.149 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.149 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965890+0000", 2024-09-14T06:18:11.149 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013677599999999999 2024-09-14T06:18:11.149 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.149 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.149 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.149 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966336+0000", 2024-09-14T06:18:11.149 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00044614900000000001 2024-09-14T06:18:11.150 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.150 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.150 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.150 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.150 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.150 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32029 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:11.150 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:18:11.150 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 67.238609451000002, 2024-09-14T06:18:11.150 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098577779000000004, 2024-09-14T06:18:11.150 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.150 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.150 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.150 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.150 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.151 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.151 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32029 2024-09-14T06:18:11.151 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.151 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.151 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.151 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.151 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:18:11.151 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.151 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.151 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.151 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.151 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:18:11.151 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.151 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.151 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.152 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.152 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868526+0000", 2024-09-14T06:18:11.152 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9300000000000006e-07 2024-09-14T06:18:11.152 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.152 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.152 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.152 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868529+0000", 2024-09-14T06:18:11.152 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5969999999999999e-06 2024-09-14T06:18:11.152 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.152 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.152 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.152 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868538+0000", 2024-09-14T06:18:11.152 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1700000000000003e-06 2024-09-14T06:18:11.153 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.153 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.153 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.153 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868560+0000", 2024-09-14T06:18:11.153 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1596000000000001e-05 2024-09-14T06:18:11.153 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.153 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.153 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.153 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966423+0000", 2024-09-14T06:18:11.153 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097863396000000005 2024-09-14T06:18:11.153 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.153 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.153 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.154 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966578+0000", 2024-09-14T06:18:11.154 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000154792 2024-09-14T06:18:11.154 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.154 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.154 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.154 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967103+0000", 2024-09-14T06:18:11.154 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00052523499999999998 2024-09-14T06:18:11.154 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.154 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.154 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.154 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.154 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.154 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32030 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:11.154 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:18:11.154 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 67.238564533000002, 2024-09-14T06:18:11.155 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099324030999999993, 2024-09-14T06:18:11.155 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.155 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.155 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.155 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.155 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.155 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.155 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32030 2024-09-14T06:18:11.155 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.155 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.155 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.155 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.155 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:18:11.155 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.156 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.156 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.156 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.156 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:18:11.156 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.156 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.156 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.156 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.156 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868571+0000", 2024-09-14T06:18:11.156 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9699999999999995e-07 2024-09-14T06:18:11.156 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.156 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.156 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.157 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868574+0000", 2024-09-14T06:18:11.157 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9529999999999999e-06 2024-09-14T06:18:11.157 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.157 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.157 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.157 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868584+0000", 2024-09-14T06:18:11.157 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7860000000000008e-06 2024-09-14T06:18:11.157 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.157 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.157 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.157 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868591+0000", 2024-09-14T06:18:11.157 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0500000000000003e-06 2024-09-14T06:18:11.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.158 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.158 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.158 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967191+0000", 2024-09-14T06:18:11.158 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098600035000000003 2024-09-14T06:18:11.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.158 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.158 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.158 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967333+0000", 2024-09-14T06:18:11.158 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014190599999999999 2024-09-14T06:18:11.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.158 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.159 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.159 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967894+0000", 2024-09-14T06:18:11.159 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000561504 2024-09-14T06:18:11.159 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.159 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.159 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.159 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.159 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32031 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:11.159 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:18:11.159 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 67.238534971000007, 2024-09-14T06:18:11.159 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.10010279, 2024-09-14T06:18:11.159 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.160 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.160 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.160 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.160 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.160 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.160 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32031 2024-09-14T06:18:11.160 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.160 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.160 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.160 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.160 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:18:11.160 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.160 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.160 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.161 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.161 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:18:11.161 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.161 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.161 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.161 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.161 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:18:11.161 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:18:11.161 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.161 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.161 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.161 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868603+0000", 2024-09-14T06:18:11.161 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.599e-06 2024-09-14T06:18:11.162 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.162 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.162 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.162 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868612+0000", 2024-09-14T06:18:11.162 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.986e-06 2024-09-14T06:18:11.162 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.162 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.162 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.162 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868619+0000", 2024-09-14T06:18:11.162 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8900000000000001e-06 2024-09-14T06:18:11.162 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.162 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.162 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.163 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967983+0000", 2024-09-14T06:18:11.163 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099363612000000004 2024-09-14T06:18:11.163 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.163 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.163 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.163 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968140+0000", 2024-09-14T06:18:11.163 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015788600000000001 2024-09-14T06:18:11.163 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.163 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.163 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.163 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968703+0000", 2024-09-14T06:18:11.163 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00056208300000000001 2024-09-14T06:18:11.163 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.164 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.164 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.164 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.164 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32032 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6258688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:11.164 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:18:11.164 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 67.238506495999999, 2024-09-14T06:18:11.164 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100488088, 2024-09-14T06:18:11.164 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.164 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.164 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.164 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.164 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.164 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.165 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32032 2024-09-14T06:18:11.165 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.165 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.165 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.165 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.165 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:18:11.165 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.165 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.165 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.165 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.165 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:18:11.165 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.165 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.165 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.166 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.166 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868629+0000", 2024-09-14T06:18:11.166 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:18:11.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.166 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.166 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.166 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868631+0000", 2024-09-14T06:18:11.166 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4099999999999998e-06 2024-09-14T06:18:11.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.166 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.166 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.166 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868640+0000", 2024-09-14T06:18:11.166 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0939999999999997e-06 2024-09-14T06:18:11.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.167 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.167 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.167 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868647+0000", 2024-09-14T06:18:11.167 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9460000000000001e-06 2024-09-14T06:18:11.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.167 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.167 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.167 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968797+0000", 2024-09-14T06:18:11.167 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100149867 2024-09-14T06:18:11.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.167 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.167 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.167 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968937+0000", 2024-09-14T06:18:11.168 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013981399999999999 2024-09-14T06:18:11.168 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.168 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.168 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.168 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969116+0000", 2024-09-14T06:18:11.168 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017927399999999999 2024-09-14T06:18:11.168 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.168 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.168 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.168 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.168 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.168 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32044 3.1e 3:7e6619d4:::rbd_data.115274ed0aa8.0000000000000068:head [set-alloc-hint object_size 1048576 write_size 1048576,write 835584~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:11.168 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:18:11.168 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 67.230817412999997, 2024-09-14T06:18:11.169 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096039367, 2024-09-14T06:18:11.169 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.169 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.169 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:11.169 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.169 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.169 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.169 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32044 2024-09-14T06:18:11.169 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.169 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.169 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.169 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.170 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:18:11.170 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.170 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.170 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.170 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.170 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:18:11.170 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.170 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.170 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.170 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.172 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876320+0000", 2024-09-14T06:18:11.172 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8270000000000002e-06 2024-09-14T06:18:11.172 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.172 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.172 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.172 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876397+0000", 2024-09-14T06:18:11.172 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6985999999999998e-05 2024-09-14T06:18:11.172 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.172 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.172 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.173 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876431+0000", 2024-09-14T06:18:11.173 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3719e-05 2024-09-14T06:18:11.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.173 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.173 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.173 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876452+0000", 2024-09-14T06:18:11.173 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0891999999999999e-05 2024-09-14T06:18:11.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.173 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.173 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.173 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969186+0000", 2024-09-14T06:18:11.173 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.092734681999999999 2024-09-14T06:18:11.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.174 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.174 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.174 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969550+0000", 2024-09-14T06:18:11.174 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036402199999999998 2024-09-14T06:18:11.174 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.174 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.174 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:18:11.174 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969987+0000", 2024-09-14T06:18:11.174 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000436759 2024-09-14T06:18:11.174 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.174 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.174 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:11.174 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971569+0000", 2024-09-14T06:18:11.174 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001581821 2024-09-14T06:18:11.174 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.175 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.175 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:11.175 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971947+0000", 2024-09-14T06:18:11.175 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378446 2024-09-14T06:18:11.175 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.175 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.175 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:11.175 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972146+0000", 2024-09-14T06:18:11.175 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019819 2024-09-14T06:18:11.175 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.175 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.175 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.175 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972357+0000", 2024-09-14T06:18:11.175 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021102299999999999 2024-09-14T06:18:11.175 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.176 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.176 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.176 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.176 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.176 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40252 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 827392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:11.176 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:18:11.176 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.390181306999999, 2024-09-14T06:18:11.176 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095633474999999996, 2024-09-14T06:18:11.176 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.176 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.176 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.176 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.176 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.176 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.177 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40252 2024-09-14T06:18:11.177 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.177 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.177 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.177 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.177 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:18:11.177 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.177 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.177 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.177 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.177 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:18:11.177 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.177 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.177 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.177 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.178 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717376+0000", 2024-09-14T06:18:11.178 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000422216 2024-09-14T06:18:11.178 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.178 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.178 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.178 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717386+0000", 2024-09-14T06:18:11.178 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0396e-05 2024-09-14T06:18:11.178 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.178 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.178 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.178 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717395+0000", 2024-09-14T06:18:11.178 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7819999999999999e-06 2024-09-14T06:18:11.178 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.178 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.178 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.179 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717400+0000", 2024-09-14T06:18:11.179 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1599999999999997e-06 2024-09-14T06:18:11.179 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.179 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.179 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.179 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.811922+0000", 2024-09-14T06:18:11.179 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.094522018999999999 2024-09-14T06:18:11.179 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.179 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.179 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.179 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812007+0000", 2024-09-14T06:18:11.179 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5328999999999996e-05 2024-09-14T06:18:11.179 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.179 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.179 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.179 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812587+0000", 2024-09-14T06:18:11.180 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000579573 2024-09-14T06:18:11.180 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.180 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.180 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.180 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.180 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.180 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40254 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:11.180 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:18:11.180 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.389723134, 2024-09-14T06:18:11.180 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095562731999999997, 2024-09-14T06:18:11.180 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.180 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.180 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.180 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.181 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.181 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.181 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40254 2024-09-14T06:18:11.181 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.181 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.181 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.181 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.181 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:18:11.181 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.181 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.181 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.181 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.181 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:18:11.181 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.181 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.182 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.182 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.182 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:18:11.182 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1699999999999999e-07 2024-09-14T06:18:11.182 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.182 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.182 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.182 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717414+0000", 2024-09-14T06:18:11.182 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:18:11.182 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.182 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.182 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.182 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717419+0000", 2024-09-14T06:18:11.183 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2839999999999997e-06 2024-09-14T06:18:11.183 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.183 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.183 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.183 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717424+0000", 2024-09-14T06:18:11.183 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1549999999999998e-06 2024-09-14T06:18:11.183 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.183 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.183 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.183 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812619+0000", 2024-09-14T06:18:11.183 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095194940000000006 2024-09-14T06:18:11.183 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.183 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.184 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.184 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812717+0000", 2024-09-14T06:18:11.184 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7647999999999997e-05 2024-09-14T06:18:11.184 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.184 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.184 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.184 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812974+0000", 2024-09-14T06:18:11.184 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025752099999999998 2024-09-14T06:18:11.184 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.184 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.184 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.184 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.184 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.185 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40256 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 843776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:11.185 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:18:11.185 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.389700529999999, 2024-09-14T06:18:11.185 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095932066999999996, 2024-09-14T06:18:11.185 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.185 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.185 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.185 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.185 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.185 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.185 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40256 2024-09-14T06:18:11.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.185 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.185 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:18:11.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:18:11.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.186 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.186 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717628+0000", 2024-09-14T06:18:11.186 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019339499999999999 2024-09-14T06:18:11.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.186 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717632+0000", 2024-09-14T06:18:11.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2230000000000001e-06 2024-09-14T06:18:11.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.187 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.187 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717641+0000", 2024-09-14T06:18:11.187 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6309999999999998e-06 2024-09-14T06:18:11.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.188 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.188 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717656+0000", 2024-09-14T06:18:11.188 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4724e-05 2024-09-14T06:18:11.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.188 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.188 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.188 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813006+0000", 2024-09-14T06:18:11.188 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095349801999999997 2024-09-14T06:18:11.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.188 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.188 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.188 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813107+0000", 2024-09-14T06:18:11.188 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000100973 2024-09-14T06:18:11.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.189 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.189 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.189 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813366+0000", 2024-09-14T06:18:11.189 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025931899999999999 2024-09-14T06:18:11.189 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.189 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.189 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.189 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.189 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40258 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 851968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:11.189 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:18:11.189 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.389464429999997, 2024-09-14T06:18:11.189 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096062543, 2024-09-14T06:18:11.189 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.190 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.190 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.190 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.190 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.190 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.190 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40258 2024-09-14T06:18:11.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.190 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.190 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.190 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:18:11.190 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.190 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.191 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:18:11.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.191 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717671+0000", 2024-09-14T06:18:11.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8299999999999997e-07 2024-09-14T06:18:11.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.191 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.191 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.191 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717674+0000", 2024-09-14T06:18:11.191 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7080000000000002e-06 2024-09-14T06:18:11.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717681+0000", 2024-09-14T06:18:11.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7889999999999995e-06 2024-09-14T06:18:11.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717687+0000", 2024-09-14T06:18:11.192 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7529999999999998e-06 2024-09-14T06:18:11.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.192 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.192 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.192 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813408+0000", 2024-09-14T06:18:11.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095720451999999998 2024-09-14T06:18:11.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.193 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.193 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813499+0000", 2024-09-14T06:18:11.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0933000000000005e-05 2024-09-14T06:18:11.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.193 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.193 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.193 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813733+0000", 2024-09-14T06:18:11.193 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00023432499999999999 2024-09-14T06:18:11.193 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.193 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.193 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.194 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40260 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 860160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:11.194 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:18:11.194 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.389438581, 2024-09-14T06:18:11.194 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096337381999999999, 2024-09-14T06:18:11.194 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.194 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.194 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.194 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.194 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.194 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.194 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40260 2024-09-14T06:18:11.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.194 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.194 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.194 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:18:11.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:18:11.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718008+0000", 2024-09-14T06:18:11.195 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00031192699999999998 2024-09-14T06:18:11.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.195 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.195 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.195 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718013+0000", 2024-09-14T06:18:11.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1830000000000002e-06 2024-09-14T06:18:11.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.196 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718036+0000", 2024-09-14T06:18:11.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2721000000000002e-05 2024-09-14T06:18:11.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.196 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718053+0000", 2024-09-14T06:18:11.196 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6736e-05 2024-09-14T06:18:11.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.196 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.196 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.196 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813767+0000", 2024-09-14T06:18:11.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095714658999999994 2024-09-14T06:18:11.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813816+0000", 2024-09-14T06:18:11.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8848999999999999e-05 2024-09-14T06:18:11.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.197 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.197 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814034+0000", 2024-09-14T06:18:11.197 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000217307 2024-09-14T06:18:11.197 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.197 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.197 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.198 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40262 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 868352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:11.198 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:18:11.198 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.389038898000003, 2024-09-14T06:18:11.198 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096354165000000006, 2024-09-14T06:18:11.198 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.198 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.198 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.198 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.198 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.198 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.198 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40262 2024-09-14T06:18:11.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.198 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.198 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.199 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:18:11.199 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.199 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:18:11.199 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.199 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.199 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718281+0000", 2024-09-14T06:18:11.199 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000185069 2024-09-14T06:18:11.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.199 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.200 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.200 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718286+0000", 2024-09-14T06:18:11.200 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.118e-06 2024-09-14T06:18:11.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.200 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.200 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.200 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718295+0000", 2024-09-14T06:18:11.200 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0429999999999996e-06 2024-09-14T06:18:11.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.200 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.200 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.200 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718302+0000", 2024-09-14T06:18:11.200 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2790000000000001e-06 2024-09-14T06:18:11.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.200 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.201 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.201 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814076+0000", 2024-09-14T06:18:11.201 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095774076999999999 2024-09-14T06:18:11.201 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.201 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.201 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.201 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814191+0000", 2024-09-14T06:18:11.201 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00011500700000000001 2024-09-14T06:18:11.201 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.201 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.201 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.201 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814450+0000", 2024-09-14T06:18:11.201 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025857199999999999 2024-09-14T06:18:11.201 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.201 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.202 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.202 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.202 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40264 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 876544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:11.202 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:18:11.202 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 61.388681830000003, 2024-09-14T06:18:11.202 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096372478999999997, 2024-09-14T06:18:11.202 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.202 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.202 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:11.202 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.202 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.202 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.202 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40264 2024-09-14T06:18:11.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.202 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718793+0000", 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00033991099999999998 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718805+0000", 2024-09-14T06:18:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2514000000000001e-05 2024-09-14T06:18:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718815+0000", 2024-09-14T06:18:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9590000000000007e-06 2024-09-14T06:18:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718822+0000", 2024-09-14T06:18:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2030000000000003e-06 2024-09-14T06:18:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.205 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814490+0000", 2024-09-14T06:18:11.205 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095667346 2024-09-14T06:18:11.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.205 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.205 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.205 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814577+0000", 2024-09-14T06:18:11.205 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6920000000000001e-05 2024-09-14T06:18:11.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.205 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.205 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.205 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814825+0000", 2024-09-14T06:18:11.205 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000248626 2024-09-14T06:18:11.205 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.205 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.205 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.206 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.206 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.206 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:11.207 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:18:11.208 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:11.203+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:11.209 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T06:18:11.209 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-09-14T06:18:11.244 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T06:18:11.244 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:18:11.308 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:11.307+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:11.405 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:11.405 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:11.405 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:18:11.405 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:11.409 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:11.407+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:11.420 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:18:11.510 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:11.507+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:11.579 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:11.579 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:11.579 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:18:11.579 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:18:11.580 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:11.594 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:18:11.632 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:18:11.633 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:18:11.640 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:11.639+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:11.652 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-14T06:18:11.213479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":271580496,"num_objects":653,"num_object_clones":77,"num_object_copies":1306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":653,"num_whiteouts":258,"num_read":105202,"num_read_kb":749393,"num_write":16972,"num_write_kb":430797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1206,"num_bytes_recovered":144366785,"num_keys_recovered":219,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14864,"ondisk_log_size":14864,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":385540,"kb_used_data":31808,"kb_used_omap":12,"kb_used_meta":353395,"kb_avail":754589180,"statfs":{"total":773094113280,"available":772699320320,"internally_reserved":0,"allocated":32571392,"data_stored":219803681,"data_compressed":6478403,"data_compressed_allocated":15679488,"data_compressed_original":209660209,"omap_allocated":12752,"internal_metadata":361877040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-92578896,"num_objects":-68,"num_object_clones":-15,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":-6,"num_read":894,"num_read_kb":770,"num_write":149,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.796751"},"pg_stats":[{"pgid":"3.1f","version":"112'579","reported_seq":3591,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.415819+0000","last_change":"2024-09-14T06:18:06.415819+0000","last_active":"2024-09-14T06:18:06.415819+0000","last_peered":"2024-09-14T06:18:06.415819+0000","last_clean":"2024-09-14T06:18:06.415819+0000","last_became_active":"2024-09-14T06:17:02.193313+0000","last_became_peered":"2024-09-14T06:17:02.193313+0000","last_unstale":"2024-09-14T06:18:06.415819+0000","last_undegraded":"2024-09-14T06:18:06.415819+0000","last_fullsized":"2024-09-14T06:18:06.415819+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:42.502193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001774461,"stat_sum":{"num_bytes":6904730,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":4583,"num_read_kb":53580,"num_write":691,"num_write_kb":19350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.1e","version":"112'387","reported_seq":4431,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.586063+0000","last_change":"2024-09-14T06:18:06.414503+0000","last_active":"2024-09-14T06:18:06.586063+0000","last_peered":"2024-09-14T06:18:06.586063+0000","last_clean":"2024-09-14T06:18:06.586063+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:18:06.586063+0000","last_undegraded":"2024-09-14T06:18:06.586063+0000","last_fullsized":"2024-09-14T06:18:06.586063+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:03:55.670662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001177558,"stat_sum":{"num_bytes":9814314,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":11,"num_read":4997,"num_read_kb":43044,"num_write":334,"num_write_kb":15481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.1d","version":"111'1115","reported_seq":2450,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.428791+0000","last_change":"2024-09-14T06:18:07.429809+0000","last_active":"2024-09-14T06:18:07.428791+0000","last_peered":"2024-09-14T06:18:07.428791+0000","last_clean":"2024-09-14T06:18:07.428791+0000","last_became_active":"2024-09-14T06:17:38.226838+0000","last_became_peered":"2024-09-14T06:17:38.226838+0000","last_unstale":"2024-09-14T06:18:07.428791+0000","last_undegraded":"2024-09-14T06:18:07.428791+0000","last_fullsized":"2024-09-14T06:18:07.428791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:52:42.905891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089295869999999992,"stat_sum":{"num_bytes":10470999,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":5125,"num_read_kb":14378,"num_write":1566,"num_write_kb":13994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10785266,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.1c","version":"112'894","reported_seq":2853,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.556482+0000","last_change":"2024-09-14T06:18:06.394725+0000","last_active":"2024-09-14T06:18:06.556482+0000","last_peered":"2024-09-14T06:18:06.556482+0000","last_clean":"2024-09-14T06:18:06.556482+0000","last_became_active":"2024-09-14T06:17:48.737020+0000","last_became_peered":"2024-09-14T06:17:48.737020+0000","last_unstale":"2024-09-14T06:18:06.556482+0000","last_undegraded":"2024-09-14T06:18:06.556482+0000","last_fullsized":"2024-09-14T06:18:06.556482+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:30.667772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023015549999999998,"stat_sum":{"num_bytes":524288,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":5685,"num_read_kb":9726,"num_write":1303,"num_write_kb":6518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":3869232,"num_keys_recovered":68,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.1b","version":"112'332","reported_seq":3720,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.393914+0000","last_change":"2024-09-14T06:18:06.393914+0000","last_active":"2024-09-14T06:18:06.393914+0000","last_peered":"2024-09-14T06:18:06.393914+0000","last_clean":"2024-09-14T06:18:06.393914+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:18:06.393914+0000","last_undegraded":"2024-09-14T06:18:06.393914+0000","last_fullsized":"2024-09-14T06:18:06.393914+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:41:12.786161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016475140000000001,"stat_sum":{"num_bytes":7304576,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":3342,"num_read_kb":39056,"num_write":353,"num_write_kb":25723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.1a","version":"113'526","reported_seq":3227,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.558803+0000","last_change":"2024-09-14T06:18:07.431193+0000","last_active":"2024-09-14T06:18:07.558803+0000","last_peered":"2024-09-14T06:18:07.558803+0000","last_clean":"2024-09-14T06:18:07.558803+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:18:07.558803+0000","last_undegraded":"2024-09-14T06:18:07.558803+0000","last_fullsized":"2024-09-14T06:18:07.558803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:16:04.128598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032412640000000002,"stat_sum":{"num_bytes":5121647,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":17,"num_read":4031,"num_read_kb":13071,"num_write":651,"num_write_kb":7511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.19","version":"112'478","reported_seq":1963,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.512432+0000","last_change":"2024-09-14T06:18:06.393452+0000","last_active":"2024-09-14T06:18:06.512432+0000","last_peered":"2024-09-14T06:18:06.512432+0000","last_clean":"2024-09-14T06:18:06.512432+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:18:06.512432+0000","last_undegraded":"2024-09-14T06:18:06.512432+0000","last_fullsized":"2024-09-14T06:18:06.512432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:18:54.562166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012335219999999999,"stat_sum":{"num_bytes":6022104,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":11,"num_read":2684,"num_read_kb":19064,"num_write":663,"num_write_kb":12862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.18","version":"111'368","reported_seq":2900,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.561319+0000","last_change":"2024-09-14T06:18:06.415172+0000","last_active":"2024-09-14T06:18:06.561319+0000","last_peered":"2024-09-14T06:18:06.561319+0000","last_clean":"2024-09-14T06:18:06.561319+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:18:06.561319+0000","last_undegraded":"2024-09-14T06:18:06.561319+0000","last_fullsized":"2024-09-14T06:18:06.561319+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:42:04.319290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017036670000000001,"stat_sum":{"num_bytes":10388823,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":3627,"num_read_kb":37626,"num_write":336,"num_write_kb":32935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.17","version":"109'367","reported_seq":1663,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.414916+0000","last_change":"2024-09-14T06:18:06.415489+0000","last_active":"2024-09-14T06:18:06.414916+0000","last_peered":"2024-09-14T06:18:06.414916+0000","last_clean":"2024-09-14T06:18:06.414916+0000","last_became_active":"2024-09-14T06:17:37.233410+0000","last_became_peered":"2024-09-14T06:17:37.233410+0000","last_unstale":"2024-09-14T06:18:06.414916+0000","last_undegraded":"2024-09-14T06:18:06.414916+0000","last_fullsized":"2024-09-14T06:18:06.414916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:23:31.419313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019166090000000001,"stat_sum":{"num_bytes":13230263,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":1385,"num_read_kb":17453,"num_write":316,"num_write_kb":17188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":23467377,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.16","version":"109'418","reported_seq":2324,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.429660+0000","last_change":"2024-09-14T06:18:07.430245+0000","last_active":"2024-09-14T06:18:07.429660+0000","last_peered":"2024-09-14T06:18:07.429660+0000","last_clean":"2024-09-14T06:18:07.429660+0000","last_became_active":"2024-09-14T06:17:37.235311+0000","last_became_peered":"2024-09-14T06:17:37.235311+0000","last_unstale":"2024-09-14T06:18:07.429660+0000","last_undegraded":"2024-09-14T06:18:07.429660+0000","last_fullsized":"2024-09-14T06:18:07.429660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:40:44.096601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086075779999999994,"stat_sum":{"num_bytes":5327508,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":2483,"num_read_kb":16488,"num_write":502,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":6785399,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.15","version":"113'397","reported_seq":3650,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.576546+0000","last_change":"2024-09-14T06:18:07.427988+0000","last_active":"2024-09-14T06:18:07.576546+0000","last_peered":"2024-09-14T06:18:07.576546+0000","last_clean":"2024-09-14T06:18:07.576546+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:18:07.576546+0000","last_undegraded":"2024-09-14T06:18:07.576546+0000","last_fullsized":"2024-09-14T06:18:07.576546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:15:45.199242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020002610000000001,"stat_sum":{"num_bytes":12093895,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":4034,"num_read_kb":64540,"num_write":487,"num_write_kb":22218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.14","version":"112'294","reported_seq":2003,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.428107+0000","last_change":"2024-09-14T06:18:07.428692+0000","last_active":"2024-09-14T06:18:07.428107+0000","last_peered":"2024-09-14T06:18:07.428107+0000","last_clean":"2024-09-14T06:18:07.428107+0000","last_became_active":"2024-09-14T06:17:37.231244+0000","last_became_peered":"2024-09-14T06:17:37.231244+0000","last_unstale":"2024-09-14T06:18:07.428107+0000","last_undegraded":"2024-09-14T06:18:07.428107+0000","last_fullsized":"2024-09-14T06:18:07.428107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:30:56.042466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082021679999999993,"stat_sum":{"num_bytes":11530735,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":2907,"num_read_kb":27191,"num_write":261,"num_write_kb":15734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":32506492,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.420805+0000","last_change":"2024-09-14T06:17:34.200709+0000","last_active":"2024-09-14T06:18:07.420805+0000","last_peered":"2024-09-14T06:18:07.420805+0000","last_clean":"2024-09-14T06:18:07.420805+0000","last_became_active":"2024-09-14T06:17:34.200238+0000","last_became_peered":"2024-09-14T06:17:34.200238+0000","last_unstale":"2024-09-14T06:18:07.420805+0000","last_undegraded":"2024-09-14T06:18:07.420805+0000","last_fullsized":"2024-09-14T06:18:07.420805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:34.864695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"112'452","reported_seq":1238,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.534352+0000","last_change":"2024-09-14T06:18:06.394712+0000","last_active":"2024-09-14T06:18:06.534352+0000","last_peered":"2024-09-14T06:18:06.534352+0000","last_clean":"2024-09-14T06:18:06.534352+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:18:06.534352+0000","last_undegraded":"2024-09-14T06:18:06.534352+0000","last_fullsized":"2024-09-14T06:18:06.534352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:24.513821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002054574,"stat_sum":{"num_bytes":1998996,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":1519,"num_read_kb":9696,"num_write":471,"num_write_kb":9164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"2.6","version":"0'0","reported_seq":200,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.392849+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:18:06.392849+0000","last_peered":"2024-09-14T06:18:06.392849+0000","last_clean":"2024-09-14T06:18:06.392849+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:18:06.392849+0000","last_undegraded":"2024-09-14T06:18:06.392849+0000","last_fullsized":"2024-09-14T06:18:06.392849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"112'360","reported_seq":3198,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.394070+0000","last_change":"2024-09-14T06:18:06.394070+0000","last_active":"2024-09-14T06:18:06.394070+0000","last_peered":"2024-09-14T06:18:06.394070+0000","last_clean":"2024-09-14T06:18:06.394070+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:18:06.394070+0000","last_undegraded":"2024-09-14T06:18:06.394070+0000","last_fullsized":"2024-09-14T06:18:06.394070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:39.034290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016880980000000001,"stat_sum":{"num_bytes":17349459,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":3293,"num_read_kb":28583,"num_write":324,"num_write_kb":15237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":73,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.421358+0000","last_change":"2024-09-14T06:17:34.196668+0000","last_active":"2024-09-14T06:18:07.421358+0000","last_peered":"2024-09-14T06:18:07.421358+0000","last_clean":"2024-09-14T06:18:07.421358+0000","last_became_active":"2024-09-14T06:17:34.196072+0000","last_became_peered":"2024-09-14T06:17:34.196072+0000","last_unstale":"2024-09-14T06:18:07.421358+0000","last_undegraded":"2024-09-14T06:18:07.421358+0000","last_fullsized":"2024-09-14T06:18:07.421358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:18:45.616834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"112'348","reported_seq":1919,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.515246+0000","last_change":"2024-09-14T06:18:06.485171+0000","last_active":"2024-09-14T06:18:06.515246+0000","last_peered":"2024-09-14T06:18:06.515246+0000","last_clean":"2024-09-14T06:18:06.515246+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:18:06.515246+0000","last_undegraded":"2024-09-14T06:18:06.515246+0000","last_fullsized":"2024-09-14T06:18:06.515246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:53:38.747989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058769599999999996,"stat_sum":{"num_bytes":23108758,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":1780,"num_read_kb":12934,"num_write":296,"num_write_kb":11582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":200,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.413014+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:18:06.413014+0000","last_peered":"2024-09-14T06:18:06.413014+0000","last_clean":"2024-09-14T06:18:06.413014+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:18:06.413014+0000","last_undegraded":"2024-09-14T06:18:06.413014+0000","last_fullsized":"2024-09-14T06:18:06.413014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"111'263","reported_seq":1657,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.485266+0000","last_change":"2024-09-14T06:18:06.485266+0000","last_active":"2024-09-14T06:18:06.485266+0000","last_peered":"2024-09-14T06:18:06.485266+0000","last_clean":"2024-09-14T06:18:06.485266+0000","last_became_active":"2024-09-14T06:16:33.547925+0000","last_became_peered":"2024-09-14T06:16:33.547925+0000","last_unstale":"2024-09-14T06:18:06.485266+0000","last_undegraded":"2024-09-14T06:18:06.485266+0000","last_fullsized":"2024-09-14T06:18:06.485266+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:44:27.666578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066389200000000002,"stat_sum":{"num_bytes":5118396,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":11,"num_read":1354,"num_read_kb":21258,"num_write":257,"num_write_kb":9954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.3","version":"113'353","reported_seq":1085,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.551377+0000","last_change":"2024-09-14T06:18:07.429214+0000","last_active":"2024-09-14T06:18:07.551377+0000","last_peered":"2024-09-14T06:18:07.551377+0000","last_clean":"2024-09-14T06:18:07.551377+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:18:07.551377+0000","last_undegraded":"2024-09-14T06:18:07.551377+0000","last_fullsized":"2024-09-14T06:18:07.551377+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:39:31.110478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028864540000000001,"stat_sum":{"num_bytes":5199972,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":8,"num_read":499,"num_read_kb":4207,"num_write":278,"num_write_kb":7109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"2.2","version":"22'2","reported_seq":77,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.420164+0000","last_change":"2024-09-14T06:17:34.203977+0000","last_active":"2024-09-14T06:18:07.420164+0000","last_peered":"2024-09-14T06:18:07.420164+0000","last_clean":"2024-09-14T06:18:07.420164+0000","last_became_active":"2024-09-14T06:17:34.199244+0000","last_became_peered":"2024-09-14T06:17:34.199244+0000","last_unstale":"2024-09-14T06:18:07.420164+0000","last_undegraded":"2024-09-14T06:18:07.420164+0000","last_fullsized":"2024-09-14T06:18:07.420164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:21.934094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"112'394","reported_seq":1798,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.480245+0000","last_change":"2024-09-14T06:18:06.414776+0000","last_active":"2024-09-14T06:18:06.480245+0000","last_peered":"2024-09-14T06:18:06.480245+0000","last_clean":"2024-09-14T06:18:06.480245+0000","last_became_active":"2024-09-14T06:16:20.196215+0000","last_became_peered":"2024-09-14T06:16:20.196215+0000","last_unstale":"2024-09-14T06:18:06.480245+0000","last_undegraded":"2024-09-14T06:18:06.480245+0000","last_fullsized":"2024-09-14T06:18:06.480245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:53.216371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013626510000000001,"stat_sum":{"num_bytes":11634292,"num_objects":20,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":2093,"num_read_kb":18967,"num_write":510,"num_write_kb":11051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":200,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.480746+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:18:06.480746+0000","last_peered":"2024-09-14T06:18:06.480746+0000","last_clean":"2024-09-14T06:18:06.480746+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:18:06.480746+0000","last_undegraded":"2024-09-14T06:18:06.480746+0000","last_fullsized":"2024-09-14T06:18:06.480746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"112'534","reported_seq":2138,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.484555+0000","last_change":"2024-09-14T06:18:06.484555+0000","last_active":"2024-09-14T06:18:06.484555+0000","last_peered":"2024-09-14T06:18:06.484555+0000","last_clean":"2024-09-14T06:18:06.484555+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:18:06.484555+0000","last_undegraded":"2024-09-14T06:18:06.484555+0000","last_fullsized":"2024-09-14T06:18:06.484555+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:24:06.789564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066675299999999996,"stat_sum":{"num_bytes":3869323,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":6,"num_read":2885,"num_read_kb":8812,"num_write":717,"num_write_kb":8784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"2.0","version":"0'0","reported_seq":202,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.419866+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:18:07.419866+0000","last_peered":"2024-09-14T06:18:07.419866+0000","last_clean":"2024-09-14T06:18:07.419866+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:18:07.419866+0000","last_undegraded":"2024-09-14T06:18:07.419866+0000","last_fullsized":"2024-09-14T06:18:07.419866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"111'448","reported_seq":1951,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.395523+0000","last_change":"2024-09-14T06:18:06.395523+0000","last_active":"2024-09-14T06:18:06.395523+0000","last_peered":"2024-09-14T06:18:06.395523+0000","last_clean":"2024-09-14T06:18:06.395523+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:18:06.395523+0000","last_undegraded":"2024-09-14T06:18:06.395523+0000","last_fullsized":"2024-09-14T06:18:06.395523+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:44:51.198049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023997739999999999,"stat_sum":{"num_bytes":2415784,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":2196,"num_read_kb":22997,"num_write":357,"num_write_kb":14523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"2.3","version":"20'1","reported_seq":223,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.420431+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:18:07.420431+0000","last_peered":"2024-09-14T06:18:07.420431+0000","last_clean":"2024-09-14T06:18:07.420431+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:18:07.420431+0000","last_undegraded":"2024-09-14T06:18:07.420431+0000","last_fullsized":"2024-09-14T06:18:07.420431+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":294,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.427920+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:18:07.427920+0000","last_peered":"2024-09-14T06:18:07.427920+0000","last_clean":"2024-09-14T06:18:07.427920+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:18:07.427920+0000","last_undegraded":"2024-09-14T06:18:07.427920+0000","last_fullsized":"2024-09-14T06:18:07.427920+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"111'426","reported_seq":1594,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.392674+0000","last_change":"2024-09-14T06:18:06.393352+0000","last_active":"2024-09-14T06:18:06.392674+0000","last_peered":"2024-09-14T06:18:06.392674+0000","last_clean":"2024-09-14T06:18:06.392674+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:18:06.392674+0000","last_undegraded":"2024-09-14T06:18:06.392674+0000","last_fullsized":"2024-09-14T06:18:06.392674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:26.008208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012336490000000001,"stat_sum":{"num_bytes":8259504,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":2130,"num_read_kb":6714,"num_write":581,"num_write_kb":6849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.9","version":"111'385","reported_seq":1801,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.427709+0000","last_change":"2024-09-14T06:18:07.428373+0000","last_active":"2024-09-14T06:18:07.427709+0000","last_peered":"2024-09-14T06:18:07.427709+0000","last_clean":"2024-09-14T06:18:07.427709+0000","last_became_active":"2024-09-14T06:17:38.224238+0000","last_became_peered":"2024-09-14T06:17:38.224238+0000","last_unstale":"2024-09-14T06:18:07.427709+0000","last_undegraded":"2024-09-14T06:18:07.427709+0000","last_fullsized":"2024-09-14T06:18:07.427709+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:47:02.196929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0078946569999999994,"stat_sum":{"num_bytes":8186252,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":2223,"num_read_kb":13651,"num_write":538,"num_write_kb":14353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8477378,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.a","version":"113'345","reported_seq":2233,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.528087+0000","last_change":"2024-09-14T06:18:07.427556+0000","last_active":"2024-09-14T06:18:07.528087+0000","last_peered":"2024-09-14T06:18:07.528087+0000","last_clean":"2024-09-14T06:18:07.528087+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:18:07.528087+0000","last_undegraded":"2024-09-14T06:18:07.528087+0000","last_fullsized":"2024-09-14T06:18:07.528087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:14:20.340187+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.007139505,"stat_sum":{"num_bytes":13141585,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":2211,"num_read_kb":16098,"num_write":288,"num_write_kb":13801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.b","version":"112'305","reported_seq":2823,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.481642+0000","last_change":"2024-09-14T06:18:06.481642+0000","last_active":"2024-09-14T06:18:06.481642+0000","last_peered":"2024-09-14T06:18:06.481642+0000","last_clean":"2024-09-14T06:18:06.481642+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:18:06.481642+0000","last_undegraded":"2024-09-14T06:18:06.481642+0000","last_fullsized":"2024-09-14T06:18:06.481642+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:10:09.819416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073018699999999998,"stat_sum":{"num_bytes":6658155,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":9,"num_read":2867,"num_read_kb":16792,"num_write":263,"num_write_kb":9901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.c","version":"109'181","reported_seq":1600,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.426266+0000","last_change":"2024-09-14T06:18:07.426577+0000","last_active":"2024-09-14T06:18:07.426266+0000","last_peered":"2024-09-14T06:18:07.426266+0000","last_clean":"2024-09-14T06:18:07.426266+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:18:07.426266+0000","last_undegraded":"2024-09-14T06:18:07.426266+0000","last_fullsized":"2024-09-14T06:18:07.426266+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:50:03.117816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069750400000000005,"stat_sum":{"num_bytes":5966452,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":1287,"num_read_kb":29081,"num_write":240,"num_write_kb":11577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.d","version":"110'662","reported_seq":2521,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.428847+0000","last_change":"2024-09-14T06:18:07.429566+0000","last_active":"2024-09-14T06:18:07.428847+0000","last_peered":"2024-09-14T06:18:07.428847+0000","last_clean":"2024-09-14T06:18:07.428847+0000","last_became_active":"2024-09-14T06:17:38.227106+0000","last_became_peered":"2024-09-14T06:17:38.227106+0000","last_unstale":"2024-09-14T06:18:07.428847+0000","last_undegraded":"2024-09-14T06:18:07.428847+0000","last_fullsized":"2024-09-14T06:18:07.428847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:35:22.181122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082906309999999997,"stat_sum":{"num_bytes":14571487,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":3737,"num_read_kb":26925,"num_write":972,"num_write_kb":20272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":29390214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.e","version":"112'730","reported_seq":5234,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.426348+0000","last_change":"2024-09-14T06:18:07.426821+0000","last_active":"2024-09-14T06:18:07.426348+0000","last_peered":"2024-09-14T06:18:07.426348+0000","last_clean":"2024-09-14T06:18:07.426348+0000","last_became_active":"2024-09-14T06:16:33.546796+0000","last_became_peered":"2024-09-14T06:16:33.546796+0000","last_unstale":"2024-09-14T06:18:07.426348+0000","last_undegraded":"2024-09-14T06:18:07.426348+0000","last_fullsized":"2024-09-14T06:18:07.426348+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:50:06.834775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085063399999999996,"stat_sum":{"num_bytes":6677733,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":9974,"num_read_kb":51459,"num_write":672,"num_write_kb":16128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.f","version":"113'638","reported_seq":2876,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.427073+0000","last_change":"2024-09-14T06:18:07.427073+0000","last_active":"2024-09-14T06:18:07.427073+0000","last_peered":"2024-09-14T06:18:07.427073+0000","last_clean":"2024-09-14T06:18:07.427073+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:18:07.427073+0000","last_undegraded":"2024-09-14T06:18:07.427073+0000","last_fullsized":"2024-09-14T06:18:07.427073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:52:10.795745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001110277,"stat_sum":{"num_bytes":20745733,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":4425,"num_read_kb":16166,"num_write":753,"num_write_kb":12749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.10","version":"112'346","reported_seq":2500,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.428504+0000","last_change":"2024-09-14T06:18:07.429118+0000","last_active":"2024-09-14T06:18:07.428504+0000","last_peered":"2024-09-14T06:18:07.428504+0000","last_clean":"2024-09-14T06:18:07.428504+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:18:07.428504+0000","last_undegraded":"2024-09-14T06:18:07.428504+0000","last_fullsized":"2024-09-14T06:18:07.428504+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:33:37.310744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001526257,"stat_sum":{"num_bytes":2960020,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":2476,"num_read_kb":47684,"num_write":331,"num_write_kb":14643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.11","version":"113'504","reported_seq":2758,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.427957+0000","last_change":"2024-09-14T06:18:07.427957+0000","last_active":"2024-09-14T06:18:07.427957+0000","last_peered":"2024-09-14T06:18:07.427957+0000","last_clean":"2024-09-14T06:18:07.427957+0000","last_became_active":"2024-09-14T06:17:38.224251+0000","last_became_peered":"2024-09-14T06:17:38.224251+0000","last_unstale":"2024-09-14T06:18:07.427957+0000","last_undegraded":"2024-09-14T06:18:07.427957+0000","last_fullsized":"2024-09-14T06:18:07.427957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:24:25.122706+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0079625440000000002,"stat_sum":{"num_bytes":5301360,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":4155,"num_read_kb":14398,"num_write":570,"num_write_kb":7659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":12239008,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.12","version":"111'490","reported_seq":2955,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.484247+0000","last_change":"2024-09-14T06:18:06.484746+0000","last_active":"2024-09-14T06:18:06.484247+0000","last_peered":"2024-09-14T06:18:06.484247+0000","last_clean":"2024-09-14T06:18:06.484247+0000","last_became_active":"2024-09-14T06:17:37.232145+0000","last_became_peered":"2024-09-14T06:17:37.232145+0000","last_unstale":"2024-09-14T06:18:06.484247+0000","last_undegraded":"2024-09-14T06:18:06.484247+0000","last_fullsized":"2024-09-14T06:18:06.484247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:33:45.201867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082120399999999997,"stat_sum":{"num_bytes":4194340,"num_objects":26,"num_object_clones":2,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":3314,"num_read_kb":11024,"num_write":581,"num_write_kb":8452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":10176554,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.13","version":"112'470","reported_seq":4015,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.421279+0000","last_change":"2024-09-14T06:18:07.422009+0000","last_active":"2024-09-14T06:18:07.421279+0000","last_peered":"2024-09-14T06:18:07.421279+0000","last_clean":"2024-09-14T06:18:07.421279+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:18:07.421279+0000","last_undegraded":"2024-09-14T06:18:07.421279+0000","last_fullsized":"2024-09-14T06:18:07.421279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:02.011457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606756,"stat_sum":{"num_bytes":4898626,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":12,"num_read":5835,"num_read_kb":16675,"num_write":407,"num_write_kb":5582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":270990109,"num_objects":649,"num_object_clones":77,"num_object_copies":1298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":649,"num_whiteouts":258,"num_read":105136,"num_read_kb":749338,"num_write":16869,"num_write_kb":429167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1203,"num_bytes_recovered":144366766,"num_keys_recovered":219,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19578880,"data_stored":186162857,"data_compressed":5514866,"data_compressed_allocated":14131200,"data_compressed_original":181697201,"omap_allocated":0,"internal_metadata":0},"log_size":14789,"ondisk_log_size":14789,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40780,"kb_used_data":5428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331060,"statfs":{"total":96636764160,"available":96595005440,"internally_reserved":0,"allocated":5558272,"data_stored":44684378,"data_compressed":1527427,"data_compressed_allocated":3481600,"data_compressed_original":43470848,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34936,"kb_used_data":3748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336904,"statfs":{"total":96636764160,"available":96600989696,"internally_reserved":0,"allocated":3837952,"data_stored":23271982,"data_compressed":499786,"data_compressed_allocated":1654784,"data_compressed_original":21961659,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75068,"kb_used_data":3480,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296772,"statfs":{"total":96636764160,"available":96559894528,"internally_reserved":0,"allocated":3563520,"data_stored":23246212,"data_compressed":708650,"data_compressed_allocated":1744896,"data_compressed_original":22229480,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57632,"kb_used_data":4232,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314208,"statfs":{"total":96636764160,"available":96577748992,"internally_reserved":0,"allocated":4333568,"data_stored":29792745,"data_compressed":748911,"data_compressed_allocated":2088960,"data_compressed_original":28389324,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37656,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334184,"statfs":{"total":96636764160,"available":96598204416,"internally_reserved":0,"allocated":2363392,"data_stored":9570825,"data_compressed":227688,"data_compressed_allocated":602112,"data_compressed_original":8588073,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44032,"kb_used_data":4524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327808,"statfs":{"total":96636764160,"available":96591675392,"internally_reserved":0,"allocated":4632576,"data_stored":31815492,"data_compressed":949994,"data_compressed_allocated":2203648,"data_compressed_original":30273750,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48768,"kb_used_data":5088,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94323072,"statfs":{"total":96636764160,"available":96586825728,"internally_reserved":0,"allocated":5210112,"data_stored":38980912,"data_compressed":1390843,"data_compressed_allocated":2809856,"data_compressed_original":37463211,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46668,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325172,"statfs":{"total":96636764160,"available":96588976128,"internally_reserved":0,"allocated":3072000,"data_stored":18441135,"data_compressed":425104,"data_compressed_allocated":1093632,"data_compressed_original":17283864,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2609152,"data_stored":23339657,"data_compressed":563815,"data_compressed_allocated":1765376,"data_compressed_original":22622156,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2002944,"data_stored":18870213,"data_compressed":561112,"data_compressed_allocated":1552384,"data_compressed_original":18510312,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2396160,"data_stored":19500133,"data_compressed":447404,"data_compressed_allocated":1548288,"data_compressed_original":18815931,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3682304,"data_stored":33940530,"data_compressed":1105265,"data_compressed_allocated":3006464,"data_compressed_original":33411072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1744896,"data_stored":17815086,"data_compressed":425104,"data_compressed_allocated":1093632,"data_compressed_original":17283864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3297280,"data_stored":34953226,"data_compressed":1332009,"data_compressed_allocated":2576384,"data_compressed_original":34350635,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2994176,"data_stored":29936032,"data_compressed":873927,"data_compressed_allocated":2068480,"data_compressed_original":29163734,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":851968,"data_stored":7807980,"data_compressed":206230,"data_compressed_allocated":520192,"data_compressed_original":7539497,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:18:11.654 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:18:11.690 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:18:11.712 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:11.711+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:11.763 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:11.764 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:18:11.764 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:18:11.764 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:18:11.764 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.764 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:3595 3.1 3:811c1c24:::rbd_data.1152bd92e04f.000000000000006e:head [set-alloc-hint object_size 2097152 write_size 2097152,write 440183~2185] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T06:18:11.764 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:18:11.764 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 105.538517689, 2024-09-14T06:18:11.764 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0365806689999999, 2024-09-14T06:18:11.764 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.764 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.764 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.764 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.764 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.764 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.765 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 3595 2024-09-14T06:18:11.765 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.765 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.765 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.765 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.765 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:18:11.765 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.765 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.765 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.765 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.765 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:18:11.765 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.765 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.766 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.766 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.766 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222823+0000", 2024-09-14T06:18:11.766 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8219999999999998e-06 2024-09-14T06:18:11.766 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.766 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.766 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.766 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222835+0000", 2024-09-14T06:18:11.766 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2052000000000001e-05 2024-09-14T06:18:11.766 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.766 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.766 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.766 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222859+0000", 2024-09-14T06:18:11.766 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3694e-05 2024-09-14T06:18:11.766 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.767 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.767 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.767 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222872+0000", 2024-09-14T06:18:11.767 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3203999999999999e-05 2024-09-14T06:18:11.767 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.767 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.767 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.767 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223068+0000", 2024-09-14T06:18:11.767 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000195995 2024-09-14T06:18:11.767 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.767 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.767 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:11.767 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223312+0000", 2024-09-14T06:18:11.767 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024424199999999998 2024-09-14T06:18:11.767 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.768 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.768 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:11.768 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223786+0000", 2024-09-14T06:18:11.768 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00047356600000000003 2024-09-14T06:18:11.768 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.768 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.768 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:11.768 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.225374+0000", 2024-09-14T06:18:11.768 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015875150000000001 2024-09-14T06:18:11.768 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.768 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.768 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.768 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.233122+0000", 2024-09-14T06:18:11.769 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0077480700000000001 2024-09-14T06:18:11.769 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.769 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.769 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:18:11.769 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.233157+0000", 2024-09-14T06:18:11.769 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5113000000000001e-05 2024-09-14T06:18:11.769 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.769 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.769 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.769 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.252998+0000", 2024-09-14T06:18:11.769 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.019841142 2024-09-14T06:18:11.769 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.769 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.769 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for degraded object", 2024-09-14T06:18:11.770 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.254121+0000", 2024-09-14T06:18:11.770 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0011227979999999999 2024-09-14T06:18:11.770 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.770 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.770 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.770 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.259263+0000", 2024-09-14T06:18:11.770 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0051426700000000002 2024-09-14T06:18:11.770 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.770 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.770 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.770 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.259401+0000", 2024-09-14T06:18:11.770 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013778600000000001 2024-09-14T06:18:11.770 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.770 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.770 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.770 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.771 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.771 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11421 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.771 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:18:11.771 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.775622502999994, 2024-09-14T06:18:11.771 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078553170000000005, 2024-09-14T06:18:11.771 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.771 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.771 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.771 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.771 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.771 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.771 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11421 2024-09-14T06:18:11.771 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.771 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.771 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.772 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.772 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:18:11.772 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.772 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.772 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.772 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.772 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:18:11.772 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.772 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.772 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.772 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.772 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985718+0000", 2024-09-14T06:18:11.772 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8840000000000001e-06 2024-09-14T06:18:11.772 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.772 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.773 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.773 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985723+0000", 2024-09-14T06:18:11.773 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8799999999999996e-06 2024-09-14T06:18:11.773 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.773 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.773 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.773 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985742+0000", 2024-09-14T06:18:11.773 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8776000000000001e-05 2024-09-14T06:18:11.773 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.773 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.773 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.773 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985759+0000", 2024-09-14T06:18:11.773 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7119999999999999e-05 2024-09-14T06:18:11.773 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.774 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.774 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.774 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064259+0000", 2024-09-14T06:18:11.774 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078499418000000001 2024-09-14T06:18:11.774 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.774 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.774 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.774 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064269+0000", 2024-09-14T06:18:11.774 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:18:11.774 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.774 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.774 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.774 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.775 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.775 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11422 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.775 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:18:11.775 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.775552536000006, 2024-09-14T06:18:11.775 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078521540000000001, 2024-09-14T06:18:11.775 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.775 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.775 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.775 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.775 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.775 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.775 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11422 2024-09-14T06:18:11.775 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.775 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.776 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.776 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.776 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:18:11.776 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.776 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.776 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.776 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.776 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:18:11.776 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.776 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.776 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.776 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.776 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985787+0000", 2024-09-14T06:18:11.776 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7630000000000001e-06 2024-09-14T06:18:11.776 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.776 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.777 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.777 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985842+0000", 2024-09-14T06:18:11.777 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4555999999999997e-05 2024-09-14T06:18:11.777 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.777 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.777 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.777 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985864+0000", 2024-09-14T06:18:11.777 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2189999999999999e-05 2024-09-14T06:18:11.777 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.777 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.777 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.777 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985879+0000", 2024-09-14T06:18:11.777 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4382999999999999e-05 2024-09-14T06:18:11.777 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.778 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.778 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.778 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064298+0000", 2024-09-14T06:18:11.778 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078419121999999994 2024-09-14T06:18:11.778 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.778 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.778 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.778 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064307+0000", 2024-09-14T06:18:11.778 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5259999999999999e-06 2024-09-14T06:18:11.778 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.778 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.778 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.778 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.778 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.778 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11423 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.778 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:18:11.779 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.775435164000001, 2024-09-14T06:18:11.779 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078437175999999997, 2024-09-14T06:18:11.779 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.779 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.779 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.779 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.779 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.779 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.779 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11423 2024-09-14T06:18:11.779 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.779 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.779 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.779 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.779 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:18:11.779 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.780 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.780 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.780 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.780 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:18:11.780 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.780 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.780 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.780 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.780 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985905+0000", 2024-09-14T06:18:11.780 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8139999999999999e-06 2024-09-14T06:18:11.780 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.780 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.780 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.780 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985911+0000", 2024-09-14T06:18:11.780 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7230000000000001e-06 2024-09-14T06:18:11.781 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.781 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.781 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.781 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985930+0000", 2024-09-14T06:18:11.781 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9360000000000001e-05 2024-09-14T06:18:11.781 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.781 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.781 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.781 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985944+0000", 2024-09-14T06:18:11.781 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4479e-05 2024-09-14T06:18:11.781 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.781 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.781 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.781 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064332+0000", 2024-09-14T06:18:11.781 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078387686999999998 2024-09-14T06:18:11.782 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.782 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.782 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.782 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064340+0000", 2024-09-14T06:18:11.782 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1130000000000001e-06 2024-09-14T06:18:11.782 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.782 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.782 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.782 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.782 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.782 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11480 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.782 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:18:11.782 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.774084688000002, 2024-09-14T06:18:11.782 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077122298000000006, 2024-09-14T06:18:11.782 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.783 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.783 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.783 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.783 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.783 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.783 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11480 2024-09-14T06:18:11.783 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.783 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.783 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.783 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.783 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:18:11.783 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.783 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.783 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.783 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.783 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:18:11.784 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.784 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.784 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.784 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.784 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987255+0000", 2024-09-14T06:18:11.784 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:18:11.784 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.784 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.784 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.784 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987263+0000", 2024-09-14T06:18:11.784 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6499999999999996e-06 2024-09-14T06:18:11.784 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.784 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.784 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.784 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987285+0000", 2024-09-14T06:18:11.785 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1860000000000001e-05 2024-09-14T06:18:11.785 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.785 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.785 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.785 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987341+0000", 2024-09-14T06:18:11.785 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6648999999999999e-05 2024-09-14T06:18:11.785 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.785 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.785 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.785 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064367+0000", 2024-09-14T06:18:11.785 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077025622000000002 2024-09-14T06:18:11.785 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.785 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.786 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.786 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064376+0000", 2024-09-14T06:18:11.786 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7499999999999992e-06 2024-09-14T06:18:11.786 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.786 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.786 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.786 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.786 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.786 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11481 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.786 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:18:11.786 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.773948985999994, 2024-09-14T06:18:11.786 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077022057000000005, 2024-09-14T06:18:11.786 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.786 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.786 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.787 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.787 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.787 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.787 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11481 2024-09-14T06:18:11.787 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.787 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.787 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.787 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.787 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:18:11.787 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.787 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.787 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.787 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.787 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:18:11.787 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.787 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.788 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.788 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.788 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987391+0000", 2024-09-14T06:18:11.788 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4839999999999999e-06 2024-09-14T06:18:11.788 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.788 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.788 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.788 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987399+0000", 2024-09-14T06:18:11.788 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8660000000000006e-06 2024-09-14T06:18:11.788 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.788 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.788 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.788 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987419+0000", 2024-09-14T06:18:11.788 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9984e-05 2024-09-14T06:18:11.788 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.789 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.789 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.789 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987434+0000", 2024-09-14T06:18:11.789 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5396e-05 2024-09-14T06:18:11.789 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.789 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.789 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.789 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064403+0000", 2024-09-14T06:18:11.789 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076969212999999995 2024-09-14T06:18:11.789 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.789 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.789 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.789 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064411+0000", 2024-09-14T06:18:11.790 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1140000000000006e-06 2024-09-14T06:18:11.790 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.790 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.790 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.790 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.790 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.790 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11482 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.790 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:18:11.790 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.773864126000007, 2024-09-14T06:18:11.790 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076971172000000004, 2024-09-14T06:18:11.790 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.790 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.790 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.790 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.790 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.791 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.791 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11482 2024-09-14T06:18:11.791 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.791 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.791 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.791 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.791 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:18:11.791 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.791 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.791 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.791 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.791 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:18:11.791 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.791 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.791 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.792 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.792 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987476+0000", 2024-09-14T06:18:11.792 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.793e-06 2024-09-14T06:18:11.792 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.792 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.792 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.792 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987482+0000", 2024-09-14T06:18:11.792 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8270000000000003e-06 2024-09-14T06:18:11.792 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.792 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.792 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.792 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987502+0000", 2024-09-14T06:18:11.792 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0656e-05 2024-09-14T06:18:11.792 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.793 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.793 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.793 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987518+0000", 2024-09-14T06:18:11.793 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:18:11.793 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.793 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.793 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.793 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064438+0000", 2024-09-14T06:18:11.793 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076919623000000006 2024-09-14T06:18:11.793 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.793 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.793 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.793 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064445+0000", 2024-09-14T06:18:11.793 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.61e-06 2024-09-14T06:18:11.794 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.794 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.794 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.794 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.794 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.794 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11483 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.794 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:18:11.794 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.773796277000002, 2024-09-14T06:18:11.794 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076936562999999999, 2024-09-14T06:18:11.794 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.794 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.794 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.794 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.794 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.794 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.794 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11483 2024-09-14T06:18:11.795 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.795 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.795 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.795 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.795 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:18:11.795 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.795 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.795 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.795 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.795 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:18:11.795 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.795 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.795 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.795 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.796 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987544+0000", 2024-09-14T06:18:11.796 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.654e-06 2024-09-14T06:18:11.796 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.796 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.796 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.796 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987550+0000", 2024-09-14T06:18:11.796 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0299999999999999e-06 2024-09-14T06:18:11.796 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.796 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.796 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.796 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987570+0000", 2024-09-14T06:18:11.796 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0080000000000001e-05 2024-09-14T06:18:11.796 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.796 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.796 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.797 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987619+0000", 2024-09-14T06:18:11.797 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9228999999999999e-05 2024-09-14T06:18:11.797 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.797 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.797 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.797 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064471+0000", 2024-09-14T06:18:11.797 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076852086999999999 2024-09-14T06:18:11.797 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.797 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.797 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.797 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064478+0000", 2024-09-14T06:18:11.797 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4830000000000002e-06 2024-09-14T06:18:11.797 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.797 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.797 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.798 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.798 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.798 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11485 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.798 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:18:11.798 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.773697784000007, 2024-09-14T06:18:11.798 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076868463999999997, 2024-09-14T06:18:11.798 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.798 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.798 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.798 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.798 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.798 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.798 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11485 2024-09-14T06:18:11.798 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.798 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.799 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.799 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.799 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:18:11.799 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.799 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.799 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.799 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.799 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:18:11.799 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.799 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.799 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.799 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.799 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987642+0000", 2024-09-14T06:18:11.799 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4100000000000001e-06 2024-09-14T06:18:11.800 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.800 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.800 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.800 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987648+0000", 2024-09-14T06:18:11.800 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2500000000000003e-06 2024-09-14T06:18:11.800 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.800 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.800 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.800 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987683+0000", 2024-09-14T06:18:11.800 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5193e-05 2024-09-14T06:18:11.800 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.800 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.800 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.800 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987698+0000", 2024-09-14T06:18:11.800 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4893e-05 2024-09-14T06:18:11.800 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.801 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.801 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.801 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064501+0000", 2024-09-14T06:18:11.801 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076803263999999996 2024-09-14T06:18:11.801 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.801 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.801 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.801 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064509+0000", 2024-09-14T06:18:11.801 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4540000000000001e-06 2024-09-14T06:18:11.801 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.801 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.801 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.801 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.801 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.801 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11490 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.802 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:18:11.802 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.773548215000005, 2024-09-14T06:18:11.802 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076755043999999994, 2024-09-14T06:18:11.802 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.802 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.802 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.802 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.802 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.802 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.802 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11490 2024-09-14T06:18:11.802 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.802 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.802 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.802 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.803 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:18:11.803 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.803 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.803 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.803 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.803 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:18:11.803 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.803 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.803 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.803 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.803 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987791+0000", 2024-09-14T06:18:11.803 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:18:11.803 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.803 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.803 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.803 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987797+0000", 2024-09-14T06:18:11.804 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7969999999999997e-06 2024-09-14T06:18:11.804 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.804 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.804 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.804 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987817+0000", 2024-09-14T06:18:11.804 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9746e-05 2024-09-14T06:18:11.804 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.804 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.804 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.804 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987832+0000", 2024-09-14T06:18:11.804 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.452e-05 2024-09-14T06:18:11.804 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.804 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.804 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.805 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064535+0000", 2024-09-14T06:18:11.805 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076703478000000005 2024-09-14T06:18:11.805 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.805 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.805 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.805 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064545+0000", 2024-09-14T06:18:11.805 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9799999999999993e-06 2024-09-14T06:18:11.805 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.805 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.805 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.805 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.805 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.805 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11492 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.805 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:18:11.805 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.773421350000007, 2024-09-14T06:18:11.805 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076661759999999995, 2024-09-14T06:18:11.806 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.806 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.806 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.806 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.806 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.806 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.806 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11492 2024-09-14T06:18:11.806 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.806 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.806 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.806 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.806 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:18:11.806 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.806 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.806 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.807 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.807 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:18:11.807 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.807 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.807 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.807 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.807 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987918+0000", 2024-09-14T06:18:11.807 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3239999999999999e-06 2024-09-14T06:18:11.807 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.807 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.807 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.807 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987924+0000", 2024-09-14T06:18:11.807 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6930000000000004e-06 2024-09-14T06:18:11.807 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.807 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.807 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.808 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987943+0000", 2024-09-14T06:18:11.808 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9487000000000002e-05 2024-09-14T06:18:11.808 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.808 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.808 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.808 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987958+0000", 2024-09-14T06:18:11.808 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4453e-05 2024-09-14T06:18:11.808 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.808 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.808 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.808 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064571+0000", 2024-09-14T06:18:11.808 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076612988000000007 2024-09-14T06:18:11.808 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.808 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.809 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.809 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064579+0000", 2024-09-14T06:18:11.809 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8150000000000006e-06 2024-09-14T06:18:11.809 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.809 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.809 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.809 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.809 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.809 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11495 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.809 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:18:11.809 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.773252661000001, 2024-09-14T06:18:11.809 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076524746000000005, 2024-09-14T06:18:11.809 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.810 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.810 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.810 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.810 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.810 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.810 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11495 2024-09-14T06:18:11.810 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.810 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.810 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.810 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.810 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:18:11.810 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.810 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.810 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.810 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.811 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:18:11.811 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.811 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.811 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.811 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.811 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988087+0000", 2024-09-14T06:18:11.811 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0499999999999999e-06 2024-09-14T06:18:11.811 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.811 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.811 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.811 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988118+0000", 2024-09-14T06:18:11.811 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1900000000000003e-05 2024-09-14T06:18:11.811 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.811 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.811 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.812 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988138+0000", 2024-09-14T06:18:11.812 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.933e-05 2024-09-14T06:18:11.812 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.812 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.812 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.812 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988152+0000", 2024-09-14T06:18:11.812 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3889e-05 2024-09-14T06:18:11.812 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.812 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.812 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.812 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064603+0000", 2024-09-14T06:18:11.812 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076451082000000004 2024-09-14T06:18:11.812 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.813 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.813 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.813 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064610+0000", 2024-09-14T06:18:11.813 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4950000000000002e-06 2024-09-14T06:18:11.813 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.813 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.813 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.813 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.813 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.813 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11498 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.813 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:18:11.813 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.773166584999998, 2024-09-14T06:18:11.813 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076471883000000004, 2024-09-14T06:18:11.814 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.814 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.814 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.814 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.814 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.814 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.814 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11498 2024-09-14T06:18:11.814 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.814 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.814 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.814 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.814 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:18:11.814 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.814 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.814 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.814 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.815 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:18:11.815 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.815 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.815 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.815 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.815 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988173+0000", 2024-09-14T06:18:11.815 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2300000000000001e-06 2024-09-14T06:18:11.815 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.815 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.815 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.815 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988178+0000", 2024-09-14T06:18:11.815 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4229999999999996e-06 2024-09-14T06:18:11.815 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.815 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.816 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.816 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988235+0000", 2024-09-14T06:18:11.816 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6616999999999997e-05 2024-09-14T06:18:11.816 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.816 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.816 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.816 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988264+0000", 2024-09-14T06:18:11.816 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9189000000000001e-05 2024-09-14T06:18:11.816 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.816 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.816 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.816 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064636+0000", 2024-09-14T06:18:11.816 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076371573999999998 2024-09-14T06:18:11.817 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.817 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.817 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.817 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064643+0000", 2024-09-14T06:18:11.817 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8499999999999994e-06 2024-09-14T06:18:11.817 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.817 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.817 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.817 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.817 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.817 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11523 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.817 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:18:11.817 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.772446436999999, 2024-09-14T06:18:11.817 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076062454000000002, 2024-09-14T06:18:11.817 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.818 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.818 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.818 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.818 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.818 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.818 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11523 2024-09-14T06:18:11.818 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.818 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.818 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.818 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.818 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:18:11.818 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.818 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.818 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.819 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.819 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:18:11.819 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.819 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.819 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.819 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.819 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988894+0000", 2024-09-14T06:18:11.819 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.843e-06 2024-09-14T06:18:11.819 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.819 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.819 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.819 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988903+0000", 2024-09-14T06:18:11.819 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0440000000000001e-06 2024-09-14T06:18:11.819 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.820 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.820 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.820 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988929+0000", 2024-09-14T06:18:11.820 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6679e-05 2024-09-14T06:18:11.820 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.820 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.820 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.820 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988967+0000", 2024-09-14T06:18:11.820 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7453000000000002e-05 2024-09-14T06:18:11.820 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.820 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.820 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.820 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064668+0000", 2024-09-14T06:18:11.820 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075701389999999993 2024-09-14T06:18:11.821 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.821 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.821 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.821 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064954+0000", 2024-09-14T06:18:11.821 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028604500000000002 2024-09-14T06:18:11.821 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.821 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.821 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.821 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.821 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.821 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11530 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.821 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:18:11.821 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.771955887000004, 2024-09-14T06:18:11.821 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075620674999999998, 2024-09-14T06:18:11.821 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.822 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.822 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.822 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.822 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.822 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.822 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11530 2024-09-14T06:18:11.822 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.822 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.822 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.822 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.822 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:18:11.822 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.822 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.822 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.823 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.823 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:18:11.823 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.823 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.823 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.823 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.823 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989384+0000", 2024-09-14T06:18:11.823 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6160000000000001e-06 2024-09-14T06:18:11.823 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.823 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.823 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.823 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989390+0000", 2024-09-14T06:18:11.823 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7039999999999999e-06 2024-09-14T06:18:11.823 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.823 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.823 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.824 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989409+0000", 2024-09-14T06:18:11.824 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.95e-05 2024-09-14T06:18:11.824 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.824 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.824 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.824 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989429+0000", 2024-09-14T06:18:11.824 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0173000000000002e-05 2024-09-14T06:18:11.824 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.824 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.824 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.824 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064992+0000", 2024-09-14T06:18:11.824 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075562764000000004 2024-09-14T06:18:11.824 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.825 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.825 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.825 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065003+0000", 2024-09-14T06:18:11.825 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0917999999999999e-05 2024-09-14T06:18:11.825 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.825 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.825 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.825 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.825 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.825 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11532 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.825 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:18:11.825 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.771885491000006, 2024-09-14T06:18:11.825 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075604656000000006, 2024-09-14T06:18:11.825 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.826 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.826 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.826 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.826 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.826 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.826 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11532 2024-09-14T06:18:11.826 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.826 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.826 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.826 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.826 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:18:11.826 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.826 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.826 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.826 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.827 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:18:11.827 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.827 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.827 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.827 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.827 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989454+0000", 2024-09-14T06:18:11.827 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6500000000000001e-06 2024-09-14T06:18:11.827 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.827 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.827 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.827 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989460+0000", 2024-09-14T06:18:11.827 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7459999999999997e-06 2024-09-14T06:18:11.827 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.828 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.828 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.828 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989480+0000", 2024-09-14T06:18:11.828 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.948e-05 2024-09-14T06:18:11.828 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.828 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.828 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.828 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989497+0000", 2024-09-14T06:18:11.828 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7153e-05 2024-09-14T06:18:11.828 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.828 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.828 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.828 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065046+0000", 2024-09-14T06:18:11.828 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075548905999999999 2024-09-14T06:18:11.828 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.829 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.829 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.829 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065057+0000", 2024-09-14T06:18:11.829 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1721e-05 2024-09-14T06:18:11.829 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.829 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.829 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.829 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.829 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.829 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11534 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.829 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:18:11.829 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.771819004999998, 2024-09-14T06:18:11.829 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075574350999999998, 2024-09-14T06:18:11.829 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.829 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.830 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.830 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.830 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.830 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.830 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11534 2024-09-14T06:18:11.830 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.830 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.830 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.830 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.830 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:18:11.830 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.830 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.830 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.830 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.831 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:18:11.831 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.831 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.831 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.831 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.831 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989521+0000", 2024-09-14T06:18:11.831 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.533e-06 2024-09-14T06:18:11.831 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.831 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.831 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.831 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989527+0000", 2024-09-14T06:18:11.831 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0700000000000003e-06 2024-09-14T06:18:11.831 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.831 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.832 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.832 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989546+0000", 2024-09-14T06:18:11.832 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.948e-05 2024-09-14T06:18:11.832 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.832 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.832 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.832 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989561+0000", 2024-09-14T06:18:11.832 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4827000000000001e-05 2024-09-14T06:18:11.832 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.832 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.832 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.832 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065086+0000", 2024-09-14T06:18:11.832 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075524542 2024-09-14T06:18:11.833 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.833 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.833 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.833 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065094+0000", 2024-09-14T06:18:11.833 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8990000000000001e-06 2024-09-14T06:18:11.833 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.833 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.833 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.833 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.833 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.833 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11535 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.833 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:18:11.833 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.771751641999998, 2024-09-14T06:18:11.833 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075542466000000003, 2024-09-14T06:18:11.834 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.834 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.834 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.834 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.834 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.834 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.834 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11535 2024-09-14T06:18:11.834 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.834 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.834 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.834 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.834 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:18:11.834 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.835 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.835 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.835 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.835 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:18:11.835 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.835 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.835 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.835 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.835 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989588+0000", 2024-09-14T06:18:11.835 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1659999999999999e-06 2024-09-14T06:18:11.835 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.835 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.835 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.835 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989593+0000", 2024-09-14T06:18:11.836 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6230000000000002e-06 2024-09-14T06:18:11.836 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.836 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.836 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.836 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989613+0000", 2024-09-14T06:18:11.836 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9604000000000001e-05 2024-09-14T06:18:11.836 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.836 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.836 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.836 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989643+0000", 2024-09-14T06:18:11.836 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0450000000000002e-05 2024-09-14T06:18:11.836 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.836 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.837 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.837 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065119+0000", 2024-09-14T06:18:11.837 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075475930999999996 2024-09-14T06:18:11.837 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.837 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.837 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.837 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065129+0000", 2024-09-14T06:18:11.837 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6919999999999997e-06 2024-09-14T06:18:11.837 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.837 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.837 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.837 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.838 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.838 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11537 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.838 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:18:11.838 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.771673719000006, 2024-09-14T06:18:11.838 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075497868999999995, 2024-09-14T06:18:11.838 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.838 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.838 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.838 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.838 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.838 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.838 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11537 2024-09-14T06:18:11.838 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.839 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.839 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.839 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.839 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:18:11.839 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.839 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.839 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.839 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.839 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:18:11.839 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.839 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.839 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.839 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.840 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989666+0000", 2024-09-14T06:18:11.840 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.48e-06 2024-09-14T06:18:11.840 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.840 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.840 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.840 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989672+0000", 2024-09-14T06:18:11.840 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7960000000000001e-06 2024-09-14T06:18:11.840 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.840 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.840 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.840 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989690+0000", 2024-09-14T06:18:11.840 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8743e-05 2024-09-14T06:18:11.840 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.840 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.841 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.841 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989704+0000", 2024-09-14T06:18:11.841 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3913e-05 2024-09-14T06:18:11.841 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.841 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.841 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.841 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065155+0000", 2024-09-14T06:18:11.841 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075450351999999998 2024-09-14T06:18:11.841 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.841 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.841 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.841 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065162+0000", 2024-09-14T06:18:11.841 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5850000000000002e-06 2024-09-14T06:18:11.842 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.842 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.842 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.842 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.842 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.842 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11543 3.12 3.4d02a8b2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:11.842 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:18:11.842 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 95.771396155000005, 2024-09-14T06:18:11.842 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074335152000000002, 2024-09-14T06:18:11.842 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:11.842 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:11.842 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:11.842 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:11.842 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:11.843 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:11.843 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11543 2024-09-14T06:18:11.843 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.843 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:11.843 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.843 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:11.843 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:18:11.843 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.843 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.843 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.843 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:11.843 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:18:11.843 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:11.843 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.843 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.844 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:11.844 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989944+0000", 2024-09-14T06:18:11.844 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.694e-06 2024-09-14T06:18:11.844 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.844 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.844 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:11.844 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989950+0000", 2024-09-14T06:18:11.844 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3389999999999998e-06 2024-09-14T06:18:11.844 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.844 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.844 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:11.844 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989969+0000", 2024-09-14T06:18:11.844 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8963000000000001e-05 2024-09-14T06:18:11.845 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.845 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.845 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:11.845 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989985+0000", 2024-09-14T06:18:11.845 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5586999999999998e-05 2024-09-14T06:18:11.845 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.845 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.845 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:11.845 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064270+0000", 2024-09-14T06:18:11.845 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074285552000000005 2024-09-14T06:18:11.845 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:11.845 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:11.845 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:11.845 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064277+0000", 2024-09-14T06:18:11.846 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0169999999999999e-06 2024-09-14T06:18:11.846 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.846 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.846 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.846 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:11.846 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:11.846 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:11.847 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:18:11.847 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:18:11.847 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:11.845+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:11.848 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:18:11.855 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:18:11.855 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:18:11.872 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:18:11.872 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:18:11.888 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:18:11.888 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:18:11.908 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:18:11.908 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:18:11.928 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:18:11.928 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:18:11.947 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:18:11.948 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:18:11.948 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:11.945+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:11.965 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:18:11.965 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:18:11.969 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:18:11.970 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:18:11.988 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-14T06:18:11.213479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":271580496,"num_objects":653,"num_object_clones":77,"num_object_copies":1306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":653,"num_whiteouts":258,"num_read":105202,"num_read_kb":749393,"num_write":16972,"num_write_kb":430797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1206,"num_bytes_recovered":144366785,"num_keys_recovered":219,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14864,"ondisk_log_size":14864,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":385540,"kb_used_data":31808,"kb_used_omap":12,"kb_used_meta":353395,"kb_avail":754589180,"statfs":{"total":773094113280,"available":772699320320,"internally_reserved":0,"allocated":32571392,"data_stored":219803681,"data_compressed":6478403,"data_compressed_allocated":15679488,"data_compressed_original":209660209,"omap_allocated":12752,"internal_metadata":361877040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-92578896,"num_objects":-68,"num_object_clones":-15,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":-6,"num_read":894,"num_read_kb":770,"num_write":149,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.796751"},"pg_stats":[{"pgid":"3.1f","version":"112'579","reported_seq":3591,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.415819+0000","last_change":"2024-09-14T06:18:06.415819+0000","last_active":"2024-09-14T06:18:06.415819+0000","last_peered":"2024-09-14T06:18:06.415819+0000","last_clean":"2024-09-14T06:18:06.415819+0000","last_became_active":"2024-09-14T06:17:02.193313+0000","last_became_peered":"2024-09-14T06:17:02.193313+0000","last_unstale":"2024-09-14T06:18:06.415819+0000","last_undegraded":"2024-09-14T06:18:06.415819+0000","last_fullsized":"2024-09-14T06:18:06.415819+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:42.502193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001774461,"stat_sum":{"num_bytes":6904730,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":4583,"num_read_kb":53580,"num_write":691,"num_write_kb":19350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.1e","version":"112'387","reported_seq":4431,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.586063+0000","last_change":"2024-09-14T06:18:06.414503+0000","last_active":"2024-09-14T06:18:06.586063+0000","last_peered":"2024-09-14T06:18:06.586063+0000","last_clean":"2024-09-14T06:18:06.586063+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:18:06.586063+0000","last_undegraded":"2024-09-14T06:18:06.586063+0000","last_fullsized":"2024-09-14T06:18:06.586063+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:03:55.670662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001177558,"stat_sum":{"num_bytes":9814314,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":11,"num_read":4997,"num_read_kb":43044,"num_write":334,"num_write_kb":15481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.1d","version":"111'1115","reported_seq":2450,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.428791+0000","last_change":"2024-09-14T06:18:07.429809+0000","last_active":"2024-09-14T06:18:07.428791+0000","last_peered":"2024-09-14T06:18:07.428791+0000","last_clean":"2024-09-14T06:18:07.428791+0000","last_became_active":"2024-09-14T06:17:38.226838+0000","last_became_peered":"2024-09-14T06:17:38.226838+0000","last_unstale":"2024-09-14T06:18:07.428791+0000","last_undegraded":"2024-09-14T06:18:07.428791+0000","last_fullsized":"2024-09-14T06:18:07.428791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:52:42.905891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089295869999999992,"stat_sum":{"num_bytes":10470999,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":5125,"num_read_kb":14378,"num_write":1566,"num_write_kb":13994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10785266,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.1c","version":"112'894","reported_seq":2853,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.556482+0000","last_change":"2024-09-14T06:18:06.394725+0000","last_active":"2024-09-14T06:18:06.556482+0000","last_peered":"2024-09-14T06:18:06.556482+0000","last_clean":"2024-09-14T06:18:06.556482+0000","last_became_active":"2024-09-14T06:17:48.737020+0000","last_became_peered":"2024-09-14T06:17:48.737020+0000","last_unstale":"2024-09-14T06:18:06.556482+0000","last_undegraded":"2024-09-14T06:18:06.556482+0000","last_fullsized":"2024-09-14T06:18:06.556482+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:30.667772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023015549999999998,"stat_sum":{"num_bytes":524288,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":5685,"num_read_kb":9726,"num_write":1303,"num_write_kb":6518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":3869232,"num_keys_recovered":68,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.1b","version":"112'332","reported_seq":3720,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.393914+0000","last_change":"2024-09-14T06:18:06.393914+0000","last_active":"2024-09-14T06:18:06.393914+0000","last_peered":"2024-09-14T06:18:06.393914+0000","last_clean":"2024-09-14T06:18:06.393914+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:18:06.393914+0000","last_undegraded":"2024-09-14T06:18:06.393914+0000","last_fullsized":"2024-09-14T06:18:06.393914+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:41:12.786161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016475140000000001,"stat_sum":{"num_bytes":7304576,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":3342,"num_read_kb":39056,"num_write":353,"num_write_kb":25723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.1a","version":"113'526","reported_seq":3227,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.558803+0000","last_change":"2024-09-14T06:18:07.431193+0000","last_active":"2024-09-14T06:18:07.558803+0000","last_peered":"2024-09-14T06:18:07.558803+0000","last_clean":"2024-09-14T06:18:07.558803+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:18:07.558803+0000","last_undegraded":"2024-09-14T06:18:07.558803+0000","last_fullsized":"2024-09-14T06:18:07.558803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:16:04.128598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032412640000000002,"stat_sum":{"num_bytes":5121647,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":17,"num_read":4031,"num_read_kb":13071,"num_write":651,"num_write_kb":7511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.19","version":"112'478","reported_seq":1963,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.512432+0000","last_change":"2024-09-14T06:18:06.393452+0000","last_active":"2024-09-14T06:18:06.512432+0000","last_peered":"2024-09-14T06:18:06.512432+0000","last_clean":"2024-09-14T06:18:06.512432+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:18:06.512432+0000","last_undegraded":"2024-09-14T06:18:06.512432+0000","last_fullsized":"2024-09-14T06:18:06.512432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:18:54.562166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012335219999999999,"stat_sum":{"num_bytes":6022104,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":11,"num_read":2684,"num_read_kb":19064,"num_write":663,"num_write_kb":12862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.18","version":"111'368","reported_seq":2900,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.561319+0000","last_change":"2024-09-14T06:18:06.415172+0000","last_active":"2024-09-14T06:18:06.561319+0000","last_peered":"2024-09-14T06:18:06.561319+0000","last_clean":"2024-09-14T06:18:06.561319+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:18:06.561319+0000","last_undegraded":"2024-09-14T06:18:06.561319+0000","last_fullsized":"2024-09-14T06:18:06.561319+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:42:04.319290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017036670000000001,"stat_sum":{"num_bytes":10388823,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":3627,"num_read_kb":37626,"num_write":336,"num_write_kb":32935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.17","version":"109'367","reported_seq":1663,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.414916+0000","last_change":"2024-09-14T06:18:06.415489+0000","last_active":"2024-09-14T06:18:06.414916+0000","last_peered":"2024-09-14T06:18:06.414916+0000","last_clean":"2024-09-14T06:18:06.414916+0000","last_became_active":"2024-09-14T06:17:37.233410+0000","last_became_peered":"2024-09-14T06:17:37.233410+0000","last_unstale":"2024-09-14T06:18:06.414916+0000","last_undegraded":"2024-09-14T06:18:06.414916+0000","last_fullsized":"2024-09-14T06:18:06.414916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:23:31.419313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019166090000000001,"stat_sum":{"num_bytes":13230263,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":1385,"num_read_kb":17453,"num_write":316,"num_write_kb":17188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":23467377,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.16","version":"109'418","reported_seq":2324,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.429660+0000","last_change":"2024-09-14T06:18:07.430245+0000","last_active":"2024-09-14T06:18:07.429660+0000","last_peered":"2024-09-14T06:18:07.429660+0000","last_clean":"2024-09-14T06:18:07.429660+0000","last_became_active":"2024-09-14T06:17:37.235311+0000","last_became_peered":"2024-09-14T06:17:37.235311+0000","last_unstale":"2024-09-14T06:18:07.429660+0000","last_undegraded":"2024-09-14T06:18:07.429660+0000","last_fullsized":"2024-09-14T06:18:07.429660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:40:44.096601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086075779999999994,"stat_sum":{"num_bytes":5327508,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":2483,"num_read_kb":16488,"num_write":502,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":6785399,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.15","version":"113'397","reported_seq":3650,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.576546+0000","last_change":"2024-09-14T06:18:07.427988+0000","last_active":"2024-09-14T06:18:07.576546+0000","last_peered":"2024-09-14T06:18:07.576546+0000","last_clean":"2024-09-14T06:18:07.576546+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:18:07.576546+0000","last_undegraded":"2024-09-14T06:18:07.576546+0000","last_fullsized":"2024-09-14T06:18:07.576546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:15:45.199242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020002610000000001,"stat_sum":{"num_bytes":12093895,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":4034,"num_read_kb":64540,"num_write":487,"num_write_kb":22218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.14","version":"112'294","reported_seq":2003,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.428107+0000","last_change":"2024-09-14T06:18:07.428692+0000","last_active":"2024-09-14T06:18:07.428107+0000","last_peered":"2024-09-14T06:18:07.428107+0000","last_clean":"2024-09-14T06:18:07.428107+0000","last_became_active":"2024-09-14T06:17:37.231244+0000","last_became_peered":"2024-09-14T06:17:37.231244+0000","last_unstale":"2024-09-14T06:18:07.428107+0000","last_undegraded":"2024-09-14T06:18:07.428107+0000","last_fullsized":"2024-09-14T06:18:07.428107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:30:56.042466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082021679999999993,"stat_sum":{"num_bytes":11530735,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":2907,"num_read_kb":27191,"num_write":261,"num_write_kb":15734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":32506492,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.420805+0000","last_change":"2024-09-14T06:17:34.200709+0000","last_active":"2024-09-14T06:18:07.420805+0000","last_peered":"2024-09-14T06:18:07.420805+0000","last_clean":"2024-09-14T06:18:07.420805+0000","last_became_active":"2024-09-14T06:17:34.200238+0000","last_became_peered":"2024-09-14T06:17:34.200238+0000","last_unstale":"2024-09-14T06:18:07.420805+0000","last_undegraded":"2024-09-14T06:18:07.420805+0000","last_fullsized":"2024-09-14T06:18:07.420805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:34.864695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"112'452","reported_seq":1238,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.534352+0000","last_change":"2024-09-14T06:18:06.394712+0000","last_active":"2024-09-14T06:18:06.534352+0000","last_peered":"2024-09-14T06:18:06.534352+0000","last_clean":"2024-09-14T06:18:06.534352+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:18:06.534352+0000","last_undegraded":"2024-09-14T06:18:06.534352+0000","last_fullsized":"2024-09-14T06:18:06.534352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:24.513821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002054574,"stat_sum":{"num_bytes":1998996,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":1519,"num_read_kb":9696,"num_write":471,"num_write_kb":9164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"2.6","version":"0'0","reported_seq":200,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.392849+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:18:06.392849+0000","last_peered":"2024-09-14T06:18:06.392849+0000","last_clean":"2024-09-14T06:18:06.392849+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:18:06.392849+0000","last_undegraded":"2024-09-14T06:18:06.392849+0000","last_fullsized":"2024-09-14T06:18:06.392849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"112'360","reported_seq":3198,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.394070+0000","last_change":"2024-09-14T06:18:06.394070+0000","last_active":"2024-09-14T06:18:06.394070+0000","last_peered":"2024-09-14T06:18:06.394070+0000","last_clean":"2024-09-14T06:18:06.394070+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:18:06.394070+0000","last_undegraded":"2024-09-14T06:18:06.394070+0000","last_fullsized":"2024-09-14T06:18:06.394070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:39.034290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016880980000000001,"stat_sum":{"num_bytes":17349459,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":3293,"num_read_kb":28583,"num_write":324,"num_write_kb":15237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":73,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.421358+0000","last_change":"2024-09-14T06:17:34.196668+0000","last_active":"2024-09-14T06:18:07.421358+0000","last_peered":"2024-09-14T06:18:07.421358+0000","last_clean":"2024-09-14T06:18:07.421358+0000","last_became_active":"2024-09-14T06:17:34.196072+0000","last_became_peered":"2024-09-14T06:17:34.196072+0000","last_unstale":"2024-09-14T06:18:07.421358+0000","last_undegraded":"2024-09-14T06:18:07.421358+0000","last_fullsized":"2024-09-14T06:18:07.421358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:18:45.616834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"112'348","reported_seq":1919,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.515246+0000","last_change":"2024-09-14T06:18:06.485171+0000","last_active":"2024-09-14T06:18:06.515246+0000","last_peered":"2024-09-14T06:18:06.515246+0000","last_clean":"2024-09-14T06:18:06.515246+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:18:06.515246+0000","last_undegraded":"2024-09-14T06:18:06.515246+0000","last_fullsized":"2024-09-14T06:18:06.515246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:53:38.747989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058769599999999996,"stat_sum":{"num_bytes":23108758,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":1780,"num_read_kb":12934,"num_write":296,"num_write_kb":11582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":200,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.413014+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:18:06.413014+0000","last_peered":"2024-09-14T06:18:06.413014+0000","last_clean":"2024-09-14T06:18:06.413014+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:18:06.413014+0000","last_undegraded":"2024-09-14T06:18:06.413014+0000","last_fullsized":"2024-09-14T06:18:06.413014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"111'263","reported_seq":1657,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.485266+0000","last_change":"2024-09-14T06:18:06.485266+0000","last_active":"2024-09-14T06:18:06.485266+0000","last_peered":"2024-09-14T06:18:06.485266+0000","last_clean":"2024-09-14T06:18:06.485266+0000","last_became_active":"2024-09-14T06:16:33.547925+0000","last_became_peered":"2024-09-14T06:16:33.547925+0000","last_unstale":"2024-09-14T06:18:06.485266+0000","last_undegraded":"2024-09-14T06:18:06.485266+0000","last_fullsized":"2024-09-14T06:18:06.485266+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:44:27.666578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066389200000000002,"stat_sum":{"num_bytes":5118396,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":11,"num_read":1354,"num_read_kb":21258,"num_write":257,"num_write_kb":9954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.3","version":"113'353","reported_seq":1085,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.551377+0000","last_change":"2024-09-14T06:18:07.429214+0000","last_active":"2024-09-14T06:18:07.551377+0000","last_peered":"2024-09-14T06:18:07.551377+0000","last_clean":"2024-09-14T06:18:07.551377+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:18:07.551377+0000","last_undegraded":"2024-09-14T06:18:07.551377+0000","last_fullsized":"2024-09-14T06:18:07.551377+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:39:31.110478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028864540000000001,"stat_sum":{"num_bytes":5199972,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":8,"num_read":499,"num_read_kb":4207,"num_write":278,"num_write_kb":7109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"2.2","version":"22'2","reported_seq":77,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.420164+0000","last_change":"2024-09-14T06:17:34.203977+0000","last_active":"2024-09-14T06:18:07.420164+0000","last_peered":"2024-09-14T06:18:07.420164+0000","last_clean":"2024-09-14T06:18:07.420164+0000","last_became_active":"2024-09-14T06:17:34.199244+0000","last_became_peered":"2024-09-14T06:17:34.199244+0000","last_unstale":"2024-09-14T06:18:07.420164+0000","last_undegraded":"2024-09-14T06:18:07.420164+0000","last_fullsized":"2024-09-14T06:18:07.420164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:21.934094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"112'394","reported_seq":1798,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.480245+0000","last_change":"2024-09-14T06:18:06.414776+0000","last_active":"2024-09-14T06:18:06.480245+0000","last_peered":"2024-09-14T06:18:06.480245+0000","last_clean":"2024-09-14T06:18:06.480245+0000","last_became_active":"2024-09-14T06:16:20.196215+0000","last_became_peered":"2024-09-14T06:16:20.196215+0000","last_unstale":"2024-09-14T06:18:06.480245+0000","last_undegraded":"2024-09-14T06:18:06.480245+0000","last_fullsized":"2024-09-14T06:18:06.480245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:53.216371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013626510000000001,"stat_sum":{"num_bytes":11634292,"num_objects":20,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":2093,"num_read_kb":18967,"num_write":510,"num_write_kb":11051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":200,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.480746+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:18:06.480746+0000","last_peered":"2024-09-14T06:18:06.480746+0000","last_clean":"2024-09-14T06:18:06.480746+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:18:06.480746+0000","last_undegraded":"2024-09-14T06:18:06.480746+0000","last_fullsized":"2024-09-14T06:18:06.480746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"112'534","reported_seq":2138,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.484555+0000","last_change":"2024-09-14T06:18:06.484555+0000","last_active":"2024-09-14T06:18:06.484555+0000","last_peered":"2024-09-14T06:18:06.484555+0000","last_clean":"2024-09-14T06:18:06.484555+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:18:06.484555+0000","last_undegraded":"2024-09-14T06:18:06.484555+0000","last_fullsized":"2024-09-14T06:18:06.484555+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:24:06.789564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066675299999999996,"stat_sum":{"num_bytes":3869323,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":6,"num_read":2885,"num_read_kb":8812,"num_write":717,"num_write_kb":8784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"2.0","version":"0'0","reported_seq":202,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.419866+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:18:07.419866+0000","last_peered":"2024-09-14T06:18:07.419866+0000","last_clean":"2024-09-14T06:18:07.419866+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:18:07.419866+0000","last_undegraded":"2024-09-14T06:18:07.419866+0000","last_fullsized":"2024-09-14T06:18:07.419866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"111'448","reported_seq":1951,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.395523+0000","last_change":"2024-09-14T06:18:06.395523+0000","last_active":"2024-09-14T06:18:06.395523+0000","last_peered":"2024-09-14T06:18:06.395523+0000","last_clean":"2024-09-14T06:18:06.395523+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:18:06.395523+0000","last_undegraded":"2024-09-14T06:18:06.395523+0000","last_fullsized":"2024-09-14T06:18:06.395523+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:44:51.198049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023997739999999999,"stat_sum":{"num_bytes":2415784,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":2196,"num_read_kb":22997,"num_write":357,"num_write_kb":14523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"2.3","version":"20'1","reported_seq":223,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.420431+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:18:07.420431+0000","last_peered":"2024-09-14T06:18:07.420431+0000","last_clean":"2024-09-14T06:18:07.420431+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:18:07.420431+0000","last_undegraded":"2024-09-14T06:18:07.420431+0000","last_fullsized":"2024-09-14T06:18:07.420431+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":294,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.427920+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:18:07.427920+0000","last_peered":"2024-09-14T06:18:07.427920+0000","last_clean":"2024-09-14T06:18:07.427920+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:18:07.427920+0000","last_undegraded":"2024-09-14T06:18:07.427920+0000","last_fullsized":"2024-09-14T06:18:07.427920+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"111'426","reported_seq":1594,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.392674+0000","last_change":"2024-09-14T06:18:06.393352+0000","last_active":"2024-09-14T06:18:06.392674+0000","last_peered":"2024-09-14T06:18:06.392674+0000","last_clean":"2024-09-14T06:18:06.392674+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:18:06.392674+0000","last_undegraded":"2024-09-14T06:18:06.392674+0000","last_fullsized":"2024-09-14T06:18:06.392674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:26.008208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012336490000000001,"stat_sum":{"num_bytes":8259504,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":2130,"num_read_kb":6714,"num_write":581,"num_write_kb":6849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.9","version":"111'385","reported_seq":1801,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.427709+0000","last_change":"2024-09-14T06:18:07.428373+0000","last_active":"2024-09-14T06:18:07.427709+0000","last_peered":"2024-09-14T06:18:07.427709+0000","last_clean":"2024-09-14T06:18:07.427709+0000","last_became_active":"2024-09-14T06:17:38.224238+0000","last_became_peered":"2024-09-14T06:17:38.224238+0000","last_unstale":"2024-09-14T06:18:07.427709+0000","last_undegraded":"2024-09-14T06:18:07.427709+0000","last_fullsized":"2024-09-14T06:18:07.427709+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:47:02.196929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0078946569999999994,"stat_sum":{"num_bytes":8186252,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":2223,"num_read_kb":13651,"num_write":538,"num_write_kb":14353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8477378,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.a","version":"113'345","reported_seq":2233,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.528087+0000","last_change":"2024-09-14T06:18:07.427556+0000","last_active":"2024-09-14T06:18:07.528087+0000","last_peered":"2024-09-14T06:18:07.528087+0000","last_clean":"2024-09-14T06:18:07.528087+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:18:07.528087+0000","last_undegraded":"2024-09-14T06:18:07.528087+0000","last_fullsized":"2024-09-14T06:18:07.528087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:14:20.340187+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.007139505,"stat_sum":{"num_bytes":13141585,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":2211,"num_read_kb":16098,"num_write":288,"num_write_kb":13801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.b","version":"112'305","reported_seq":2823,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.481642+0000","last_change":"2024-09-14T06:18:06.481642+0000","last_active":"2024-09-14T06:18:06.481642+0000","last_peered":"2024-09-14T06:18:06.481642+0000","last_clean":"2024-09-14T06:18:06.481642+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:18:06.481642+0000","last_undegraded":"2024-09-14T06:18:06.481642+0000","last_fullsized":"2024-09-14T06:18:06.481642+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:10:09.819416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073018699999999998,"stat_sum":{"num_bytes":6658155,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":9,"num_read":2867,"num_read_kb":16792,"num_write":263,"num_write_kb":9901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.c","version":"109'181","reported_seq":1600,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.426266+0000","last_change":"2024-09-14T06:18:07.426577+0000","last_active":"2024-09-14T06:18:07.426266+0000","last_peered":"2024-09-14T06:18:07.426266+0000","last_clean":"2024-09-14T06:18:07.426266+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:18:07.426266+0000","last_undegraded":"2024-09-14T06:18:07.426266+0000","last_fullsized":"2024-09-14T06:18:07.426266+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:50:03.117816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069750400000000005,"stat_sum":{"num_bytes":5966452,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":1287,"num_read_kb":29081,"num_write":240,"num_write_kb":11577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.d","version":"110'662","reported_seq":2521,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.428847+0000","last_change":"2024-09-14T06:18:07.429566+0000","last_active":"2024-09-14T06:18:07.428847+0000","last_peered":"2024-09-14T06:18:07.428847+0000","last_clean":"2024-09-14T06:18:07.428847+0000","last_became_active":"2024-09-14T06:17:38.227106+0000","last_became_peered":"2024-09-14T06:17:38.227106+0000","last_unstale":"2024-09-14T06:18:07.428847+0000","last_undegraded":"2024-09-14T06:18:07.428847+0000","last_fullsized":"2024-09-14T06:18:07.428847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:35:22.181122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082906309999999997,"stat_sum":{"num_bytes":14571487,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":3737,"num_read_kb":26925,"num_write":972,"num_write_kb":20272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":29390214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.e","version":"112'730","reported_seq":5234,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.426348+0000","last_change":"2024-09-14T06:18:07.426821+0000","last_active":"2024-09-14T06:18:07.426348+0000","last_peered":"2024-09-14T06:18:07.426348+0000","last_clean":"2024-09-14T06:18:07.426348+0000","last_became_active":"2024-09-14T06:16:33.546796+0000","last_became_peered":"2024-09-14T06:16:33.546796+0000","last_unstale":"2024-09-14T06:18:07.426348+0000","last_undegraded":"2024-09-14T06:18:07.426348+0000","last_fullsized":"2024-09-14T06:18:07.426348+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:50:06.834775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085063399999999996,"stat_sum":{"num_bytes":6677733,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":9974,"num_read_kb":51459,"num_write":672,"num_write_kb":16128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.f","version":"113'638","reported_seq":2876,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.427073+0000","last_change":"2024-09-14T06:18:07.427073+0000","last_active":"2024-09-14T06:18:07.427073+0000","last_peered":"2024-09-14T06:18:07.427073+0000","last_clean":"2024-09-14T06:18:07.427073+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:18:07.427073+0000","last_undegraded":"2024-09-14T06:18:07.427073+0000","last_fullsized":"2024-09-14T06:18:07.427073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:52:10.795745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001110277,"stat_sum":{"num_bytes":20745733,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":4425,"num_read_kb":16166,"num_write":753,"num_write_kb":12749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.10","version":"112'346","reported_seq":2500,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.428504+0000","last_change":"2024-09-14T06:18:07.429118+0000","last_active":"2024-09-14T06:18:07.428504+0000","last_peered":"2024-09-14T06:18:07.428504+0000","last_clean":"2024-09-14T06:18:07.428504+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:18:07.428504+0000","last_undegraded":"2024-09-14T06:18:07.428504+0000","last_fullsized":"2024-09-14T06:18:07.428504+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:33:37.310744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001526257,"stat_sum":{"num_bytes":2960020,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":2476,"num_read_kb":47684,"num_write":331,"num_write_kb":14643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.11","version":"113'504","reported_seq":2758,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.427957+0000","last_change":"2024-09-14T06:18:07.427957+0000","last_active":"2024-09-14T06:18:07.427957+0000","last_peered":"2024-09-14T06:18:07.427957+0000","last_clean":"2024-09-14T06:18:07.427957+0000","last_became_active":"2024-09-14T06:17:38.224251+0000","last_became_peered":"2024-09-14T06:17:38.224251+0000","last_unstale":"2024-09-14T06:18:07.427957+0000","last_undegraded":"2024-09-14T06:18:07.427957+0000","last_fullsized":"2024-09-14T06:18:07.427957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:24:25.122706+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0079625440000000002,"stat_sum":{"num_bytes":5301360,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":4155,"num_read_kb":14398,"num_write":570,"num_write_kb":7659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":12239008,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.12","version":"111'490","reported_seq":2955,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-14T06:18:06.484247+0000","last_change":"2024-09-14T06:18:06.484746+0000","last_active":"2024-09-14T06:18:06.484247+0000","last_peered":"2024-09-14T06:18:06.484247+0000","last_clean":"2024-09-14T06:18:06.484247+0000","last_became_active":"2024-09-14T06:17:37.232145+0000","last_became_peered":"2024-09-14T06:17:37.232145+0000","last_unstale":"2024-09-14T06:18:06.484247+0000","last_undegraded":"2024-09-14T06:18:06.484247+0000","last_fullsized":"2024-09-14T06:18:06.484247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:33:45.201867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082120399999999997,"stat_sum":{"num_bytes":4194340,"num_objects":26,"num_object_clones":2,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":3314,"num_read_kb":11024,"num_write":581,"num_write_kb":8452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":10176554,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"9"}]},{"pgid":"3.13","version":"112'470","reported_seq":4015,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.421279+0000","last_change":"2024-09-14T06:18:07.422009+0000","last_active":"2024-09-14T06:18:07.421279+0000","last_peered":"2024-09-14T06:18:07.421279+0000","last_clean":"2024-09-14T06:18:07.421279+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:18:07.421279+0000","last_undegraded":"2024-09-14T06:18:07.421279+0000","last_fullsized":"2024-09-14T06:18:07.421279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:02.011457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606756,"stat_sum":{"num_bytes":4898626,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":12,"num_read":5835,"num_read_kb":16675,"num_write":407,"num_write_kb":5582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":270990109,"num_objects":649,"num_object_clones":77,"num_object_copies":1298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":649,"num_whiteouts":258,"num_read":105136,"num_read_kb":749338,"num_write":16869,"num_write_kb":429167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1203,"num_bytes_recovered":144366766,"num_keys_recovered":219,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19578880,"data_stored":186162857,"data_compressed":5514866,"data_compressed_allocated":14131200,"data_compressed_original":181697201,"omap_allocated":0,"internal_metadata":0},"log_size":14789,"ondisk_log_size":14789,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40780,"kb_used_data":5428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331060,"statfs":{"total":96636764160,"available":96595005440,"internally_reserved":0,"allocated":5558272,"data_stored":44684378,"data_compressed":1527427,"data_compressed_allocated":3481600,"data_compressed_original":43470848,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34936,"kb_used_data":3748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336904,"statfs":{"total":96636764160,"available":96600989696,"internally_reserved":0,"allocated":3837952,"data_stored":23271982,"data_compressed":499786,"data_compressed_allocated":1654784,"data_compressed_original":21961659,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75068,"kb_used_data":3480,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296772,"statfs":{"total":96636764160,"available":96559894528,"internally_reserved":0,"allocated":3563520,"data_stored":23246212,"data_compressed":708650,"data_compressed_allocated":1744896,"data_compressed_original":22229480,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57632,"kb_used_data":4232,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314208,"statfs":{"total":96636764160,"available":96577748992,"internally_reserved":0,"allocated":4333568,"data_stored":29792745,"data_compressed":748911,"data_compressed_allocated":2088960,"data_compressed_original":28389324,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37656,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334184,"statfs":{"total":96636764160,"available":96598204416,"internally_reserved":0,"allocated":2363392,"data_stored":9570825,"data_compressed":227688,"data_compressed_allocated":602112,"data_compressed_original":8588073,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44032,"kb_used_data":4524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327808,"statfs":{"total":96636764160,"available":96591675392,"internally_reserved":0,"allocated":4632576,"data_stored":31815492,"data_compressed":949994,"data_compressed_allocated":2203648,"data_compressed_original":30273750,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48768,"kb_used_data":5088,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94323072,"statfs":{"total":96636764160,"available":96586825728,"internally_reserved":0,"allocated":5210112,"data_stored":38980912,"data_compressed":1390843,"data_compressed_allocated":2809856,"data_compressed_original":37463211,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46668,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325172,"statfs":{"total":96636764160,"available":96588976128,"internally_reserved":0,"allocated":3072000,"data_stored":18441135,"data_compressed":425104,"data_compressed_allocated":1093632,"data_compressed_original":17283864,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2609152,"data_stored":23339657,"data_compressed":563815,"data_compressed_allocated":1765376,"data_compressed_original":22622156,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2002944,"data_stored":18870213,"data_compressed":561112,"data_compressed_allocated":1552384,"data_compressed_original":18510312,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2396160,"data_stored":19500133,"data_compressed":447404,"data_compressed_allocated":1548288,"data_compressed_original":18815931,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3682304,"data_stored":33940530,"data_compressed":1105265,"data_compressed_allocated":3006464,"data_compressed_original":33411072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1744896,"data_stored":17815086,"data_compressed":425104,"data_compressed_allocated":1093632,"data_compressed_original":17283864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3297280,"data_stored":34953226,"data_compressed":1332009,"data_compressed_allocated":2576384,"data_compressed_original":34350635,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2994176,"data_stored":29936032,"data_compressed":873927,"data_compressed_allocated":2068480,"data_compressed_original":29163734,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":851968,"data_stored":7807980,"data_compressed":206230,"data_compressed_allocated":520192,"data_compressed_original":7539497,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:18:11.990 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T06:18:11.990 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:18:11.990 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T06:18:11.991 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T06:18:12.019 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:12.019 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:12.033 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:18:12.075 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:12.073+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:12.150 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:12.147+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:12.157 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:12.157 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:12.171 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:18:12.256 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:12.255+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:12.331 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:18:12.331 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:12.331 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 117, 2024-09-14T06:18:12.331 INFO:teuthology.orchestra.run.smithi039.stdout: "fsid": "038563b1-268f-461f-b6a6-4b4b41ad0236", 2024-09-14T06:18:12.331 INFO:teuthology.orchestra.run.smithi039.stdout: "created": "2024-09-14T06:15:49.493035+0000", 2024-09-14T06:18:12.331 INFO:teuthology.orchestra.run.smithi039.stdout: "modified": "2024-09-14T06:18:11.199142+0000", 2024-09-14T06:18:12.331 INFO:teuthology.orchestra.run.smithi039.stdout: "last_up_change": "2024-09-14T06:16:02.014989+0000", 2024-09-14T06:18:12.331 INFO:teuthology.orchestra.run.smithi039.stdout: "last_in_change": "2024-09-14T06:16:26.221200+0000", 2024-09-14T06:18:12.331 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T06:18:12.331 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_num": 5806080, 2024-09-14T06:18:12.331 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_set": [ 2024-09-14T06:18:12.331 INFO:teuthology.orchestra.run.smithi039.stdout: "nodeep-scrub", 2024-09-14T06:18:12.332 INFO:teuthology.orchestra.run.smithi039.stdout: "noscrub", 2024-09-14T06:18:12.332 INFO:teuthology.orchestra.run.smithi039.stdout: "pglog_hardlimit", 2024-09-14T06:18:12.332 INFO:teuthology.orchestra.run.smithi039.stdout: "purged_snapdirs", 2024-09-14T06:18:12.332 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_deletes", 2024-09-14T06:18:12.332 INFO:teuthology.orchestra.run.smithi039.stdout: "sortbitwise" 2024-09-14T06:18:12.332 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:18:12.332 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_version": 5, 2024-09-14T06:18:12.332 INFO:teuthology.orchestra.run.smithi039.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T06:18:12.332 INFO:teuthology.orchestra.run.smithi039.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T06:18:12.332 INFO:teuthology.orchestra.run.smithi039.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T06:18:12.332 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_snapshot": "", 2024-09-14T06:18:12.332 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_max": 3, 2024-09-14T06:18:12.332 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd": 8, 2024-09-14T06:18:12.332 INFO:teuthology.orchestra.run.smithi039.stdout: "require_min_compat_client": "luminous", 2024-09-14T06:18:12.333 INFO:teuthology.orchestra.run.smithi039.stdout: "min_compat_client": "luminous", 2024-09-14T06:18:12.333 INFO:teuthology.orchestra.run.smithi039.stdout: "require_osd_release": "squid", 2024-09-14T06:18:12.333 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_crimson": false, 2024-09-14T06:18:12.333 INFO:teuthology.orchestra.run.smithi039.stdout: "pools": [ 2024-09-14T06:18:12.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.333 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 1, 2024-09-14T06:18:12.333 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": ".mgr", 2024-09-14T06:18:12.333 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-09-14T06:16:02.429225+0000", 2024-09-14T06:18:12.333 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-09-14T06:18:12.333 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool", 2024-09-14T06:18:12.333 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-09-14T06:18:12.333 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-09-14T06:18:12.333 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-09-14T06:18:12.333 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-09-14T06:18:12.334 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:18:12.334 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:18:12.334 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:18:12.334 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:18:12.334 INFO:teuthology.orchestra.run.smithi039.stdout: "is_stretch_pool": false, 2024-09-14T06:18:12.334 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-09-14T06:18:12.334 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:18:12.334 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 1, 2024-09-14T06:18:12.334 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 1, 2024-09-14T06:18:12.334 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 1, 2024-09-14T06:18:12.334 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 1, 2024-09-14T06:18:12.334 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 1, 2024-09-14T06:18:12.335 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-09-14T06:18:12.335 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-09-14T06:18:12.335 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-09-14T06:18:12.335 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-09-14T06:18:12.335 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-09-14T06:18:12.335 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-09-14T06:18:12.335 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-09-14T06:18:12.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.335 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "18", 2024-09-14T06:18:12.335 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-09-14T06:18:12.335 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:18:12.335 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:18:12.335 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-09-14T06:18:12.335 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:18:12.336 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 0, 2024-09-14T06:18:12.336 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 0, 2024-09-14T06:18:12.336 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-09-14T06:18:12.336 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-09-14T06:18:12.336 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-09-14T06:18:12.336 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-09-14T06:18:12.336 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-09-14T06:18:12.336 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-09-14T06:18:12.336 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-09-14T06:18:12.336 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-09-14T06:18:12.336 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-09-14T06:18:12.336 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-09-14T06:18:12.336 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-09-14T06:18:12.337 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:18:12.337 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:18:12.337 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:18:12.337 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-09-14T06:18:12.337 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-09-14T06:18:12.337 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-09-14T06:18:12.337 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-09-14T06:18:12.337 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-09-14T06:18:12.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.337 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-09-14T06:18:12.337 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-09-14T06:18:12.337 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-09-14T06:18:12.337 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:18:12.337 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:18:12.338 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:18:12.338 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:18:12.338 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-09-14T06:18:12.338 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-09-14T06:18:12.338 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-09-14T06:18:12.338 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-09-14T06:18:12.338 INFO:teuthology.orchestra.run.smithi039.stdout: "options": { 2024-09-14T06:18:12.338 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_max": 32, 2024-09-14T06:18:12.338 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_min": 1 2024-09-14T06:18:12.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.338 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-09-14T06:18:12.338 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": {} 2024-09-14T06:18:12.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.338 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-09-14T06:18:12.338 INFO:teuthology.orchestra.run.smithi039.stdout: "score_type": "Fair distribution", 2024-09-14T06:18:12.339 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 6.9000000953674316, 2024-09-14T06:18:12.339 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 6.9000000953674316, 2024-09-14T06:18:12.339 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T06:18:12.339 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 2, 2024-09-14T06:18:12.339 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 2, 2024-09-14T06:18:12.339 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:18:12.339 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 1, 2024-09-14T06:18:12.339 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:18:12.339 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.339 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-14T06:18:12.340 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": "rbd", 2024-09-14T06:18:12.340 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-09-14T06:16:08.129454+0000", 2024-09-14T06:18:12.340 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 8193, 2024-09-14T06:18:12.340 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:18:12.340 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-09-14T06:18:12.340 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-09-14T06:18:12.340 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-09-14T06:18:12.340 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-09-14T06:18:12.340 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:18:12.340 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:18:12.340 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:18:12.341 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:18:12.341 INFO:teuthology.orchestra.run.smithi039.stdout: "is_stretch_pool": false, 2024-09-14T06:18:12.341 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-09-14T06:18:12.341 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:18:12.341 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 8, 2024-09-14T06:18:12.341 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 8, 2024-09-14T06:18:12.341 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 8, 2024-09-14T06:18:12.341 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 8, 2024-09-14T06:18:12.341 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 8, 2024-09-14T06:18:12.341 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-09-14T06:18:12.341 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-09-14T06:18:12.341 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-09-14T06:18:12.341 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-09-14T06:18:12.342 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-09-14T06:18:12.342 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-09-14T06:18:12.342 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-09-14T06:18:12.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.342 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "22", 2024-09-14T06:18:12.342 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-09-14T06:18:12.342 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:18:12.342 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:18:12.342 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-09-14T06:18:12.342 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:18:12.342 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 2, 2024-09-14T06:18:12.342 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 22, 2024-09-14T06:18:12.342 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-09-14T06:18:12.342 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-09-14T06:18:12.343 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-09-14T06:18:12.343 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-09-14T06:18:12.343 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-09-14T06:18:12.343 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-09-14T06:18:12.343 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-09-14T06:18:12.343 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-09-14T06:18:12.343 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-09-14T06:18:12.343 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-09-14T06:18:12.343 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-09-14T06:18:12.343 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:18:12.343 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:18:12.343 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:18:12.343 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-09-14T06:18:12.343 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-09-14T06:18:12.344 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-09-14T06:18:12.344 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-09-14T06:18:12.344 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-09-14T06:18:12.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.344 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-09-14T06:18:12.344 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-09-14T06:18:12.344 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-09-14T06:18:12.344 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:18:12.344 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:18:12.344 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:18:12.344 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:18:12.344 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-09-14T06:18:12.344 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-09-14T06:18:12.344 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-09-14T06:18:12.345 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-09-14T06:18:12.345 INFO:teuthology.orchestra.run.smithi039.stdout: "options": {}, 2024-09-14T06:18:12.345 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-09-14T06:18:12.345 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd": {} 2024-09-14T06:18:12.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.345 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-09-14T06:18:12.345 INFO:teuthology.orchestra.run.smithi039.stdout: "score_type": "Fair distribution", 2024-09-14T06:18:12.345 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 4.380000114440918, 2024-09-14T06:18:12.345 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 4.380000114440918, 2024-09-14T06:18:12.345 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 1, 2024-09-14T06:18:12.345 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 4.380000114440918, 2024-09-14T06:18:12.345 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 4.380000114440918, 2024-09-14T06:18:12.345 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:18:12.345 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 1, 2024-09-14T06:18:12.345 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:18:12.346 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.346 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 3, 2024-09-14T06:18:12.346 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": "pool_client.0", 2024-09-14T06:18:12.346 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-09-14T06:16:18.928556+0000", 2024-09-14T06:18:12.346 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 8193, 2024-09-14T06:18:12.346 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:18:12.346 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-09-14T06:18:12.346 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-09-14T06:18:12.346 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-09-14T06:18:12.346 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-09-14T06:18:12.346 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:18:12.347 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:18:12.347 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:18:12.347 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:18:12.347 INFO:teuthology.orchestra.run.smithi039.stdout: "is_stretch_pool": false, 2024-09-14T06:18:12.347 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-09-14T06:18:12.347 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:18:12.347 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 32, 2024-09-14T06:18:12.347 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 32, 2024-09-14T06:18:12.347 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 32, 2024-09-14T06:18:12.347 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 32, 2024-09-14T06:18:12.347 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 32, 2024-09-14T06:18:12.347 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-09-14T06:18:12.347 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-09-14T06:18:12.348 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-09-14T06:18:12.348 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-09-14T06:18:12.348 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-09-14T06:18:12.348 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-09-14T06:18:12.348 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-09-14T06:18:12.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.348 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "117", 2024-09-14T06:18:12.348 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-09-14T06:18:12.348 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:18:12.348 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:18:12.348 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-09-14T06:18:12.348 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:18:12.348 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 33, 2024-09-14T06:18:12.349 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 117, 2024-09-14T06:18:12.349 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-09-14T06:18:12.349 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-09-14T06:18:12.349 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-09-14T06:18:12.349 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-09-14T06:18:12.349 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-09-14T06:18:12.349 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-09-14T06:18:12.349 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-09-14T06:18:12.349 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-09-14T06:18:12.349 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-09-14T06:18:12.349 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-09-14T06:18:12.349 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-09-14T06:18:12.349 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:18:12.349 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:18:12.350 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:18:12.350 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-09-14T06:18:12.350 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-09-14T06:18:12.350 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-09-14T06:18:12.350 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-09-14T06:18:12.350 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-09-14T06:18:12.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.350 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-09-14T06:18:12.350 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-09-14T06:18:12.350 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-09-14T06:18:12.350 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:18:12.350 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:18:12.350 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:18:12.351 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:18:12.351 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-09-14T06:18:12.351 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-09-14T06:18:12.351 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-09-14T06:18:12.351 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-09-14T06:18:12.351 INFO:teuthology.orchestra.run.smithi039.stdout: "options": {}, 2024-09-14T06:18:12.351 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-09-14T06:18:12.351 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd": {} 2024-09-14T06:18:12.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.351 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-09-14T06:18:12.351 INFO:teuthology.orchestra.run.smithi039.stdout: "score_type": "Fair distribution", 2024-09-14T06:18:12.351 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 1.75, 2024-09-14T06:18:12.351 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 1.75, 2024-09-14T06:18:12.351 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 1.1399999856948853, 2024-09-14T06:18:12.351 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 2, 2024-09-14T06:18:12.351 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 2, 2024-09-14T06:18:12.352 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:18:12.352 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 1, 2024-09-14T06:18:12.352 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:18:12.352 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.352 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.352 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:18:12.352 INFO:teuthology.orchestra.run.smithi039.stdout: "osds": [ 2024-09-14T06:18:12.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.352 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 0, 2024-09-14T06:18:12.352 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "c669ea76-ed19-4c12-9e56-d79d62846f95", 2024-09-14T06:18:12.352 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:18:12.352 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:18:12.352 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:18:12.352 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:18:12.353 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:18:12.353 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:18:12.353 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:18:12.353 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 92, 2024-09-14T06:18:12.353 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:18:12.353 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:18:12.353 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:18:12.353 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.353 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.353 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6808", 2024-09-14T06:18:12.353 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:18:12.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.354 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.354 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6809", 2024-09-14T06:18:12.354 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:18:12.354 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.354 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.354 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:18:12.354 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.354 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.354 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6810", 2024-09-14T06:18:12.354 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:18:12.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.355 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.355 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6811", 2024-09-14T06:18:12.355 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:18:12.355 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.355 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.355 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:18:12.355 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.355 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.355 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6814", 2024-09-14T06:18:12.355 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:18:12.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.356 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.356 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6815", 2024-09-14T06:18:12.356 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:18:12.356 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.356 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.356 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:18:12.356 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.356 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.356 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6812", 2024-09-14T06:18:12.356 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:18:12.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.357 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.357 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6813", 2024-09-14T06:18:12.357 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1157559201 2024-09-14T06:18:12.357 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.357 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.357 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6809/1157559201", 2024-09-14T06:18:12.357 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6811/1157559201", 2024-09-14T06:18:12.357 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6815/1157559201", 2024-09-14T06:18:12.357 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6813/1157559201", 2024-09-14T06:18:12.357 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:18:12.357 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:18:12.357 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:18:12.358 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.358 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 1, 2024-09-14T06:18:12.358 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b", 2024-09-14T06:18:12.358 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:18:12.358 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:18:12.358 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:18:12.358 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:18:12.358 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:18:12.358 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:18:12.358 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:18:12.358 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 84, 2024-09-14T06:18:12.358 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:18:12.358 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:18:12.359 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:18:12.359 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.359 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.359 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6816", 2024-09-14T06:18:12.359 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:18:12.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.359 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.359 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6817", 2024-09-14T06:18:12.359 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:18:12.359 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.359 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.359 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:18:12.360 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.360 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.360 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6818", 2024-09-14T06:18:12.360 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:18:12.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.360 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.360 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6819", 2024-09-14T06:18:12.360 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:18:12.360 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.360 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.361 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:18:12.361 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.361 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.361 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6822", 2024-09-14T06:18:12.361 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:18:12.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.361 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.361 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6823", 2024-09-14T06:18:12.361 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:18:12.361 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.361 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.362 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:18:12.362 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.362 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.362 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6820", 2024-09-14T06:18:12.362 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:18:12.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.362 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.362 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6821", 2024-09-14T06:18:12.362 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4062089212 2024-09-14T06:18:12.362 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.362 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.362 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6817/4062089212", 2024-09-14T06:18:12.363 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6819/4062089212", 2024-09-14T06:18:12.363 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6823/4062089212", 2024-09-14T06:18:12.363 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6821/4062089212", 2024-09-14T06:18:12.363 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:18:12.363 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:18:12.363 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:18:12.363 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.363 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 2, 2024-09-14T06:18:12.363 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "9bda988f-e615-4545-849b-318c408c4c42", 2024-09-14T06:18:12.363 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:18:12.363 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:18:12.363 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:18:12.364 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 0, 2024-09-14T06:18:12.364 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:18:12.364 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:18:12.364 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:18:12.364 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 84, 2024-09-14T06:18:12.364 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:18:12.364 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:18:12.364 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:18:12.364 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.364 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.364 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6824", 2024-09-14T06:18:12.364 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:18:12.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.365 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6825", 2024-09-14T06:18:12.365 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:18:12.365 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.365 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.365 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:18:12.365 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.365 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6826", 2024-09-14T06:18:12.365 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:18:12.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.366 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6827", 2024-09-14T06:18:12.366 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:18:12.366 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.366 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.366 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:18:12.366 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.366 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.366 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6830", 2024-09-14T06:18:12.366 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:18:12.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.366 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.366 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6831", 2024-09-14T06:18:12.367 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:18:12.367 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.367 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.367 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:18:12.367 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.367 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.367 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6828", 2024-09-14T06:18:12.367 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:18:12.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.367 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.367 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6829", 2024-09-14T06:18:12.367 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1724773882 2024-09-14T06:18:12.368 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.368 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.368 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6825/1724773882", 2024-09-14T06:18:12.368 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6827/1724773882", 2024-09-14T06:18:12.368 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6831/1724773882", 2024-09-14T06:18:12.368 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6829/1724773882", 2024-09-14T06:18:12.368 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:18:12.368 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:18:12.368 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:18:12.368 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.368 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 3, 2024-09-14T06:18:12.369 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "8ea6cd69-2ef1-42ea-b339-7394bf8f4a87", 2024-09-14T06:18:12.369 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:18:12.369 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:18:12.369 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:18:12.369 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:18:12.369 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:18:12.369 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:18:12.369 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 15, 2024-09-14T06:18:12.369 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 85, 2024-09-14T06:18:12.369 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:18:12.369 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:18:12.369 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:18:12.369 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.370 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.370 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6800", 2024-09-14T06:18:12.370 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:18:12.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.370 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.370 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6801", 2024-09-14T06:18:12.370 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:18:12.370 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.370 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.370 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:18:12.370 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.371 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.371 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6802", 2024-09-14T06:18:12.371 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:18:12.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.371 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.371 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6803", 2024-09-14T06:18:12.371 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:18:12.371 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.371 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.371 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:18:12.371 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.372 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.372 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6806", 2024-09-14T06:18:12.372 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:18:12.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.372 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.372 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6807", 2024-09-14T06:18:12.372 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:18:12.372 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.372 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.372 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:18:12.372 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.372 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.373 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6804", 2024-09-14T06:18:12.373 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:18:12.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.373 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.373 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6805", 2024-09-14T06:18:12.373 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3775031293 2024-09-14T06:18:12.373 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.373 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.373 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6801/3775031293", 2024-09-14T06:18:12.373 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6803/3775031293", 2024-09-14T06:18:12.373 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6807/3775031293", 2024-09-14T06:18:12.373 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6805/3775031293", 2024-09-14T06:18:12.374 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:18:12.374 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:18:12.374 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:18:12.374 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.374 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 4, 2024-09-14T06:18:12.374 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "fea45e8a-e8ea-4963-b93f-0651b84c9e74", 2024-09-14T06:18:12.374 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:18:12.374 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:18:12.374 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:18:12.374 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:18:12.374 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:18:12.374 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:18:12.374 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:18:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 93, 2024-09-14T06:18:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:18:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:18:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:18:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6824", 2024-09-14T06:18:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:18:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6825", 2024-09-14T06:18:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:18:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.376 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.376 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:18:12.376 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.376 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.376 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6826", 2024-09-14T06:18:12.376 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:18:12.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.376 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.376 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6827", 2024-09-14T06:18:12.376 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:18:12.376 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.376 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.377 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:18:12.377 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.377 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.377 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6830", 2024-09-14T06:18:12.377 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:18:12.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.377 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.377 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6831", 2024-09-14T06:18:12.377 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:18:12.377 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.378 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.378 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:18:12.378 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.378 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.378 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6828", 2024-09-14T06:18:12.378 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:18:12.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.378 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.378 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6829", 2024-09-14T06:18:12.378 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2632317380 2024-09-14T06:18:12.378 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.378 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.379 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6825/2632317380", 2024-09-14T06:18:12.379 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6827/2632317380", 2024-09-14T06:18:12.379 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6831/2632317380", 2024-09-14T06:18:12.379 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6829/2632317380", 2024-09-14T06:18:12.379 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:18:12.379 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:18:12.379 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:18:12.379 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.379 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 5, 2024-09-14T06:18:12.379 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "b601e852-a180-477f-86fc-6a213fa866f0", 2024-09-14T06:18:12.379 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:18:12.379 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:18:12.379 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:18:12.380 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:18:12.380 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:18:12.380 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:18:12.380 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:18:12.380 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 84, 2024-09-14T06:18:12.380 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:18:12.380 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:18:12.380 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:18:12.380 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.380 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.380 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6816", 2024-09-14T06:18:12.380 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:18:12.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.381 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.381 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6817", 2024-09-14T06:18:12.381 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:18:12.381 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.381 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.381 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:18:12.381 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.381 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.381 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6818", 2024-09-14T06:18:12.381 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:18:12.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.382 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.382 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6819", 2024-09-14T06:18:12.382 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:18:12.382 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.382 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.382 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:18:12.382 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.382 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.382 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6822", 2024-09-14T06:18:12.382 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:18:12.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.383 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.383 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6823", 2024-09-14T06:18:12.383 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:18:12.383 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.383 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.383 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:18:12.383 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.383 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.383 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6820", 2024-09-14T06:18:12.383 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:18:12.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.384 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.384 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6821", 2024-09-14T06:18:12.384 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3501664149 2024-09-14T06:18:12.384 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.384 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.384 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6817/3501664149", 2024-09-14T06:18:12.384 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6819/3501664149", 2024-09-14T06:18:12.384 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6823/3501664149", 2024-09-14T06:18:12.384 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6821/3501664149", 2024-09-14T06:18:12.384 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:18:12.384 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:18:12.384 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:18:12.384 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.385 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 6, 2024-09-14T06:18:12.385 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "1113afe0-f524-41b1-8e51-eb5164602d38", 2024-09-14T06:18:12.385 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:18:12.385 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:18:12.385 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:18:12.385 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:18:12.385 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:18:12.385 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:18:12.385 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:18:12.385 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 84, 2024-09-14T06:18:12.385 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:18:12.385 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:18:12.386 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:18:12.386 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.386 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.386 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6800", 2024-09-14T06:18:12.386 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:18:12.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.386 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.386 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6801", 2024-09-14T06:18:12.386 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:18:12.386 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.387 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.387 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:18:12.387 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.387 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.387 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6802", 2024-09-14T06:18:12.387 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:18:12.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.388 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.388 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6803", 2024-09-14T06:18:12.388 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:18:12.388 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.388 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.388 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:18:12.388 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.388 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.388 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6806", 2024-09-14T06:18:12.388 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:18:12.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.389 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.389 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6807", 2024-09-14T06:18:12.389 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:18:12.389 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.389 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.389 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:18:12.389 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.389 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.389 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6804", 2024-09-14T06:18:12.390 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:18:12.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.390 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.390 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6805", 2024-09-14T06:18:12.390 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1395936437 2024-09-14T06:18:12.390 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.390 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.390 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6801/1395936437", 2024-09-14T06:18:12.390 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6803/1395936437", 2024-09-14T06:18:12.390 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6807/1395936437", 2024-09-14T06:18:12.390 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6805/1395936437", 2024-09-14T06:18:12.390 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:18:12.391 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:18:12.391 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:18:12.391 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.391 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 7, 2024-09-14T06:18:12.391 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "c9162545-318b-4591-be64-cc8ace9097fe", 2024-09-14T06:18:12.391 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-09-14T06:18:12.391 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-09-14T06:18:12.391 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-09-14T06:18:12.391 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-09-14T06:18:12.391 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-09-14T06:18:12.392 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-09-14T06:18:12.392 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 14, 2024-09-14T06:18:12.392 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 23, 2024-09-14T06:18:12.392 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-09-14T06:18:12.392 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-09-14T06:18:12.392 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-09-14T06:18:12.392 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.392 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.392 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6808", 2024-09-14T06:18:12.392 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:18:12.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.393 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.393 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6809", 2024-09-14T06:18:12.393 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:18:12.393 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.393 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.393 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-09-14T06:18:12.393 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.393 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.393 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6810", 2024-09-14T06:18:12.393 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:18:12.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.394 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.394 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6811", 2024-09-14T06:18:12.394 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:18:12.394 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.394 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.394 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-09-14T06:18:12.394 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.394 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.394 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6814", 2024-09-14T06:18:12.394 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:18:12.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.395 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.395 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6815", 2024-09-14T06:18:12.395 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:18:12.395 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.395 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.395 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-09-14T06:18:12.395 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-09-14T06:18:12.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.395 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-09-14T06:18:12.395 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6812", 2024-09-14T06:18:12.395 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:18:12.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.396 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-14T06:18:12.396 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.87:6813", 2024-09-14T06:18:12.396 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 711112216 2024-09-14T06:18:12.396 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.396 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.396 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.87:6809/711112216", 2024-09-14T06:18:12.396 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.87:6811/711112216", 2024-09-14T06:18:12.396 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.87:6815/711112216", 2024-09-14T06:18:12.396 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.87:6813/711112216", 2024-09-14T06:18:12.396 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-09-14T06:18:12.396 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-09-14T06:18:12.396 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-09-14T06:18:12.396 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.396 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.397 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:18:12.397 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_xinfo": [ 2024-09-14T06:18:12.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.397 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 0, 2024-09-14T06:18:12.397 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:18:12.397 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:18:12.397 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:18:12.397 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:18:12.397 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:18:12.397 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.705575+0000", 2024-09-14T06:18:12.397 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:18:12.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.397 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 1, 2024-09-14T06:18:12.398 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:18:12.398 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:18:12.398 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:18:12.398 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:18:12.398 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:18:12.398 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.806946+0000", 2024-09-14T06:18:12.398 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:18:12.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.398 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 2, 2024-09-14T06:18:12.398 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:18:12.398 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:18:12.398 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:18:12.398 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:18:12.398 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:18:12.399 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.853831+0000", 2024-09-14T06:18:12.399 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:18:12.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.399 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 3, 2024-09-14T06:18:12.399 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:18:12.399 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:18:12.399 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:18:12.399 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:18:12.399 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:18:12.399 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.690315+0000", 2024-09-14T06:18:12.399 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:18:12.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.399 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 4, 2024-09-14T06:18:12.400 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:18:12.400 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:18:12.400 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:18:12.400 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:18:12.400 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:18:12.400 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.494526+0000", 2024-09-14T06:18:12.400 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:18:12.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.400 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 5, 2024-09-14T06:18:12.400 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:18:12.400 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:18:12.400 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:18:12.400 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:18:12.400 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:18:12.401 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.546122+0000", 2024-09-14T06:18:12.401 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:18:12.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.401 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 6, 2024-09-14T06:18:12.401 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:18:12.401 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:18:12.401 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:18:12.401 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:18:12.401 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:18:12.401 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.173691+0000", 2024-09-14T06:18:12.401 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:18:12.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 7, 2024-09-14T06:18:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-09-14T06:18:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-09-14T06:18:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-09-14T06:18:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540719139924082687, 2024-09-14T06:18:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-09-14T06:18:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-09-14T06:15:58.148509+0000", 2024-09-14T06:18:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-09-14T06:18:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:18:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap": [ 2024-09-14T06:18:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: "pgid": "3.1c", 2024-09-14T06:18:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: "osds": [ 2024-09-14T06:18:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: 4, 2024-09-14T06:18:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: 6 2024-09-14T06:18:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:18:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap_items": [ 2024-09-14T06:18:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: "pgid": "3.1f", 2024-09-14T06:18:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: "mappings": [ 2024-09-14T06:18:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: "from": 1, 2024-09-14T06:18:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: "to": 2 2024-09-14T06:18:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.404 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.404 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:18:12.404 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap_primaries": [], 2024-09-14T06:18:12.404 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_temp": [], 2024-09-14T06:18:12.404 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_temp": [], 2024-09-14T06:18:12.404 INFO:teuthology.orchestra.run.smithi039.stdout: "blocklist": {}, 2024-09-14T06:18:12.404 INFO:teuthology.orchestra.run.smithi039.stdout: "range_blocklist": {}, 2024-09-14T06:18:12.404 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profiles": { 2024-09-14T06:18:12.404 INFO:teuthology.orchestra.run.smithi039.stdout: "default": { 2024-09-14T06:18:12.404 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-failure-domain": "osd", 2024-09-14T06:18:12.404 INFO:teuthology.orchestra.run.smithi039.stdout: "k": "2", 2024-09-14T06:18:12.404 INFO:teuthology.orchestra.run.smithi039.stdout: "m": "1", 2024-09-14T06:18:12.404 INFO:teuthology.orchestra.run.smithi039.stdout: "plugin": "jerasure", 2024-09-14T06:18:12.404 INFO:teuthology.orchestra.run.smithi039.stdout: "technique": "reed_sol_van" 2024-09-14T06:18:12.405 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:12.405 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps_queue": [ 2024-09-14T06:18:12.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.405 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 3, 2024-09-14T06:18:12.405 INFO:teuthology.orchestra.run.smithi039.stdout: "snaps": [ 2024-09-14T06:18:12.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.405 INFO:teuthology.orchestra.run.smithi039.stdout: "begin": 7, 2024-09-14T06:18:12.405 INFO:teuthology.orchestra.run.smithi039.stdout: "length": 5 2024-09-14T06:18:12.405 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.405 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.405 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.405 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:18:12.405 INFO:teuthology.orchestra.run.smithi039.stdout: "new_removed_snaps": [ 2024-09-14T06:18:12.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.406 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 3, 2024-09-14T06:18:12.406 INFO:teuthology.orchestra.run.smithi039.stdout: "snaps": [ 2024-09-14T06:18:12.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:12.406 INFO:teuthology.orchestra.run.smithi039.stdout: "begin": 7, 2024-09-14T06:18:12.406 INFO:teuthology.orchestra.run.smithi039.stdout: "length": 1 2024-09-14T06:18:12.406 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.406 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:12.406 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.406 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-14T06:18:12.406 INFO:teuthology.orchestra.run.smithi039.stdout: "new_purged_snaps": [], 2024-09-14T06:18:12.406 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_node_flags": {}, 2024-09-14T06:18:12.406 INFO:teuthology.orchestra.run.smithi039.stdout: "device_class_flags": {}, 2024-09-14T06:18:12.406 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode": { 2024-09-14T06:18:12.407 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode_enabled": false, 2024-09-14T06:18:12.407 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_bucket_count": 0, 2024-09-14T06:18:12.407 INFO:teuthology.orchestra.run.smithi039.stdout: "degraded_stretch_mode": 0, 2024-09-14T06:18:12.407 INFO:teuthology.orchestra.run.smithi039.stdout: "recovering_stretch_mode": 0, 2024-09-14T06:18:12.407 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode_bucket": 0 2024-09-14T06:18:12.407 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:12.407 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:12.409 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:12.409 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:12.410 INFO:tasks.thrashosds.thrasher:j is {'epoch': 117, 'fsid': '038563b1-268f-461f-b6a6-4b4b41ad0236', 'created': '2024-09-14T06:15:49.493035+0000', 'modified': '2024-09-14T06:18:11.199142+0000', 'last_up_change': '2024-09-14T06:16:02.014989+0000', 'last_in_change': '2024-09-14T06:16:26.221200+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:16:02.429225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:16:08.129454+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.380000114440918, 'score_stable': 4.380000114440918, 'optimal_score': 1, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:16:18.928556+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 117, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'c669ea76-ed19-4c12-9e56-d79d62846f95', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6808', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6809', 'nonce': 1157559201}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6810', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6811', 'nonce': 1157559201}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6814', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6815', 'nonce': 1157559201}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6812', 'nonce': 1157559201}, {'type': 'v1', 'addr': '172.21.15.39:6813', 'nonce': 1157559201}]}, 'public_addr': '172.21.15.39:6809/1157559201', 'cluster_addr': '172.21.15.39:6811/1157559201', 'heartbeat_back_addr': '172.21.15.39:6815/1157559201', 'heartbeat_front_addr': '172.21.15.39:6813/1157559201', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6816', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6817', 'nonce': 4062089212}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6818', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6819', 'nonce': 4062089212}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6822', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6823', 'nonce': 4062089212}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6820', 'nonce': 4062089212}, {'type': 'v1', 'addr': '172.21.15.39:6821', 'nonce': 4062089212}]}, 'public_addr': '172.21.15.39:6817/4062089212', 'cluster_addr': '172.21.15.39:6819/4062089212', 'heartbeat_back_addr': '172.21.15.39:6823/4062089212', 'heartbeat_front_addr': '172.21.15.39:6821/4062089212', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9bda988f-e615-4545-849b-318c408c4c42', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6824', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6825', 'nonce': 1724773882}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6826', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6827', 'nonce': 1724773882}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6830', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6831', 'nonce': 1724773882}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6828', 'nonce': 1724773882}, {'type': 'v1', 'addr': '172.21.15.39:6829', 'nonce': 1724773882}]}, 'public_addr': '172.21.15.39:6825/1724773882', 'cluster_addr': '172.21.15.39:6827/1724773882', 'heartbeat_back_addr': '172.21.15.39:6831/1724773882', 'heartbeat_front_addr': '172.21.15.39:6829/1724773882', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8ea6cd69-2ef1-42ea-b339-7394bf8f4a87', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 85, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6800', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6801', 'nonce': 3775031293}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6802', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6803', 'nonce': 3775031293}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6806', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6807', 'nonce': 3775031293}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6804', 'nonce': 3775031293}, {'type': 'v1', 'addr': '172.21.15.39:6805', 'nonce': 3775031293}]}, 'public_addr': '172.21.15.39:6801/3775031293', 'cluster_addr': '172.21.15.39:6803/3775031293', 'heartbeat_back_addr': '172.21.15.39:6807/3775031293', 'heartbeat_front_addr': '172.21.15.39:6805/3775031293', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fea45e8a-e8ea-4963-b93f-0651b84c9e74', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6824', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6825', 'nonce': 2632317380}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6826', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6827', 'nonce': 2632317380}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6830', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6831', 'nonce': 2632317380}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6828', 'nonce': 2632317380}, {'type': 'v1', 'addr': '172.21.15.87:6829', 'nonce': 2632317380}]}, 'public_addr': '172.21.15.87:6825/2632317380', 'cluster_addr': '172.21.15.87:6827/2632317380', 'heartbeat_back_addr': '172.21.15.87:6831/2632317380', 'heartbeat_front_addr': '172.21.15.87:6829/2632317380', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b601e852-a180-477f-86fc-6a213fa866f0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6816', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6817', 'nonce': 3501664149}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6818', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6819', 'nonce': 3501664149}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6822', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6823', 'nonce': 3501664149}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6820', 'nonce': 3501664149}, {'type': 'v1', 'addr': '172.21.15.87:6821', 'nonce': 3501664149}]}, 'public_addr': '172.21.15.87:6817/3501664149', 'cluster_addr': '172.21.15.87:6819/3501664149', 'heartbeat_back_addr': '172.21.15.87:6823/3501664149', 'heartbeat_front_addr': '172.21.15.87:6821/3501664149', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '1113afe0-f524-41b1-8e51-eb5164602d38', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6800', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6801', 'nonce': 1395936437}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6802', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6803', 'nonce': 1395936437}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6806', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6807', 'nonce': 1395936437}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6804', 'nonce': 1395936437}, {'type': 'v1', 'addr': '172.21.15.87:6805', 'nonce': 1395936437}]}, 'public_addr': '172.21.15.87:6801/1395936437', 'cluster_addr': '172.21.15.87:6803/1395936437', 'heartbeat_back_addr': '172.21.15.87:6807/1395936437', 'heartbeat_front_addr': '172.21.15.87:6805/1395936437', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'c9162545-318b-4591-be64-cc8ace9097fe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6808', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6809', 'nonce': 711112216}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6810', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6811', 'nonce': 711112216}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6814', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6815', 'nonce': 711112216}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6812', 'nonce': 711112216}, {'type': 'v1', 'addr': '172.21.15.87:6813', 'nonce': 711112216}]}, 'public_addr': '172.21.15.87:6809/711112216', 'cluster_addr': '172.21.15.87:6811/711112216', 'heartbeat_back_addr': '172.21.15.87:6815/711112216', 'heartbeat_front_addr': '172.21.15.87:6813/711112216', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.705575+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.806946+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.853831+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.690315+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.494526+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.546122+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.173691+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:15:58.148509+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1c', 'osds': [4, 6]}], 'pg_upmap_items': [{'pgid': '3.1f', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 7, 'length': 5}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 7, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T06:18:12.410 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:18:12.410 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:12.405+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:12.411 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:18:12.509 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:12.505+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:12.583 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:12.584 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:12.601 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:18:12.610 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:12.607+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:12.711 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:12.707+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:12.755 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:12.755 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:12.767 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:18:12.780 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:18:12.781 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:18:12.797 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-14T06:18:12.446396+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":193720611,"num_objects":513,"num_object_clones":67,"num_object_copies":1026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":513,"num_whiteouts":255,"num_read":105810,"num_read_kb":749898,"num_write":17200,"num_write_kb":430828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1206,"num_bytes_recovered":144366785,"num_keys_recovered":219,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15117,"ondisk_log_size":15117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":367248,"kb_used_data":25984,"kb_used_omap":12,"kb_used_meta":340915,"kb_avail":754607472,"statfs":{"total":773094113280,"available":772718051328,"internally_reserved":0,"allocated":26607616,"data_stored":149890775,"data_compressed":4008937,"data_compressed_allocated":10792960,"data_compressed_original":140661729,"omap_allocated":12752,"internal_metadata":349097520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-77859885,"num_objects":-140,"num_object_clones":-10,"num_object_copies":-280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-140,"num_whiteouts":-3,"num_read":608,"num_read_kb":505,"num_write":228,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.014624"},"pg_stats":[{"pgid":"3.1f","version":"117'585","reported_seq":3633,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.461656+0000","last_change":"2024-09-14T06:18:11.332901+0000","last_active":"2024-09-14T06:18:11.461656+0000","last_peered":"2024-09-14T06:18:11.461656+0000","last_clean":"2024-09-14T06:18:11.461656+0000","last_became_active":"2024-09-14T06:17:02.193313+0000","last_became_peered":"2024-09-14T06:17:02.193313+0000","last_unstale":"2024-09-14T06:18:11.461656+0000","last_undegraded":"2024-09-14T06:18:11.461656+0000","last_fullsized":"2024-09-14T06:18:11.461656+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:03:35.377617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011553410000000001,"stat_sum":{"num_bytes":6904730,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":4589,"num_read_kb":53584,"num_write":697,"num_write_kb":19351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"3.1e","version":"117'403","reported_seq":4485,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.377866+0000","last_change":"2024-09-14T06:18:11.331424+0000","last_active":"2024-09-14T06:18:11.377866+0000","last_peered":"2024-09-14T06:18:11.377866+0000","last_clean":"2024-09-14T06:18:11.377866+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:18:11.377866+0000","last_undegraded":"2024-09-14T06:18:11.377866+0000","last_fullsized":"2024-09-14T06:18:11.377866+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:50:06.946541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074994500000000002,"stat_sum":{"num_bytes":2030126,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":10,"num_read":5022,"num_read_kb":43064,"num_write":350,"num_write_kb":15485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"3.1d","version":"111'1115","reported_seq":2450,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.428791+0000","last_change":"2024-09-14T06:18:07.429809+0000","last_active":"2024-09-14T06:18:07.428791+0000","last_peered":"2024-09-14T06:18:07.428791+0000","last_clean":"2024-09-14T06:18:07.428791+0000","last_became_active":"2024-09-14T06:17:38.226838+0000","last_became_peered":"2024-09-14T06:17:38.226838+0000","last_unstale":"2024-09-14T06:18:07.428791+0000","last_undegraded":"2024-09-14T06:18:07.428791+0000","last_fullsized":"2024-09-14T06:18:07.428791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:52:42.905891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089295869999999992,"stat_sum":{"num_bytes":10470999,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":5125,"num_read_kb":14378,"num_write":1566,"num_write_kb":13994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10785266,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.1c","version":"117'906","reported_seq":2943,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.441737+0000","last_change":"2024-09-14T06:18:11.215609+0000","last_active":"2024-09-14T06:18:11.441737+0000","last_peered":"2024-09-14T06:18:11.441737+0000","last_clean":"2024-09-14T06:18:11.441737+0000","last_became_active":"2024-09-14T06:17:48.737020+0000","last_became_peered":"2024-09-14T06:17:48.737020+0000","last_unstale":"2024-09-14T06:18:11.441737+0000","last_undegraded":"2024-09-14T06:18:11.441737+0000","last_fullsized":"2024-09-14T06:18:11.441737+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:23:10.184095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023006290000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":5786,"num_read_kb":9817,"num_write":1320,"num_write_kb":6518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":3869232,"num_keys_recovered":68,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"3.1b","version":"117'345","reported_seq":3768,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.391115+0000","last_change":"2024-09-14T06:18:11.214859+0000","last_active":"2024-09-14T06:18:11.391115+0000","last_peered":"2024-09-14T06:18:11.391115+0000","last_clean":"2024-09-14T06:18:11.391115+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:18:11.391115+0000","last_undegraded":"2024-09-14T06:18:11.391115+0000","last_fullsized":"2024-09-14T06:18:11.391115+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:55:14.515098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001814272,"stat_sum":{"num_bytes":1909892,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":9,"num_read":3342,"num_read_kb":39056,"num_write":364,"num_write_kb":25723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"3.1a","version":"113'526","reported_seq":3227,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.558803+0000","last_change":"2024-09-14T06:18:07.431193+0000","last_active":"2024-09-14T06:18:07.558803+0000","last_peered":"2024-09-14T06:18:07.558803+0000","last_clean":"2024-09-14T06:18:07.558803+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:18:07.558803+0000","last_undegraded":"2024-09-14T06:18:07.558803+0000","last_fullsized":"2024-09-14T06:18:07.558803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:16:04.128598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032412640000000002,"stat_sum":{"num_bytes":5121647,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":17,"num_read":4031,"num_read_kb":13071,"num_write":651,"num_write_kb":7511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.19","version":"117'520","reported_seq":2115,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.414241+0000","last_change":"2024-09-14T06:18:11.214616+0000","last_active":"2024-09-14T06:18:11.414241+0000","last_peered":"2024-09-14T06:18:11.414241+0000","last_clean":"2024-09-14T06:18:11.414241+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:18:11.414241+0000","last_undegraded":"2024-09-14T06:18:11.414241+0000","last_fullsized":"2024-09-14T06:18:11.414241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:17:09.807448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015896199999999999,"stat_sum":{"num_bytes":5497677,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":11,"num_read":2883,"num_read_kb":19234,"num_write":702,"num_write_kb":12873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"3.18","version":"117'397","reported_seq":3036,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.385261+0000","last_change":"2024-09-14T06:18:11.331807+0000","last_active":"2024-09-14T06:18:11.385261+0000","last_peered":"2024-09-14T06:18:11.385261+0000","last_clean":"2024-09-14T06:18:11.385261+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:18:11.385261+0000","last_undegraded":"2024-09-14T06:18:11.385261+0000","last_fullsized":"2024-09-14T06:18:11.385261+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:14:56.508721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045874300000000001,"stat_sum":{"num_bytes":1606999,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":3795,"num_read_kb":37772,"num_write":359,"num_write_kb":32941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"3.17","version":"117'389","reported_seq":1724,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.385901+0000","last_change":"2024-09-14T06:18:11.332239+0000","last_active":"2024-09-14T06:18:11.385901+0000","last_peered":"2024-09-14T06:18:11.385901+0000","last_clean":"2024-09-14T06:18:11.385901+0000","last_became_active":"2024-09-14T06:17:37.233410+0000","last_became_peered":"2024-09-14T06:17:37.233410+0000","last_unstale":"2024-09-14T06:18:11.385901+0000","last_undegraded":"2024-09-14T06:18:11.385901+0000","last_fullsized":"2024-09-14T06:18:11.385901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:28:09.373743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053620299999999996,"stat_sum":{"num_bytes":1252822,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":1416,"num_read_kb":17477,"num_write":336,"num_write_kb":17193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":23467377,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"3.16","version":"109'418","reported_seq":2324,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.429660+0000","last_change":"2024-09-14T06:18:07.430245+0000","last_active":"2024-09-14T06:18:07.429660+0000","last_peered":"2024-09-14T06:18:07.429660+0000","last_clean":"2024-09-14T06:18:07.429660+0000","last_became_active":"2024-09-14T06:17:37.235311+0000","last_became_peered":"2024-09-14T06:17:37.235311+0000","last_unstale":"2024-09-14T06:18:07.429660+0000","last_undegraded":"2024-09-14T06:18:07.429660+0000","last_fullsized":"2024-09-14T06:18:07.429660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:40:44.096601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086075779999999994,"stat_sum":{"num_bytes":5327508,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":2483,"num_read_kb":16488,"num_write":502,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":6785399,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.15","version":"113'397","reported_seq":3650,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.576546+0000","last_change":"2024-09-14T06:18:07.427988+0000","last_active":"2024-09-14T06:18:07.576546+0000","last_peered":"2024-09-14T06:18:07.576546+0000","last_clean":"2024-09-14T06:18:07.576546+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:18:07.576546+0000","last_undegraded":"2024-09-14T06:18:07.576546+0000","last_fullsized":"2024-09-14T06:18:07.576546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:15:45.199242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020002610000000001,"stat_sum":{"num_bytes":12093895,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":4034,"num_read_kb":64540,"num_write":487,"num_write_kb":22218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.14","version":"112'294","reported_seq":2003,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.428107+0000","last_change":"2024-09-14T06:18:07.428692+0000","last_active":"2024-09-14T06:18:07.428107+0000","last_peered":"2024-09-14T06:18:07.428107+0000","last_clean":"2024-09-14T06:18:07.428107+0000","last_became_active":"2024-09-14T06:17:37.231244+0000","last_became_peered":"2024-09-14T06:17:37.231244+0000","last_unstale":"2024-09-14T06:18:07.428107+0000","last_undegraded":"2024-09-14T06:18:07.428107+0000","last_fullsized":"2024-09-14T06:18:07.428107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:30:56.042466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082021679999999993,"stat_sum":{"num_bytes":11530735,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":2907,"num_read_kb":27191,"num_write":261,"num_write_kb":15734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":32506492,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.420805+0000","last_change":"2024-09-14T06:17:34.200709+0000","last_active":"2024-09-14T06:18:07.420805+0000","last_peered":"2024-09-14T06:18:07.420805+0000","last_clean":"2024-09-14T06:18:07.420805+0000","last_became_active":"2024-09-14T06:17:34.200238+0000","last_became_peered":"2024-09-14T06:17:34.200238+0000","last_unstale":"2024-09-14T06:18:07.420805+0000","last_undegraded":"2024-09-14T06:18:07.420805+0000","last_fullsized":"2024-09-14T06:18:07.420805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:34.864695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"117'466","reported_seq":1291,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.417477+0000","last_change":"2024-09-14T06:18:11.215286+0000","last_active":"2024-09-14T06:18:11.417477+0000","last_peered":"2024-09-14T06:18:11.417477+0000","last_clean":"2024-09-14T06:18:11.417477+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:18:11.417477+0000","last_undegraded":"2024-09-14T06:18:11.417477+0000","last_fullsized":"2024-09-14T06:18:11.417477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:02:11.334761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019732230000000001,"stat_sum":{"num_bytes":1998954,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":6,"num_read":1527,"num_read_kb":9700,"num_write":480,"num_write_kb":9164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"2.6","version":"0'0","reported_seq":210,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.213240+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:18:11.213240+0000","last_peered":"2024-09-14T06:18:11.213240+0000","last_clean":"2024-09-14T06:18:11.213240+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:18:11.213240+0000","last_undegraded":"2024-09-14T06:18:11.213240+0000","last_fullsized":"2024-09-14T06:18:11.213240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"117'371","reported_seq":3244,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.391906+0000","last_change":"2024-09-14T06:18:11.216267+0000","last_active":"2024-09-14T06:18:11.391906+0000","last_peered":"2024-09-14T06:18:11.391906+0000","last_clean":"2024-09-14T06:18:11.391906+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:18:11.391906+0000","last_undegraded":"2024-09-14T06:18:11.391906+0000","last_fullsized":"2024-09-14T06:18:11.391906+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:10:49.774359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028061150000000001,"stat_sum":{"num_bytes":10468179,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":3293,"num_read_kb":28583,"num_write":333,"num_write_kb":15237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":73,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.421358+0000","last_change":"2024-09-14T06:17:34.196668+0000","last_active":"2024-09-14T06:18:07.421358+0000","last_peered":"2024-09-14T06:18:07.421358+0000","last_clean":"2024-09-14T06:18:07.421358+0000","last_became_active":"2024-09-14T06:17:34.196072+0000","last_became_peered":"2024-09-14T06:17:34.196072+0000","last_unstale":"2024-09-14T06:18:07.421358+0000","last_undegraded":"2024-09-14T06:18:07.421358+0000","last_fullsized":"2024-09-14T06:18:07.421358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:18:45.616834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"117'359","reported_seq":1964,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.364060+0000","last_change":"2024-09-14T06:18:11.301702+0000","last_active":"2024-09-14T06:18:11.364060+0000","last_peered":"2024-09-14T06:18:11.364060+0000","last_clean":"2024-09-14T06:18:11.364060+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:18:11.364060+0000","last_undegraded":"2024-09-14T06:18:11.364060+0000","last_fullsized":"2024-09-14T06:18:11.364060+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:43:09.140249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001073774,"stat_sum":{"num_bytes":7836653,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":1780,"num_read_kb":12934,"num_write":303,"num_write_kb":11582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":210,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.331443+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:18:11.331443+0000","last_peered":"2024-09-14T06:18:11.331443+0000","last_clean":"2024-09-14T06:18:11.331443+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:18:11.331443+0000","last_undegraded":"2024-09-14T06:18:11.331443+0000","last_fullsized":"2024-09-14T06:18:11.331443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"117'267","reported_seq":1701,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.422164+0000","last_change":"2024-09-14T06:18:11.301509+0000","last_active":"2024-09-14T06:18:11.422164+0000","last_peered":"2024-09-14T06:18:11.422164+0000","last_clean":"2024-09-14T06:18:11.422164+0000","last_became_active":"2024-09-14T06:16:33.547925+0000","last_became_peered":"2024-09-14T06:16:33.547925+0000","last_unstale":"2024-09-14T06:18:11.422164+0000","last_undegraded":"2024-09-14T06:18:11.422164+0000","last_fullsized":"2024-09-14T06:18:11.422164+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:29:59.326367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011146140000000001,"stat_sum":{"num_bytes":5118344,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":11,"num_read":1362,"num_read_kb":21261,"num_write":261,"num_write_kb":9954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"3.3","version":"113'353","reported_seq":1085,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.551377+0000","last_change":"2024-09-14T06:18:07.429214+0000","last_active":"2024-09-14T06:18:07.551377+0000","last_peered":"2024-09-14T06:18:07.551377+0000","last_clean":"2024-09-14T06:18:07.551377+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:18:07.551377+0000","last_undegraded":"2024-09-14T06:18:07.551377+0000","last_fullsized":"2024-09-14T06:18:07.551377+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:39:31.110478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028864540000000001,"stat_sum":{"num_bytes":5199972,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":8,"num_read":499,"num_read_kb":4207,"num_write":278,"num_write_kb":7109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"2.2","version":"22'2","reported_seq":77,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.420164+0000","last_change":"2024-09-14T06:17:34.203977+0000","last_active":"2024-09-14T06:18:07.420164+0000","last_peered":"2024-09-14T06:18:07.420164+0000","last_clean":"2024-09-14T06:18:07.420164+0000","last_became_active":"2024-09-14T06:17:34.199244+0000","last_became_peered":"2024-09-14T06:17:34.199244+0000","last_unstale":"2024-09-14T06:18:07.420164+0000","last_undegraded":"2024-09-14T06:18:07.420164+0000","last_fullsized":"2024-09-14T06:18:07.420164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:21.934094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"117'404","reported_seq":1845,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.464047+0000","last_change":"2024-09-14T06:18:11.332133+0000","last_active":"2024-09-14T06:18:11.464047+0000","last_peered":"2024-09-14T06:18:11.464047+0000","last_clean":"2024-09-14T06:18:11.464047+0000","last_became_active":"2024-09-14T06:16:20.196215+0000","last_became_peered":"2024-09-14T06:16:20.196215+0000","last_unstale":"2024-09-14T06:18:11.464047+0000","last_undegraded":"2024-09-14T06:18:11.464047+0000","last_fullsized":"2024-09-14T06:18:11.464047+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:59:36.204023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001080159,"stat_sum":{"num_bytes":2582979,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":8,"num_read":2097,"num_read_kb":18969,"num_write":518,"num_write_kb":11051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":210,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.233327+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:18:11.233327+0000","last_peered":"2024-09-14T06:18:11.233327+0000","last_clean":"2024-09-14T06:18:11.233327+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:18:11.233327+0000","last_undegraded":"2024-09-14T06:18:11.233327+0000","last_fullsized":"2024-09-14T06:18:11.233327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"117'540","reported_seq":2179,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.394774+0000","last_change":"2024-09-14T06:18:11.301001+0000","last_active":"2024-09-14T06:18:11.394774+0000","last_peered":"2024-09-14T06:18:11.394774+0000","last_clean":"2024-09-14T06:18:11.394774+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:18:11.394774+0000","last_undegraded":"2024-09-14T06:18:11.394774+0000","last_fullsized":"2024-09-14T06:18:11.394774+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:15:48.843975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913443,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2885,"num_read_kb":8812,"num_write":721,"num_write_kb":8784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"2.0","version":"0'0","reported_seq":202,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.419866+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:18:07.419866+0000","last_peered":"2024-09-14T06:18:07.419866+0000","last_clean":"2024-09-14T06:18:07.419866+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:18:07.419866+0000","last_undegraded":"2024-09-14T06:18:07.419866+0000","last_fullsized":"2024-09-14T06:18:07.419866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"117'465","reported_seq":2008,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.396157+0000","last_change":"2024-09-14T06:18:11.216329+0000","last_active":"2024-09-14T06:18:11.396157+0000","last_peered":"2024-09-14T06:18:11.396157+0000","last_clean":"2024-09-14T06:18:11.396157+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:18:11.396157+0000","last_undegraded":"2024-09-14T06:18:11.396157+0000","last_fullsized":"2024-09-14T06:18:11.396157+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:58:32.005856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002964679,"stat_sum":{"num_bytes":1891430,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":2217,"num_read_kb":23012,"num_write":373,"num_write_kb":14523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"2.3","version":"20'1","reported_seq":223,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.420431+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:18:07.420431+0000","last_peered":"2024-09-14T06:18:07.420431+0000","last_clean":"2024-09-14T06:18:07.420431+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:18:07.420431+0000","last_undegraded":"2024-09-14T06:18:07.420431+0000","last_fullsized":"2024-09-14T06:18:07.420431+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":294,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.427920+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:18:07.427920+0000","last_peered":"2024-09-14T06:18:07.427920+0000","last_clean":"2024-09-14T06:18:07.427920+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:18:07.427920+0000","last_undegraded":"2024-09-14T06:18:07.427920+0000","last_fullsized":"2024-09-14T06:18:07.427920+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"117'436","reported_seq":1641,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.397866+0000","last_change":"2024-09-14T06:18:11.214227+0000","last_active":"2024-09-14T06:18:11.397866+0000","last_peered":"2024-09-14T06:18:11.397866+0000","last_clean":"2024-09-14T06:18:11.397866+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:18:11.397866+0000","last_undegraded":"2024-09-14T06:18:11.397866+0000","last_fullsized":"2024-09-14T06:18:11.397866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:35:29.564700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012590100000000001,"stat_sum":{"num_bytes":1968032,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":12,"num_read":2133,"num_read_kb":6715,"num_write":589,"num_write_kb":6849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"3.9","version":"111'385","reported_seq":1801,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.427709+0000","last_change":"2024-09-14T06:18:07.428373+0000","last_active":"2024-09-14T06:18:07.427709+0000","last_peered":"2024-09-14T06:18:07.427709+0000","last_clean":"2024-09-14T06:18:07.427709+0000","last_became_active":"2024-09-14T06:17:38.224238+0000","last_became_peered":"2024-09-14T06:17:38.224238+0000","last_unstale":"2024-09-14T06:18:07.427709+0000","last_undegraded":"2024-09-14T06:18:07.427709+0000","last_fullsized":"2024-09-14T06:18:07.427709+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:47:02.196929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0078946569999999994,"stat_sum":{"num_bytes":8186252,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":2223,"num_read_kb":13651,"num_write":538,"num_write_kb":14353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8477378,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.a","version":"113'345","reported_seq":2233,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.528087+0000","last_change":"2024-09-14T06:18:07.427556+0000","last_active":"2024-09-14T06:18:07.528087+0000","last_peered":"2024-09-14T06:18:07.528087+0000","last_clean":"2024-09-14T06:18:07.528087+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:18:07.528087+0000","last_undegraded":"2024-09-14T06:18:07.528087+0000","last_fullsized":"2024-09-14T06:18:07.528087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:14:20.340187+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.007139505,"stat_sum":{"num_bytes":13141585,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":2211,"num_read_kb":16098,"num_write":288,"num_write_kb":13801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.b","version":"117'312","reported_seq":2867,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.372151+0000","last_change":"2024-09-14T06:18:11.235348+0000","last_active":"2024-09-14T06:18:11.372151+0000","last_peered":"2024-09-14T06:18:11.372151+0000","last_clean":"2024-09-14T06:18:11.372151+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:18:11.372151+0000","last_undegraded":"2024-09-14T06:18:11.372151+0000","last_fullsized":"2024-09-14T06:18:11.372151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:43.994194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015026690000000001,"stat_sum":{"num_bytes":6133851,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":9,"num_read":2870,"num_read_kb":16793,"num_write":270,"num_write_kb":9901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"3.c","version":"109'181","reported_seq":1600,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.426266+0000","last_change":"2024-09-14T06:18:07.426577+0000","last_active":"2024-09-14T06:18:07.426266+0000","last_peered":"2024-09-14T06:18:07.426266+0000","last_clean":"2024-09-14T06:18:07.426266+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:18:07.426266+0000","last_undegraded":"2024-09-14T06:18:07.426266+0000","last_fullsized":"2024-09-14T06:18:07.426266+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:50:03.117816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069750400000000005,"stat_sum":{"num_bytes":5966452,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":1287,"num_read_kb":29081,"num_write":240,"num_write_kb":11577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.d","version":"110'662","reported_seq":2521,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.428847+0000","last_change":"2024-09-14T06:18:07.429566+0000","last_active":"2024-09-14T06:18:07.428847+0000","last_peered":"2024-09-14T06:18:07.428847+0000","last_clean":"2024-09-14T06:18:07.428847+0000","last_became_active":"2024-09-14T06:17:38.227106+0000","last_became_peered":"2024-09-14T06:17:38.227106+0000","last_unstale":"2024-09-14T06:18:07.428847+0000","last_undegraded":"2024-09-14T06:18:07.428847+0000","last_fullsized":"2024-09-14T06:18:07.428847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:35:22.181122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082906309999999997,"stat_sum":{"num_bytes":14571487,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":3737,"num_read_kb":26925,"num_write":972,"num_write_kb":20272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":29390214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.e","version":"112'730","reported_seq":5234,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.426348+0000","last_change":"2024-09-14T06:18:07.426821+0000","last_active":"2024-09-14T06:18:07.426348+0000","last_peered":"2024-09-14T06:18:07.426348+0000","last_clean":"2024-09-14T06:18:07.426348+0000","last_became_active":"2024-09-14T06:16:33.546796+0000","last_became_peered":"2024-09-14T06:16:33.546796+0000","last_unstale":"2024-09-14T06:18:07.426348+0000","last_undegraded":"2024-09-14T06:18:07.426348+0000","last_fullsized":"2024-09-14T06:18:07.426348+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:50:06.834775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085063399999999996,"stat_sum":{"num_bytes":6677733,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":9974,"num_read_kb":51459,"num_write":672,"num_write_kb":16128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.f","version":"113'638","reported_seq":2876,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.427073+0000","last_change":"2024-09-14T06:18:07.427073+0000","last_active":"2024-09-14T06:18:07.427073+0000","last_peered":"2024-09-14T06:18:07.427073+0000","last_clean":"2024-09-14T06:18:07.427073+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:18:07.427073+0000","last_undegraded":"2024-09-14T06:18:07.427073+0000","last_fullsized":"2024-09-14T06:18:07.427073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:52:10.795745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001110277,"stat_sum":{"num_bytes":20745733,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":4425,"num_read_kb":16166,"num_write":753,"num_write_kb":12749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.10","version":"112'346","reported_seq":2500,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.428504+0000","last_change":"2024-09-14T06:18:07.429118+0000","last_active":"2024-09-14T06:18:07.428504+0000","last_peered":"2024-09-14T06:18:07.428504+0000","last_clean":"2024-09-14T06:18:07.428504+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:18:07.428504+0000","last_undegraded":"2024-09-14T06:18:07.428504+0000","last_fullsized":"2024-09-14T06:18:07.428504+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:33:37.310744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001526257,"stat_sum":{"num_bytes":2960020,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":2476,"num_read_kb":47684,"num_write":331,"num_write_kb":14643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.11","version":"113'504","reported_seq":2758,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.427957+0000","last_change":"2024-09-14T06:18:07.427957+0000","last_active":"2024-09-14T06:18:07.427957+0000","last_peered":"2024-09-14T06:18:07.427957+0000","last_clean":"2024-09-14T06:18:07.427957+0000","last_became_active":"2024-09-14T06:17:38.224251+0000","last_became_peered":"2024-09-14T06:17:38.224251+0000","last_unstale":"2024-09-14T06:18:07.427957+0000","last_undegraded":"2024-09-14T06:18:07.427957+0000","last_fullsized":"2024-09-14T06:18:07.427957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:24:25.122706+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0079625440000000002,"stat_sum":{"num_bytes":5301360,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":4155,"num_read_kb":14398,"num_write":570,"num_write_kb":7659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":12239008,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]},{"pgid":"3.12","version":"117'513","reported_seq":3017,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T06:18:11.394383+0000","last_change":"2024-09-14T06:18:11.301052+0000","last_active":"2024-09-14T06:18:11.394383+0000","last_peered":"2024-09-14T06:18:11.394383+0000","last_clean":"2024-09-14T06:18:11.394383+0000","last_became_active":"2024-09-14T06:17:37.232145+0000","last_became_peered":"2024-09-14T06:17:37.232145+0000","last_unstale":"2024-09-14T06:18:11.394383+0000","last_undegraded":"2024-09-14T06:18:11.394383+0000","last_fullsized":"2024-09-14T06:18:11.394383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:12:23.931090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085550299999999997,"stat_sum":{"num_bytes":3735552,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":11,"num_read":3345,"num_read_kb":11048,"num_write":605,"num_write_kb":8456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":10176554,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"7","length":"5"}]},{"pgid":"3.13","version":"112'470","reported_seq":4015,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T06:18:07.421279+0000","last_change":"2024-09-14T06:18:07.422009+0000","last_active":"2024-09-14T06:18:07.421279+0000","last_peered":"2024-09-14T06:18:07.421279+0000","last_clean":"2024-09-14T06:18:07.421279+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:18:07.421279+0000","last_undegraded":"2024-09-14T06:18:07.421279+0000","last_fullsized":"2024-09-14T06:18:07.421279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:02.011457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606756,"stat_sum":{"num_bytes":4898626,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":12,"num_read":5835,"num_read_kb":16675,"num_write":407,"num_write_kb":5582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":193130224,"num_objects":509,"num_object_clones":67,"num_object_copies":1018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":509,"num_whiteouts":255,"num_read":105744,"num_read_kb":749843,"num_write":17097,"num_write_kb":429198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1203,"num_bytes_recovered":144366766,"num_keys_recovered":219,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13815808,"data_stored":115762282,"data_compressed":3079999,"data_compressed_allocated":9105408,"data_compressed_original":111850465,"omap_allocated":0,"internal_metadata":0},"log_size":15042,"ondisk_log_size":15042,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40780,"kb_used_data":5428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331060,"statfs":{"total":96636764160,"available":96595005440,"internally_reserved":0,"allocated":5558272,"data_stored":44684378,"data_compressed":1527427,"data_compressed_allocated":3481600,"data_compressed_original":43470848,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34936,"kb_used_data":3748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336904,"statfs":{"total":96636764160,"available":96600989696,"internally_reserved":0,"allocated":3837952,"data_stored":23271982,"data_compressed":499786,"data_compressed_allocated":1654784,"data_compressed_original":21961659,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75068,"kb_used_data":3480,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296772,"statfs":{"total":96636764160,"available":96559894528,"internally_reserved":0,"allocated":3563520,"data_stored":23246212,"data_compressed":708650,"data_compressed_allocated":1744896,"data_compressed_original":22229480,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57632,"kb_used_data":4232,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314208,"statfs":{"total":96636764160,"available":96577748992,"internally_reserved":0,"allocated":4333568,"data_stored":29792745,"data_compressed":748911,"data_compressed_allocated":2088960,"data_compressed_original":28389324,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37136,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334704,"statfs":{"total":96636764160,"available":96598736896,"internally_reserved":0,"allocated":1822720,"data_stored":3200888,"data_compressed":43836,"data_compressed_allocated":200704,"data_compressed_original":2347008,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46488,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325352,"statfs":{"total":96636764160,"available":96589160448,"internally_reserved":0,"allocated":2875392,"data_stored":12186151,"data_compressed":239561,"data_compressed_allocated":815104,"data_compressed_original":10953764,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542172,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29456,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342384,"statfs":{"total":96636764160,"available":96606601216,"internally_reserved":0,"allocated":2478080,"data_stored":9073285,"data_compressed":160561,"data_compressed_allocated":573440,"data_compressed_original":7974442,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542174,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45752,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326088,"statfs":{"total":96636764160,"available":96589914112,"internally_reserved":0,"allocated":2138112,"data_stored":4435134,"data_compressed":80205,"data_compressed_allocated":233472,"data_compressed_original":3335204,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2609152,"data_stored":23339657,"data_compressed":563815,"data_compressed_allocated":1765376,"data_compressed_original":22622156,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2002944,"data_stored":18870213,"data_compressed":561112,"data_compressed_allocated":1552384,"data_compressed_original":18510312,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2396160,"data_stored":19500133,"data_compressed":447404,"data_compressed_allocated":1548288,"data_compressed_original":18815931,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3682304,"data_stored":33940530,"data_compressed":1105265,"data_compressed_allocated":3006464,"data_compressed_original":33411072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":749568,"data_stored":3777564,"data_compressed":80205,"data_compressed_allocated":233472,"data_compressed_original":3335204,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":851968,"data_stored":5964350,"data_compressed":133030,"data_compressed_allocated":376832,"data_compressed_original":5557802,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1208320,"data_stored":8934409,"data_compressed":175113,"data_compressed_allocated":544768,"data_compressed_original":8365092,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":315392,"data_stored":1435426,"data_compressed":14055,"data_compressed_allocated":77824,"data_compressed_original":1232896,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:18:12.799 INFO:tasks.thrashosds.thrasher:Setting 3.0 to [4, 5] 2024-09-14T06:18:12.799 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.0', '4', '5'] 2024-09-14T06:18:12.799 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.0 4 5 2024-09-14T06:18:12.839 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:18:12.837+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:18:12.882 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:12.882 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:12.896 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:18:12.940 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:12.937+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:13.014 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:13.011+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:13.065 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:13.065 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:13.079 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:18:13.123 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:13.121+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:13.189 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:13.190 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:13.203 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:18:13.244 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:18:13.248 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:13.245+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:13.316 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:13.315+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:13.368 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:13.368 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:13.382 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:18:13.417 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:13.415+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:13.465 INFO:teuthology.orchestra.run.smithi039.stderr:set 3.0 pg_upmap mapping to [4,5] 2024-09-14T06:18:13.504 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:13.504 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:13.518 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:18:13.513+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:18:13.521 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:18:13.619 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:13.615+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:13.676 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:13.676 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:13.689 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:18:13.735 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:13.733+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:13.811 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:13.811 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:13.822 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:13.819+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:13.825 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:18:13.922 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:13.919+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:13.983 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:18:13.994 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:13.995 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:14.010 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:18:14.023 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:14.019+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:14.123 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:18:14.123 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:18:14.123 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:14.119+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:14.140 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:18:14.141 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:18:14.156 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:18:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:18:14.172 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:18:14.172 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:18:14.178 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:14.179 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-14T06:18:14.179 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-09-14T06:18:14.179 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-09-14T06:18:14.179 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:14.189 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-09-14T06:18:14.197 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:18:14.197 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:18:14.198 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:18:14.219 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:18:14.219 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:18:14.224 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:14.221+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:14.239 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:18:14.240 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:18:14.259 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:18:14.259 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:18:14.315 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:14.315 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-09-14T06:18:14.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-09-14T06:18:14.315 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-14T06:18:14.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.315 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11265 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.315 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:18:14.316 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 98.320636741000001, 2024-09-14T06:18:14.316 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050813732, 2024-09-14T06:18:14.316 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.316 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.316 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:14.316 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.316 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.316 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.316 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11265 2024-09-14T06:18:14.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.316 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.316 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.316 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.316 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:18:14.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:18:14.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.317 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991592+0000", 2024-09-14T06:18:14.317 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:18:14.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.317 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991596+0000", 2024-09-14T06:18:14.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.817e-06 2024-09-14T06:18:14.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991600+0000", 2024-09-14T06:18:14.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.292e-06 2024-09-14T06:18:14.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991602+0000", 2024-09-14T06:18:14.318 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4600000000000002e-06 2024-09-14T06:18:14.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.318 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.318 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041858+0000", 2024-09-14T06:18:14.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050255668000000003 2024-09-14T06:18:14.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.319 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.319 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.041930+0000", 2024-09-14T06:18:14.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:18:14.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.319 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.319 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.042406+0000", 2024-09-14T06:18:14.319 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047580700000000002 2024-09-14T06:18:14.319 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.319 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.319 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.320 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11266 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 565248~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.320 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:18:14.320 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 98.320623669, 2024-09-14T06:18:14.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051736793000000003, 2024-09-14T06:18:14.320 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.320 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.320 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:14.320 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.320 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.320 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.321 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11266 2024-09-14T06:18:14.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.321 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.321 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.321 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:18:14.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.321 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.321 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991605+0000", 2024-09-14T06:18:14.321 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991606+0000", 2024-09-14T06:18:14.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.53e-07 2024-09-14T06:18:14.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991607+0000", 2024-09-14T06:18:14.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0780000000000001e-06 2024-09-14T06:18:14.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.322 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.322 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991612+0000", 2024-09-14T06:18:14.322 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0289999999999996e-06 2024-09-14T06:18:14.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.323 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991616+0000", 2024-09-14T06:18:14.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5639999999999998e-06 2024-09-14T06:18:14.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.323 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043049+0000", 2024-09-14T06:18:14.323 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051432822000000003 2024-09-14T06:18:14.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.323 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.324 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043124+0000", 2024-09-14T06:18:14.324 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5086000000000006e-05 2024-09-14T06:18:14.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.324 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.324 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043342+0000", 2024-09-14T06:18:14.324 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021786099999999999 2024-09-14T06:18:14.324 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.324 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.324 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.324 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11267 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.325 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:18:14.325 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 98.320606237000007, 2024-09-14T06:18:14.325 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052053716999999999, 2024-09-14T06:18:14.325 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.325 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.325 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:14.325 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.325 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.325 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.325 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11267 2024-09-14T06:18:14.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.325 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.325 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:18:14.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:18:14.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991623+0000", 2024-09-14T06:18:14.326 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:18:14.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.326 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.326 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991625+0000", 2024-09-14T06:18:14.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:18:14.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.327 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.327 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991632+0000", 2024-09-14T06:18:14.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:18:14.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.327 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.327 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991636+0000", 2024-09-14T06:18:14.327 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5299999999999998e-06 2024-09-14T06:18:14.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.327 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043373+0000", 2024-09-14T06:18:14.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737005000000003 2024-09-14T06:18:14.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043471+0000", 2024-09-14T06:18:14.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7602000000000005e-05 2024-09-14T06:18:14.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.328 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.328 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043676+0000", 2024-09-14T06:18:14.328 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00020553999999999999 2024-09-14T06:18:14.328 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.328 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.328 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.329 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11268 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 811008~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.329 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:18:14.329 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 98.320585132000005, 2024-09-14T06:18:14.329 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052362364000000002, 2024-09-14T06:18:14.329 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.329 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.329 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:14.329 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.329 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.329 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.330 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11268 2024-09-14T06:18:14.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.330 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:18:14.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.330 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.330 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:18:14.330 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991644+0000", 2024-09-14T06:18:14.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:18:14.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991647+0000", 2024-09-14T06:18:14.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2290000000000002e-06 2024-09-14T06:18:14.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.331 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.331 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991653+0000", 2024-09-14T06:18:14.331 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.02e-06 2024-09-14T06:18:14.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991657+0000", 2024-09-14T06:18:14.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6779999999999999e-06 2024-09-14T06:18:14.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.332 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043706+0000", 2024-09-14T06:18:14.332 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.052048540999999997 2024-09-14T06:18:14.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.332 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.043779+0000", 2024-09-14T06:18:14.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2900999999999999e-05 2024-09-14T06:18:14.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.333 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.333 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044006+0000", 2024-09-14T06:18:14.333 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022758399999999999 2024-09-14T06:18:14.333 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.333 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.333 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.333 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:11269 3.11 3:8bedd018:::rbd_data.1152f87cbac6.0000000000000001:6 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.333 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:18:14.334 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 98.320564024000006, 2024-09-14T06:18:14.334 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053503811999999998, 2024-09-14T06:18:14.334 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.334 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.334 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:14.334 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.334 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.334 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.334 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 11269 2024-09-14T06:18:14.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.334 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.334 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:18:14.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:18:14.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.335 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.335 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991665+0000", 2024-09-14T06:18:14.335 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2200000000000004e-07 2024-09-14T06:18:14.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991668+0000", 2024-09-14T06:18:14.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:18:14.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991674+0000", 2024-09-14T06:18:14.336 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4919999999999998e-06 2024-09-14T06:18:14.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.336 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.336 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:35.991679+0000", 2024-09-14T06:18:14.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7679999999999999e-06 2024-09-14T06:18:14.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.337 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.337 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044904+0000", 2024-09-14T06:18:14.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053224859999999999 2024-09-14T06:18:14.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.337 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.337 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.044973+0000", 2024-09-14T06:18:14.337 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.8773e-05 2024-09-14T06:18:14.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.338 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.338 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:16:36.045169+0000", 2024-09-14T06:18:14.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000196033 2024-09-14T06:18:14.338 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.338 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.338 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.338 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48234 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:14.338 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:18:14.338 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.279292732000002, 2024-09-14T06:18:14.338 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051730353999999999, 2024-09-14T06:18:14.338 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.338 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.339 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.339 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.339 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.339 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.339 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48234 2024-09-14T06:18:14.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.339 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.339 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.339 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:18:14.339 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032936+0000", 2024-09-14T06:18:14.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032937+0000", 2024-09-14T06:18:14.340 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:18:14.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.340 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.340 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032940+0000", 2024-09-14T06:18:14.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0580000000000002e-06 2024-09-14T06:18:14.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032946+0000", 2024-09-14T06:18:14.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2960000000000004e-06 2024-09-14T06:18:14.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.341 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.341 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.032951+0000", 2024-09-14T06:18:14.341 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.938e-06 2024-09-14T06:18:14.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074419+0000", 2024-09-14T06:18:14.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.041467979000000002 2024-09-14T06:18:14.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074490+0000", 2024-09-14T06:18:14.342 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1324999999999999e-05 2024-09-14T06:18:14.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.342 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:14.342 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.074859+0000", 2024-09-14T06:18:14.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00036914899999999999 2024-09-14T06:18:14.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078464+0000", 2024-09-14T06:18:14.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036043170000000001 2024-09-14T06:18:14.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:14.343 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084547+0000", 2024-09-14T06:18:14.343 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.006083097 2024-09-14T06:18:14.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.343 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:14.344 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084614+0000", 2024-09-14T06:18:14.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7509999999999996e-05 2024-09-14T06:18:14.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.344 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.344 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084667+0000", 2024-09-14T06:18:14.344 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:18:14.344 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.344 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.344 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.345 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48237 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:14.345 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:18:14.345 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.279212987999998, 2024-09-14T06:18:14.345 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051772830999999998, 2024-09-14T06:18:14.345 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.345 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.345 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.345 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.345 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.345 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.345 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48237 2024-09-14T06:18:14.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.345 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:18:14.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:18:14.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.346 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.346 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033016+0000", 2024-09-14T06:18:14.346 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0499999999999999e-07 2024-09-14T06:18:14.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033018+0000", 2024-09-14T06:18:14.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0949999999999998e-06 2024-09-14T06:18:14.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033023+0000", 2024-09-14T06:18:14.347 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1620000000000001e-06 2024-09-14T06:18:14.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.347 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.347 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033026+0000", 2024-09-14T06:18:14.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0120000000000001e-06 2024-09-14T06:18:14.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075349+0000", 2024-09-14T06:18:14.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042323373999999997 2024-09-14T06:18:14.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.348 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.348 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075394+0000", 2024-09-14T06:18:14.348 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4861000000000003e-05 2024-09-14T06:18:14.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:14.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075578+0000", 2024-09-14T06:18:14.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00018414899999999999 2024-09-14T06:18:14.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078894+0000", 2024-09-14T06:18:14.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0033158480000000001 2024-09-14T06:18:14.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.349 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:14.349 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084677+0000", 2024-09-14T06:18:14.349 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0057832120000000002 2024-09-14T06:18:14.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:14.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084752+0000", 2024-09-14T06:18:14.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5123000000000006e-05 2024-09-14T06:18:14.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.350 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.350 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084789+0000", 2024-09-14T06:18:14.350 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6690000000000003e-05 2024-09-14T06:18:14.350 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.350 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.350 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.351 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48243 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:14.351 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:18:14.351 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.279131863000003, 2024-09-14T06:18:14.351 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051780221000000001, 2024-09-14T06:18:14.351 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.351 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.351 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.351 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.351 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.351 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.351 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48243 2024-09-14T06:18:14.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.351 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:18:14.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:18:14.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.352 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.352 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033097+0000", 2024-09-14T06:18:14.352 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:18:14.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033100+0000", 2024-09-14T06:18:14.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:18:14.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033108+0000", 2024-09-14T06:18:14.353 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1760000000000004e-06 2024-09-14T06:18:14.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.353 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.353 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033111+0000", 2024-09-14T06:18:14.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0790000000000001e-06 2024-09-14T06:18:14.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075903+0000", 2024-09-14T06:18:14.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.042791942999999999 2024-09-14T06:18:14.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.354 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.075987+0000", 2024-09-14T06:18:14.354 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4467000000000003e-05 2024-09-14T06:18:14.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.354 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:14.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.076259+0000", 2024-09-14T06:18:14.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00027201900000000003 2024-09-14T06:18:14.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078947+0000", 2024-09-14T06:18:14.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0026875879999999999 2024-09-14T06:18:14.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.355 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:14.355 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084797+0000", 2024-09-14T06:18:14.355 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0058500380000000001 2024-09-14T06:18:14.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:14.356 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084843+0000", 2024-09-14T06:18:14.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6298999999999998e-05 2024-09-14T06:18:14.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.356 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.356 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084877+0000", 2024-09-14T06:18:14.356 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4378999999999997e-05 2024-09-14T06:18:14.356 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.356 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.356 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.357 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48287 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:14.357 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:18:14.357 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.279092062999993, 2024-09-14T06:18:14.357 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051924706000000001, 2024-09-14T06:18:14.357 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.357 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.357 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.357 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.357 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.357 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.357 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48287 2024-09-14T06:18:14.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.357 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:18:14.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033137+0000", 2024-09-14T06:18:14.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.358 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.358 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033138+0000", 2024-09-14T06:18:14.358 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6599999999999996e-07 2024-09-14T06:18:14.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033141+0000", 2024-09-14T06:18:14.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0879999999999999e-06 2024-09-14T06:18:14.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.359 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.359 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033147+0000", 2024-09-14T06:18:14.359 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2249999999999997e-06 2024-09-14T06:18:14.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033153+0000", 2024-09-14T06:18:14.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.066e-06 2024-09-14T06:18:14.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.360 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.360 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077790+0000", 2024-09-14T06:18:14.360 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.044637255000000001 2024-09-14T06:18:14.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.077845+0000", 2024-09-14T06:18:14.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4410999999999998e-05 2024-09-14T06:18:14.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:14.361 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078062+0000", 2024-09-14T06:18:14.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00021755599999999999 2024-09-14T06:18:14.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.361 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.080991+0000", 2024-09-14T06:18:14.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029282969999999998 2024-09-14T06:18:14.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:14.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084885+0000", 2024-09-14T06:18:14.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003894495 2024-09-14T06:18:14.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.362 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:14.362 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085020+0000", 2024-09-14T06:18:14.362 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013500799999999999 2024-09-14T06:18:14.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.363 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.363 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085062+0000", 2024-09-14T06:18:14.363 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1638999999999999e-05 2024-09-14T06:18:14.363 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.363 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.363 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.363 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48288 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:14.363 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:18:14.363 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.279068363999997, 2024-09-14T06:18:14.364 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051994018000000003, 2024-09-14T06:18:14.364 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.364 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.364 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.364 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.364 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.364 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.364 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48288 2024-09-14T06:18:14.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.364 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.364 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.364 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:18:14.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:18:14.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.365 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.365 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033161+0000", 2024-09-14T06:18:14.365 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.89e-07 2024-09-14T06:18:14.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033164+0000", 2024-09-14T06:18:14.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.779e-06 2024-09-14T06:18:14.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033171+0000", 2024-09-14T06:18:14.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6279999999999999e-06 2024-09-14T06:18:14.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.366 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.366 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.033176+0000", 2024-09-14T06:18:14.366 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1610000000000002e-06 2024-09-14T06:18:14.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.078991+0000", 2024-09-14T06:18:14.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045815348999999998 2024-09-14T06:18:14.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079086+0000", 2024-09-14T06:18:14.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5282e-05 2024-09-14T06:18:14.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.367 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:14.367 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.079397+0000", 2024-09-14T06:18:14.367 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00031055999999999998 2024-09-14T06:18:14.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.084432+0000", 2024-09-14T06:18:14.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050355599999999997 2024-09-14T06:18:14.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:14.368 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085073+0000", 2024-09-14T06:18:14.368 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064015199999999995 2024-09-14T06:18:14.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.368 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:14.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085120+0000", 2024-09-14T06:18:14.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.7107000000000001e-05 2024-09-14T06:18:14.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.369 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.369 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085155+0000", 2024-09-14T06:18:14.369 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4950999999999998e-05 2024-09-14T06:18:14.369 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.369 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.369 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.369 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48416 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:14.369 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:18:14.370 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.276827952000005, 2024-09-14T06:18:14.370 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053690144000000002, 2024-09-14T06:18:14.370 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.370 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.370 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.370 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.370 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.370 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.370 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48416 2024-09-14T06:18:14.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.370 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.370 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.370 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:18:14.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.371 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035401+0000", 2024-09-14T06:18:14.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.371 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035402+0000", 2024-09-14T06:18:14.371 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7999999999999995e-07 2024-09-14T06:18:14.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.371 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.371 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035405+0000", 2024-09-14T06:18:14.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3960000000000002e-06 2024-09-14T06:18:14.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035413+0000", 2024-09-14T06:18:14.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2209999999999997e-06 2024-09-14T06:18:14.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.372 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.372 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035420+0000", 2024-09-14T06:18:14.372 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1539999999999996e-06 2024-09-14T06:18:14.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081021+0000", 2024-09-14T06:18:14.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.045600389999999998 2024-09-14T06:18:14.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081091+0000", 2024-09-14T06:18:14.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0325000000000002e-05 2024-09-14T06:18:14.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.373 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:14.373 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081434+0000", 2024-09-14T06:18:14.373 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000342519 2024-09-14T06:18:14.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:14.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085254+0000", 2024-09-14T06:18:14.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038201609999999999 2024-09-14T06:18:14.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088977+0000", 2024-09-14T06:18:14.374 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003722805 2024-09-14T06:18:14.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.374 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:14.374 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089048+0000", 2024-09-14T06:18:14.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1266999999999994e-05 2024-09-14T06:18:14.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.375 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.375 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089091+0000", 2024-09-14T06:18:14.375 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3325999999999998e-05 2024-09-14T06:18:14.375 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.375 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.375 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.375 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48417 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:14.375 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:18:14.376 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.276799283000003, 2024-09-14T06:18:14.376 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053805032000000003, 2024-09-14T06:18:14.376 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.376 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.376 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.376 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.376 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.376 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.376 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48417 2024-09-14T06:18:14.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.376 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.376 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:18:14.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:18:14.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.377 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.377 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035430+0000", 2024-09-14T06:18:14.377 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.9500000000000003e-07 2024-09-14T06:18:14.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.378 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.378 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035433+0000", 2024-09-14T06:18:14.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.796e-06 2024-09-14T06:18:14.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.378 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.378 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035442+0000", 2024-09-14T06:18:14.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7120000000000006e-06 2024-09-14T06:18:14.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.378 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.378 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035448+0000", 2024-09-14T06:18:14.378 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.015e-06 2024-09-14T06:18:14.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081855+0000", 2024-09-14T06:18:14.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.046407634000000003 2024-09-14T06:18:14.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.379 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.081938+0000", 2024-09-14T06:18:14.379 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2687000000000003e-05 2024-09-14T06:18:14.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.379 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.379 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:14.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082313+0000", 2024-09-14T06:18:14.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000374872 2024-09-14T06:18:14.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:14.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087616+0000", 2024-09-14T06:18:14.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053032670000000004 2024-09-14T06:18:14.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.380 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.380 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089140+0000", 2024-09-14T06:18:14.380 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0015234350000000001 2024-09-14T06:18:14.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:14.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089184+0000", 2024-09-14T06:18:14.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4804e-05 2024-09-14T06:18:14.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.381 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.381 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089235+0000", 2024-09-14T06:18:14.381 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0315000000000002e-05 2024-09-14T06:18:14.381 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.381 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.381 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.382 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48418 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:14.382 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:18:14.382 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.276755144000006, 2024-09-14T06:18:14.382 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053936425000000003, 2024-09-14T06:18:14.382 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.382 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.382 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.382 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.382 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.382 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.382 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48418 2024-09-14T06:18:14.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.382 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.382 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:18:14.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035474+0000", 2024-09-14T06:18:14.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.383 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.383 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.383 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035475+0000", 2024-09-14T06:18:14.383 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7000000000000003e-07 2024-09-14T06:18:14.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035479+0000", 2024-09-14T06:18:14.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2350000000000001e-06 2024-09-14T06:18:14.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035488+0000", 2024-09-14T06:18:14.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8820000000000007e-06 2024-09-14T06:18:14.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.384 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.384 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035494+0000", 2024-09-14T06:18:14.384 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3060000000000003e-06 2024-09-14T06:18:14.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082826+0000", 2024-09-14T06:18:14.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.047332008000000002 2024-09-14T06:18:14.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.082883+0000", 2024-09-14T06:18:14.385 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7124999999999999e-05 2024-09-14T06:18:14.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.385 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.385 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:14.385 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.083385+0000", 2024-09-14T06:18:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050134099999999998 2024-09-14T06:18:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088888+0000", 2024-09-14T06:18:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005503127 2024-09-14T06:18:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089294+0000", 2024-09-14T06:18:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040669700000000003 2024-09-14T06:18:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:14.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089348+0000", 2024-09-14T06:18:14.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3729e-05 2024-09-14T06:18:14.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.387 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.387 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089410+0000", 2024-09-14T06:18:14.387 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2105000000000003e-05 2024-09-14T06:18:14.387 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.387 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.387 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.387 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48419 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:14.387 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:18:14.387 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.276677513999999, 2024-09-14T06:18:14.388 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054007545999999997, 2024-09-14T06:18:14.388 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.388 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.388 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.388 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.388 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.388 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.388 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48419 2024-09-14T06:18:14.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.388 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.388 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:18:14.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035551+0000", 2024-09-14T06:18:14.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.389 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.389 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.389 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035552+0000", 2024-09-14T06:18:14.389 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4099999999999998e-07 2024-09-14T06:18:14.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035557+0000", 2024-09-14T06:18:14.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1819999999999997e-06 2024-09-14T06:18:14.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.390 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035566+0000", 2024-09-14T06:18:14.390 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1120000000000002e-06 2024-09-14T06:18:14.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.390 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.390 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.391 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035572+0000", 2024-09-14T06:18:14.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0430000000000004e-06 2024-09-14T06:18:14.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.391 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.391 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085304+0000", 2024-09-14T06:18:14.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.049731707 2024-09-14T06:18:14.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.391 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.391 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085371+0000", 2024-09-14T06:18:14.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.6771000000000002e-05 2024-09-14T06:18:14.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:14.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.085601+0000", 2024-09-14T06:18:14.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00022968000000000001 2024-09-14T06:18:14.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:14.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088893+0000", 2024-09-14T06:18:14.392 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032927780000000001 2024-09-14T06:18:14.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.392 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.392 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.392 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089467+0000", 2024-09-14T06:18:14.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057330399999999996 2024-09-14T06:18:14.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:14.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089520+0000", 2024-09-14T06:18:14.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3133000000000001e-05 2024-09-14T06:18:14.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.393 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.393 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089559+0000", 2024-09-14T06:18:14.393 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9195e-05 2024-09-14T06:18:14.393 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.393 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.393 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.394 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48420 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:14.394 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:18:14.394 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.276588283999999, 2024-09-14T06:18:14.394 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056401932000000002, 2024-09-14T06:18:14.394 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.394 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.394 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.394 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.394 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.394 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.394 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48420 2024-09-14T06:18:14.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.395 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:18:14.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.395 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035641+0000", 2024-09-14T06:18:14.395 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.395 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.395 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.396 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035642+0000", 2024-09-14T06:18:14.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1519999999999999e-06 2024-09-14T06:18:14.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.396 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.396 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035647+0000", 2024-09-14T06:18:14.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.2299999999999999e-06 2024-09-14T06:18:14.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.396 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.396 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035660+0000", 2024-09-14T06:18:14.396 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3166999999999999e-05 2024-09-14T06:18:14.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035666+0000", 2024-09-14T06:18:14.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1720000000000004e-06 2024-09-14T06:18:14.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086104+0000", 2024-09-14T06:18:14.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.050437956999999999 2024-09-14T06:18:14.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.397 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.397 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.397 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086164+0000", 2024-09-14T06:18:14.397 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9592999999999997e-05 2024-09-14T06:18:14.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:14.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.086402+0000", 2024-09-14T06:18:14.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000238314 2024-09-14T06:18:14.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:14.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089103+0000", 2024-09-14T06:18:14.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027011600000000002 2024-09-14T06:18:14.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.398 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.398 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.398 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091962+0000", 2024-09-14T06:18:14.398 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028581259999999999 2024-09-14T06:18:14.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:14.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092005+0000", 2024-09-14T06:18:14.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3111000000000001e-05 2024-09-14T06:18:14.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.399 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.399 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092043+0000", 2024-09-14T06:18:14.399 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7950000000000001e-05 2024-09-14T06:18:14.399 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.399 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.399 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.399 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.400 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48424 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:14.400 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:18:14.400 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.276324338999999, 2024-09-14T06:18:14.400 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.056273173000000003, 2024-09-14T06:18:14.400 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.400 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.400 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.400 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.400 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.400 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.400 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48424 2024-09-14T06:18:14.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.401 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.401 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.401 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:18:14.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.401 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.401 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:18:14.401 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.401 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.401 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.401 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035905+0000", 2024-09-14T06:18:14.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2900000000000003e-07 2024-09-14T06:18:14.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035909+0000", 2024-09-14T06:18:14.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6490000000000001e-06 2024-09-14T06:18:14.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.402 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035918+0000", 2024-09-14T06:18:14.402 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5089999999999999e-06 2024-09-14T06:18:14.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.402 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.402 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.035924+0000", 2024-09-14T06:18:14.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7089999999999999e-06 2024-09-14T06:18:14.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087662+0000", 2024-09-14T06:18:14.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.051737621999999997 2024-09-14T06:18:14.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.403 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.403 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.087744+0000", 2024-09-14T06:18:14.403 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2174000000000003e-05 2024-09-14T06:18:14.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:14.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.088106+0000", 2024-09-14T06:18:14.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000361947 2024-09-14T06:18:14.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:14.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092053+0000", 2024-09-14T06:18:14.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0039473080000000001 2024-09-14T06:18:14.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.404 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.404 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.404 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092087+0000", 2024-09-14T06:18:14.404 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4153000000000003e-05 2024-09-14T06:18:14.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:14.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092138+0000", 2024-09-14T06:18:14.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0804999999999999e-05 2024-09-14T06:18:14.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.405 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.405 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.092178+0000", 2024-09-14T06:18:14.405 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9668e-05 2024-09-14T06:18:14.405 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.405 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.405 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.406 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48508 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:14.406 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:18:14.406 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.271249257999997, 2024-09-14T06:18:14.406 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.054906017000000001, 2024-09-14T06:18:14.406 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.406 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.406 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.406 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.406 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.406 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.406 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48508 2024-09-14T06:18:14.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.407 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:18:14.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.407 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040980+0000", 2024-09-14T06:18:14.407 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.407 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040981+0000", 2024-09-14T06:18:14.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7700000000000004e-07 2024-09-14T06:18:14.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040984+0000", 2024-09-14T06:18:14.408 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1480000000000002e-06 2024-09-14T06:18:14.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.408 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.408 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.408 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040991+0000", 2024-09-14T06:18:14.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4580000000000004e-06 2024-09-14T06:18:14.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.040997+0000", 2024-09-14T06:18:14.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7799999999999997e-06 2024-09-14T06:18:14.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.409 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.409 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089601+0000", 2024-09-14T06:18:14.409 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048604264000000001 2024-09-14T06:18:14.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.089708+0000", 2024-09-14T06:18:14.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00010681200000000001 2024-09-14T06:18:14.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:14.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090384+0000", 2024-09-14T06:18:14.410 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00067557499999999998 2024-09-14T06:18:14.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.410 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.410 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.410 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093879+0000", 2024-09-14T06:18:14.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0034954679999999998 2024-09-14T06:18:14.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:14.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095764+0000", 2024-09-14T06:18:14.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0018848070000000001 2024-09-14T06:18:14.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.411 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.411 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:14.411 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095837+0000", 2024-09-14T06:18:14.411 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3433e-05 2024-09-14T06:18:14.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095886+0000", 2024-09-14T06:18:14.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8495000000000002e-05 2024-09-14T06:18:14.412 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.412 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.412 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.412 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:48537 3.9 3:91e7a41d:::rbd_object_map.11525b713fb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:14.412 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:18:14.412 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.269885488, 2024-09-14T06:18:14.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.053681833999999998, 2024-09-14T06:18:14.413 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.413 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.413 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.413 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.413 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 48537 2024-09-14T06:18:14.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.413 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.413 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.413 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:18:14.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:18:14.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042344+0000", 2024-09-14T06:18:14.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2299999999999999e-07 2024-09-14T06:18:14.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042350+0000", 2024-09-14T06:18:14.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5690000000000004e-06 2024-09-14T06:18:14.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042361+0000", 2024-09-14T06:18:14.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1337e-05 2024-09-14T06:18:14.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.042369+0000", 2024-09-14T06:18:14.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8359999999999992e-06 2024-09-14T06:18:14.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090914+0000", 2024-09-14T06:18:14.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.048544945999999999 2024-09-14T06:18:14.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.090995+0000", 2024-09-14T06:18:14.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.0548000000000002e-05 2024-09-14T06:18:14.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:14.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.091508+0000", 2024-09-14T06:18:14.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000513723 2024-09-14T06:18:14.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.093900+0000", 2024-09-14T06:18:14.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0023920429999999999 2024-09-14T06:18:14.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-09-14T06:18:14.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095944+0000", 2024-09-14T06:18:14.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0020437379999999998 2024-09-14T06:18:14.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-09-14T06:18:14.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.095987+0000", 2024-09-14T06:18:14.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2521999999999997e-05 2024-09-14T06:18:14.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:10.096025+0000", 2024-09-14T06:18:14.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8649000000000002e-05 2024-09-14T06:18:14.418 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.418 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.418 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.418 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:63810 3.1d 3.c887d99d (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-14T06:18:14.418 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:18:14.418 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 40.732127601999998, 2024-09-14T06:18:14.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.61346978900000004, 2024-09-14T06:18:14.418 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.418 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.418 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-09-14T06:18:14.418 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.418 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.418 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.418 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 63810 2024-09-14T06:18:14.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.419 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:18:14.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580101+0000", 2024-09-14T06:18:14.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580104+0000", 2024-09-14T06:18:14.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4650000000000001e-06 2024-09-14T06:18:14.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580112+0000", 2024-09-14T06:18:14.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9219999999999997e-06 2024-09-14T06:18:14.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580130+0000", 2024-09-14T06:18:14.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8128999999999999e-05 2024-09-14T06:18:14.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580140+0000", 2024-09-14T06:18:14.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0285000000000001e-05 2024-09-14T06:18:14.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580201+0000", 2024-09-14T06:18:14.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0517999999999999e-05 2024-09-14T06:18:14.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:18:14.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:33.580209+0000", 2024-09-14T06:18:14.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.6430000000000001e-06 2024-09-14T06:18:14.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:34.193554+0000", 2024-09-14T06:18:14.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.61334482899999998 2024-09-14T06:18:14.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:34.193571+0000", 2024-09-14T06:18:14.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6997999999999999e-05 2024-09-14T06:18:14.422 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.422 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.422 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.423 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4434.0:63832 3.1d 3:b99be113:::rbd_object_map.11525ae33f05:head [read 0~52] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T06:18:14.423 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:18:14.423 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 37.074131215999998, 2024-09-14T06:18:14.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.99066972200000003, 2024-09-14T06:18:14.423 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-14T06:18:14.423 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-14T06:18:14.423 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-14T06:18:14.423 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-09-14T06:18:14.423 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4434", 2024-09-14T06:18:14.423 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.423 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 63832 2024-09-14T06:18:14.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.424 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-14T06:18:14.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-14T06:18:14.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:18:14.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-14T06:18:14.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238098+0000", 2024-09-14T06:18:14.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-14T06:18:14.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-14T06:18:14.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238100+0000", 2024-09-14T06:18:14.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0849999999999999e-06 2024-09-14T06:18:14.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-14T06:18:14.425 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238108+0000", 2024-09-14T06:18:14.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.089e-06 2024-09-14T06:18:14.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-14T06:18:14.425 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238123+0000", 2024-09-14T06:18:14.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5418e-05 2024-09-14T06:18:14.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.426 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238136+0000", 2024-09-14T06:18:14.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2204e-05 2024-09-14T06:18:14.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.426 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238913+0000", 2024-09-14T06:18:14.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00077720399999999998 2024-09-14T06:18:14.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for peered", 2024-09-14T06:18:14.426 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:37.238934+0000", 2024-09-14T06:18:14.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1311999999999999e-05 2024-09-14T06:18:14.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-14T06:18:14.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228134+0000", 2024-09-14T06:18:14.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.98919991200000001 2024-09-14T06:18:14.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-14T06:18:14.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228261+0000", 2024-09-14T06:18:14.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000126742 2024-09-14T06:18:14.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-14T06:18:14.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-14T06:18:14.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-09-14T06:18:14.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-14T06:17:38.228768+0000", 2024-09-14T06:18:14.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050675600000000003 2024-09-14T06:18:14.428 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.428 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.428 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.428 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-14T06:18:14.428 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-14T06:18:14.428 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:14.429 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:18:14.430 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:14.427+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:14.531 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:14.527+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:14.561 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:14.562 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:14.562 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:18:14.562 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:14.576 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:18:14.632 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:14.629+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:14.733 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:14.729+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:14.742 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:14.743 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:14.743 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:18:14.743 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:18:14.743 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:14.757 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:18:14.834 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-14T06:18:14.829+0000 7fe976f7c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26363) UID: 0 2024-09-14T06:18:14.881 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:14.881 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:18:14.882 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:18:14.882 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:18:14.882 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.882 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13229 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.882 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:18:14.882 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.780709380999994, 2024-09-14T06:18:14.882 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081798868999999996, 2024-09-14T06:18:14.882 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.882 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.882 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.882 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.882 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.882 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.883 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13229 2024-09-14T06:18:14.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.883 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.883 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.883 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.883 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:18:14.883 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.883 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.883 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.883 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098709+0000", 2024-09-14T06:18:14.883 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.883 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.884 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.884 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098710+0000", 2024-09-14T06:18:14.884 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8199999999999999e-07 2024-09-14T06:18:14.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.884 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.884 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.884 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098714+0000", 2024-09-14T06:18:14.884 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:18:14.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.884 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.884 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.884 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098727+0000", 2024-09-14T06:18:14.884 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2520999999999999e-05 2024-09-14T06:18:14.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098749+0000", 2024-09-14T06:18:14.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2773999999999999e-05 2024-09-14T06:18:14.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160234+0000", 2024-09-14T06:18:14.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.061484430999999999 2024-09-14T06:18:14.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.885 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.885 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.885 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160281+0000", 2024-09-14T06:18:14.885 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7437e-05 2024-09-14T06:18:14.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.886 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.886 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160569+0000", 2024-09-14T06:18:14.886 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028788700000000002 2024-09-14T06:18:14.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.886 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.886 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163121+0000", 2024-09-14T06:18:14.886 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025514090000000001 2024-09-14T06:18:14.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.886 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.886 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.887 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180410+0000", 2024-09-14T06:18:14.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.017289098999999999 2024-09-14T06:18:14.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.887 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.887 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.887 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180472+0000", 2024-09-14T06:18:14.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2547999999999998e-05 2024-09-14T06:18:14.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.887 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.887 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.887 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180508+0000", 2024-09-14T06:18:14.887 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5534000000000002e-05 2024-09-14T06:18:14.887 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.887 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.888 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.888 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.888 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.888 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13230 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.888 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:18:14.888 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.780628179000004, 2024-09-14T06:18:14.888 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081851558000000005, 2024-09-14T06:18:14.888 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.888 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.888 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.888 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.888 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.888 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.889 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13230 2024-09-14T06:18:14.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.889 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.889 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.889 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.889 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:18:14.889 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.889 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.889 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.889 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098790+0000", 2024-09-14T06:18:14.889 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.889 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.890 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.890 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098791+0000", 2024-09-14T06:18:14.890 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8800000000000001e-07 2024-09-14T06:18:14.890 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.890 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.890 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.890 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098795+0000", 2024-09-14T06:18:14.890 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.391e-06 2024-09-14T06:18:14.890 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.890 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.890 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.890 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098804+0000", 2024-09-14T06:18:14.890 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3440000000000001e-06 2024-09-14T06:18:14.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.891 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.891 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.891 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.098810+0000", 2024-09-14T06:18:14.891 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2339999999999999e-06 2024-09-14T06:18:14.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.891 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.891 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.891 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.160930+0000", 2024-09-14T06:18:14.891 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062120018999999999 2024-09-14T06:18:14.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.891 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.891 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.892 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161017+0000", 2024-09-14T06:18:14.892 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6868999999999998e-05 2024-09-14T06:18:14.892 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.892 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.892 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.892 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161300+0000", 2024-09-14T06:18:14.892 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028349499999999998 2024-09-14T06:18:14.892 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.892 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.892 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.892 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164612+0000", 2024-09-14T06:18:14.892 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033119680000000002 2024-09-14T06:18:14.892 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.893 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.893 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.893 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180559+0000", 2024-09-14T06:18:14.893 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.015946612999999998 2024-09-14T06:18:14.893 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.893 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.893 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.893 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180605+0000", 2024-09-14T06:18:14.893 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5890000000000003e-05 2024-09-14T06:18:14.893 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.893 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.893 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.893 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180642+0000", 2024-09-14T06:18:14.893 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:18:14.894 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.894 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.894 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.894 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.894 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.894 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13231 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.894 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:18:14.894 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.780365764999999, 2024-09-14T06:18:14.894 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081750685000000003, 2024-09-14T06:18:14.894 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.894 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.894 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.894 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.895 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.895 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.895 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13231 2024-09-14T06:18:14.895 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.895 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.895 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.895 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.895 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:18:14.895 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.895 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.895 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.895 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.895 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099052+0000", 2024-09-14T06:18:14.895 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.896 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.896 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.896 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099053+0000", 2024-09-14T06:18:14.896 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:18:14.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.896 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.896 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.896 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099059+0000", 2024-09-14T06:18:14.896 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1789999999999999e-06 2024-09-14T06:18:14.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.896 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.896 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.896 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099072+0000", 2024-09-14T06:18:14.897 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3179999999999999e-05 2024-09-14T06:18:14.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.897 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.897 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.897 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099080+0000", 2024-09-14T06:18:14.897 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9030000000000004e-06 2024-09-14T06:18:14.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.897 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.897 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.897 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161681+0000", 2024-09-14T06:18:14.897 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.062601641999999999 2024-09-14T06:18:14.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.897 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.897 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.898 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.161766+0000", 2024-09-14T06:18:14.898 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.4909999999999998e-05 2024-09-14T06:18:14.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.898 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.898 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.898 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162109+0000", 2024-09-14T06:18:14.898 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00034264899999999999 2024-09-14T06:18:14.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.898 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.898 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.898 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164615+0000", 2024-09-14T06:18:14.898 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0025061979999999998 2024-09-14T06:18:14.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.898 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.899 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.899 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180718+0000", 2024-09-14T06:18:14.899 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.016103240000000001 2024-09-14T06:18:14.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.899 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.899 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.899 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180768+0000", 2024-09-14T06:18:14.899 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9781999999999999e-05 2024-09-14T06:18:14.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.899 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.899 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.899 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180803+0000", 2024-09-14T06:18:14.899 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5021999999999998e-05 2024-09-14T06:18:14.900 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.900 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.900 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.900 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.900 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13232 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.900 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:18:14.900 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.780323991000003, 2024-09-14T06:18:14.900 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081861981, 2024-09-14T06:18:14.900 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.900 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.900 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.900 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.901 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.901 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.901 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13232 2024-09-14T06:18:14.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.901 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.901 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.901 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:18:14.901 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.901 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.901 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.901 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099094+0000", 2024-09-14T06:18:14.901 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.902 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.902 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.902 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099095+0000", 2024-09-14T06:18:14.902 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:18:14.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.902 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.902 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.902 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099098+0000", 2024-09-14T06:18:14.902 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5420000000000001e-06 2024-09-14T06:18:14.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.902 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.902 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.902 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099106+0000", 2024-09-14T06:18:14.902 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.824e-06 2024-09-14T06:18:14.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.903 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099112+0000", 2024-09-14T06:18:14.903 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9789999999999998e-06 2024-09-14T06:18:14.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.903 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162576+0000", 2024-09-14T06:18:14.903 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.063463723999999999 2024-09-14T06:18:14.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.903 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.903 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.903 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162627+0000", 2024-09-14T06:18:14.904 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1530000000000003e-05 2024-09-14T06:18:14.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.904 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.904 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.904 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.162854+0000", 2024-09-14T06:18:14.904 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000226159 2024-09-14T06:18:14.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.904 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.904 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.904 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166090+0000", 2024-09-14T06:18:14.904 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032362319999999999 2024-09-14T06:18:14.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.904 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.905 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.905 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180877+0000", 2024-09-14T06:18:14.905 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.014786726 2024-09-14T06:18:14.905 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.905 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.905 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.905 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180920+0000", 2024-09-14T06:18:14.905 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3427000000000003e-05 2024-09-14T06:18:14.905 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.905 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.905 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.905 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180956+0000", 2024-09-14T06:18:14.905 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.6195999999999997e-05 2024-09-14T06:18:14.905 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.905 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.906 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.906 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.906 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.906 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13233 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.906 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:18:14.906 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.780298728000005, 2024-09-14T06:18:14.906 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081924860000000002, 2024-09-14T06:18:14.906 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.906 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.906 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.906 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.906 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.906 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.906 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13233 2024-09-14T06:18:14.907 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.907 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.907 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.907 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.907 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:18:14.907 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.907 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.907 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.907 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.907 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099119+0000", 2024-09-14T06:18:14.907 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.907 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.907 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.907 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.907 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099120+0000", 2024-09-14T06:18:14.908 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:18:14.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.908 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.908 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.908 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099122+0000", 2024-09-14T06:18:14.908 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:18:14.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.908 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.908 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.908 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099126+0000", 2024-09-14T06:18:14.908 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2540000000000003e-06 2024-09-14T06:18:14.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.908 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.908 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.908 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099130+0000", 2024-09-14T06:18:14.909 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.4060000000000001e-06 2024-09-14T06:18:14.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.909 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163169+0000", 2024-09-14T06:18:14.909 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064039415000000002 2024-09-14T06:18:14.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.909 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.909 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163260+0000", 2024-09-14T06:18:14.909 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1038000000000005e-05 2024-09-14T06:18:14.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.909 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.910 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.910 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.163640+0000", 2024-09-14T06:18:14.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037954699999999999 2024-09-14T06:18:14.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.910 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.910 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166093+0000", 2024-09-14T06:18:14.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024533229999999999 2024-09-14T06:18:14.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.910 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.910 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.910 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.180993+0000", 2024-09-14T06:18:14.910 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.01489997 2024-09-14T06:18:14.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.911 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.911 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.911 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181020+0000", 2024-09-14T06:18:14.911 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6806999999999999e-05 2024-09-14T06:18:14.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.911 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.911 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.911 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181044+0000", 2024-09-14T06:18:14.911 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4405000000000001e-05 2024-09-14T06:18:14.911 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.911 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.911 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.912 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.912 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13234 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.912 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:18:14.912 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.780256523000006, 2024-09-14T06:18:14.912 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082015597999999995, 2024-09-14T06:18:14.912 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.912 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.912 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.912 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.912 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.912 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.912 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13234 2024-09-14T06:18:14.912 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.912 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.913 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.913 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.913 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:18:14.913 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.913 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.913 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.913 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:18:14.913 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.913 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.913 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.913 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099162+0000", 2024-09-14T06:18:14.913 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0999999999999998e-07 2024-09-14T06:18:14.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099166+0000", 2024-09-14T06:18:14.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3950000000000001e-06 2024-09-14T06:18:14.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099171+0000", 2024-09-14T06:18:14.914 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6949999999999997e-06 2024-09-14T06:18:14.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.914 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.914 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.914 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099175+0000", 2024-09-14T06:18:14.915 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9029999999999997e-06 2024-09-14T06:18:14.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.915 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.915 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164063+0000", 2024-09-14T06:18:14.915 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.064887608999999999 2024-09-14T06:18:14.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.915 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.915 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164111+0000", 2024-09-14T06:18:14.915 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7648999999999997e-05 2024-09-14T06:18:14.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.915 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.915 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164351+0000", 2024-09-14T06:18:14.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024072899999999999 2024-09-14T06:18:14.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167561+0000", 2024-09-14T06:18:14.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0032094179999999999 2024-09-14T06:18:14.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.916 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181093+0000", 2024-09-14T06:18:14.916 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013532245 2024-09-14T06:18:14.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.916 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.916 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.917 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181139+0000", 2024-09-14T06:18:14.917 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6483999999999997e-05 2024-09-14T06:18:14.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.917 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.917 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.917 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181177+0000", 2024-09-14T06:18:14.917 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7861000000000002e-05 2024-09-14T06:18:14.917 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.917 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.917 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.917 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.917 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13235 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.917 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:18:14.918 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.780149249000004, 2024-09-14T06:18:14.918 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081988088000000001, 2024-09-14T06:18:14.918 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.918 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.918 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.918 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.918 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.918 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.918 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13235 2024-09-14T06:18:14.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.918 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.918 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.918 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.919 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:18:14.919 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.919 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.919 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.919 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099269+0000", 2024-09-14T06:18:14.919 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.919 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.919 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.919 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099270+0000", 2024-09-14T06:18:14.919 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.7400000000000003e-07 2024-09-14T06:18:14.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.919 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.920 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099276+0000", 2024-09-14T06:18:14.920 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:18:14.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.920 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.920 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099284+0000", 2024-09-14T06:18:14.920 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7979999999999995e-06 2024-09-14T06:18:14.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.920 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.920 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.920 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099289+0000", 2024-09-14T06:18:14.920 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:18:14.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.921 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.921 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164664+0000", 2024-09-14T06:18:14.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.065374901999999999 2024-09-14T06:18:14.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.921 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.921 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.164755+0000", 2024-09-14T06:18:14.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0790000000000003e-05 2024-09-14T06:18:14.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.921 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.921 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.921 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165143+0000", 2024-09-14T06:18:14.921 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000387904 2024-09-14T06:18:14.922 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.922 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.922 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.922 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167564+0000", 2024-09-14T06:18:14.922 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0024208989999999998 2024-09-14T06:18:14.922 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.922 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.922 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.922 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181212+0000", 2024-09-14T06:18:14.922 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.013648664 2024-09-14T06:18:14.922 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.922 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.923 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.923 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181237+0000", 2024-09-14T06:18:14.923 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5177999999999998e-05 2024-09-14T06:18:14.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.923 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.923 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.923 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181257+0000", 2024-09-14T06:18:14.923 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9660000000000002e-05 2024-09-14T06:18:14.923 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.923 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.923 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.923 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.923 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13236 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.924 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:18:14.924 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.780042429999995, 2024-09-14T06:18:14.924 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082010939000000005, 2024-09-14T06:18:14.924 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.924 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.924 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.924 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.924 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.924 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.924 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13236 2024-09-14T06:18:14.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.924 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.924 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:18:14.925 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.925 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:18:14.925 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.925 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.925 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.925 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.925 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099376+0000", 2024-09-14T06:18:14.925 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.61e-07 2024-09-14T06:18:14.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.926 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.926 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099382+0000", 2024-09-14T06:18:14.926 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3589999999999999e-06 2024-09-14T06:18:14.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.926 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.926 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099388+0000", 2024-09-14T06:18:14.926 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9619999999999998e-06 2024-09-14T06:18:14.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.926 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.926 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.926 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099393+0000", 2024-09-14T06:18:14.927 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3410000000000002e-06 2024-09-14T06:18:14.927 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.927 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.927 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.927 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165553+0000", 2024-09-14T06:18:14.928 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066159604999999996 2024-09-14T06:18:14.928 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.928 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.928 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.928 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165603+0000", 2024-09-14T06:18:14.928 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.0427999999999999e-05 2024-09-14T06:18:14.928 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.928 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.165827+0000", 2024-09-14T06:18:14.929 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00022358299999999999 2024-09-14T06:18:14.929 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177005+0000", 2024-09-14T06:18:14.929 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.011178044 2024-09-14T06:18:14.929 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.929 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.929 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.929 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181302+0000", 2024-09-14T06:18:14.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042970669999999999 2024-09-14T06:18:14.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.930 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181349+0000", 2024-09-14T06:18:14.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.7287999999999999e-05 2024-09-14T06:18:14.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.930 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.930 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.930 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181387+0000", 2024-09-14T06:18:14.930 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7601000000000001e-05 2024-09-14T06:18:14.930 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.930 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.931 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.931 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.931 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13237 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.931 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:18:14.931 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.779884283000001, 2024-09-14T06:18:14.931 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081927299999999995, 2024-09-14T06:18:14.931 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.931 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.931 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.931 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.931 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.931 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.931 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13237 2024-09-14T06:18:14.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.932 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.932 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.932 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.932 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:18:14.932 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.932 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.932 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.932 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099534+0000", 2024-09-14T06:18:14.932 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.932 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.932 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.933 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099535+0000", 2024-09-14T06:18:14.933 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.4199999999999995e-07 2024-09-14T06:18:14.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.933 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.933 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099538+0000", 2024-09-14T06:18:14.933 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:18:14.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.933 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.933 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099545+0000", 2024-09-14T06:18:14.933 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.6320000000000002e-06 2024-09-14T06:18:14.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.933 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099551+0000", 2024-09-14T06:18:14.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.1770000000000003e-06 2024-09-14T06:18:14.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166131+0000", 2024-09-14T06:18:14.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.066580499000000001 2024-09-14T06:18:14.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.934 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.934 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.934 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166221+0000", 2024-09-14T06:18:14.934 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0000000000000006e-05 2024-09-14T06:18:14.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.935 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.935 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.935 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.166622+0000", 2024-09-14T06:18:14.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00040029799999999999 2024-09-14T06:18:14.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.935 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.935 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.935 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177008+0000", 2024-09-14T06:18:14.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010386772000000001 2024-09-14T06:18:14.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.935 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.935 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.935 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181420+0000", 2024-09-14T06:18:14.935 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044119190000000003 2024-09-14T06:18:14.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.936 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.936 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.936 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181443+0000", 2024-09-14T06:18:14.936 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2603999999999999e-05 2024-09-14T06:18:14.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.936 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.936 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.936 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181461+0000", 2024-09-14T06:18:14.936 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8405000000000001e-05 2024-09-14T06:18:14.936 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.936 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.936 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13238 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:18:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.779847974999996, 2024-09-14T06:18:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082014927000000001, 2024-09-14T06:18:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13238 2024-09-14T06:18:14.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:18:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099570+0000", 2024-09-14T06:18:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.938 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099571+0000", 2024-09-14T06:18:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9699999999999996e-07 2024-09-14T06:18:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099577+0000", 2024-09-14T06:18:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2890000000000003e-06 2024-09-14T06:18:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099587+0000", 2024-09-14T06:18:14.939 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0002e-05 2024-09-14T06:18:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099593+0000", 2024-09-14T06:18:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4539999999999998e-06 2024-09-14T06:18:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167024+0000", 2024-09-14T06:18:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067431480000000002 2024-09-14T06:18:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.940 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167074+0000", 2024-09-14T06:18:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9996e-05 2024-09-14T06:18:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167288+0000", 2024-09-14T06:18:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021410900000000001 2024-09-14T06:18:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177997+0000", 2024-09-14T06:18:14.941 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.010708756 2024-09-14T06:18:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181508+0000", 2024-09-14T06:18:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003511363 2024-09-14T06:18:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181552+0000", 2024-09-14T06:18:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.3609999999999998e-05 2024-09-14T06:18:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.942 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181585+0000", 2024-09-14T06:18:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3271e-05 2024-09-14T06:18:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13239 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:18:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.779756156999994, 2024-09-14T06:18:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081998772999999997, 2024-09-14T06:18:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.943 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13239 2024-09-14T06:18:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:18:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.944 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099662+0000", 2024-09-14T06:18:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099663+0000", 2024-09-14T06:18:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6100000000000001e-07 2024-09-14T06:18:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099666+0000", 2024-09-14T06:18:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.056e-06 2024-09-14T06:18:14.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099672+0000", 2024-09-14T06:18:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8120000000000002e-06 2024-09-14T06:18:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099676+0000", 2024-09-14T06:18:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:18:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.946 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167602+0000", 2024-09-14T06:18:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.067925867000000001 2024-09-14T06:18:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.167693+0000", 2024-09-14T06:18:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0736000000000007e-05 2024-09-14T06:18:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168065+0000", 2024-09-14T06:18:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037245399999999998 2024-09-14T06:18:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.947 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178003+0000", 2024-09-14T06:18:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0099372360000000003 2024-09-14T06:18:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181621+0000", 2024-09-14T06:18:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036178489999999998 2024-09-14T06:18:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181643+0000", 2024-09-14T06:18:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2155e-05 2024-09-14T06:18:14.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181661+0000", 2024-09-14T06:18:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8147000000000001e-05 2024-09-14T06:18:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13240 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:18:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.779637063999999, 2024-09-14T06:18:14.949 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081942607000000001, 2024-09-14T06:18:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13240 2024-09-14T06:18:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:18:14.950 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099781+0000", 2024-09-14T06:18:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099782+0000", 2024-09-14T06:18:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3900000000000004e-07 2024-09-14T06:18:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.951 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099787+0000", 2024-09-14T06:18:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6189999999999999e-06 2024-09-14T06:18:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099793+0000", 2024-09-14T06:18:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.998e-06 2024-09-14T06:18:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.099797+0000", 2024-09-14T06:18:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8449999999999996e-06 2024-09-14T06:18:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.952 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168494+0000", 2024-09-14T06:18:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.068696500999999993 2024-09-14T06:18:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168590+0000", 2024-09-14T06:18:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6733999999999999e-05 2024-09-14T06:18:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.168973+0000", 2024-09-14T06:18:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038229399999999998 2024-09-14T06:18:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.953 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178013+0000", 2024-09-14T06:18:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0090397199999999994 2024-09-14T06:18:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181683+0000", 2024-09-14T06:18:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0036705819999999999 2024-09-14T06:18:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181705+0000", 2024-09-14T06:18:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1551000000000001e-05 2024-09-14T06:18:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.954 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.181724+0000", 2024-09-14T06:18:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9123999999999999e-05 2024-09-14T06:18:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13243 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:18:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.779103333999998, 2024-09-14T06:18:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081767815999999993, 2024-09-14T06:18:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.955 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13243 2024-09-14T06:18:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:18:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:18:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100315+0000", 2024-09-14T06:18:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1e-07 2024-09-14T06:18:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100317+0000", 2024-09-14T06:18:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.092e-06 2024-09-14T06:18:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.957 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100321+0000", 2024-09-14T06:18:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7809999999999999e-06 2024-09-14T06:18:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100324+0000", 2024-09-14T06:18:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.117e-06 2024-09-14T06:18:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171113+0000", 2024-09-14T06:18:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.070788816000000004 2024-09-14T06:18:14.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171202+0000", 2024-09-14T06:18:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8843000000000003e-05 2024-09-14T06:18:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171584+0000", 2024-09-14T06:18:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00038231099999999998 2024-09-14T06:18:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.959 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178101+0000", 2024-09-14T06:18:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0065168329999999997 2024-09-14T06:18:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182041+0000", 2024-09-14T06:18:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0039401810000000001 2024-09-14T06:18:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182065+0000", 2024-09-14T06:18:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3623000000000001e-05 2024-09-14T06:18:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.960 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182083+0000", 2024-09-14T06:18:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7909000000000001e-05 2024-09-14T06:18:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13244 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:18:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.779088513999994, 2024-09-14T06:18:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081837800000000002, 2024-09-14T06:18:14.961 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13244 2024-09-14T06:18:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:18:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:18:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100330+0000", 2024-09-14T06:18:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9299999999999999e-07 2024-09-14T06:18:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100337+0000", 2024-09-14T06:18:14.963 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.939e-06 2024-09-14T06:18:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100341+0000", 2024-09-14T06:18:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2320000000000003e-06 2024-09-14T06:18:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100344+0000", 2024-09-14T06:18:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0759999999999999e-06 2024-09-14T06:18:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.964 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.171996+0000", 2024-09-14T06:18:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.071651919999999994 2024-09-14T06:18:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172079+0000", 2024-09-14T06:18:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.2579999999999999e-05 2024-09-14T06:18:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172457+0000", 2024-09-14T06:18:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037785100000000001 2024-09-14T06:18:14.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178104+0000", 2024-09-14T06:18:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0056471730000000001 2024-09-14T06:18:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182123+0000", 2024-09-14T06:18:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040190019999999998 2024-09-14T06:18:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182147+0000", 2024-09-14T06:18:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4241000000000001e-05 2024-09-14T06:18:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182168+0000", 2024-09-14T06:18:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0492999999999998e-05 2024-09-14T06:18:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.967 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13245 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:18:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.779069007000004, 2024-09-14T06:18:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081893899000000006, 2024-09-14T06:18:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13245 2024-09-14T06:18:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:18:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100349+0000", 2024-09-14T06:18:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100350+0000", 2024-09-14T06:18:14.969 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5499999999999999e-07 2024-09-14T06:18:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100353+0000", 2024-09-14T06:18:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:18:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100357+0000", 2024-09-14T06:18:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.8500000000000004e-06 2024-09-14T06:18:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.970 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100360+0000", 2024-09-14T06:18:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8700000000000001e-06 2024-09-14T06:18:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172862+0000", 2024-09-14T06:18:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.072501992000000001 2024-09-14T06:18:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.172950+0000", 2024-09-14T06:18:14.971 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8988999999999998e-05 2024-09-14T06:18:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173324+0000", 2024-09-14T06:18:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00037366199999999997 2024-09-14T06:18:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178107+0000", 2024-09-14T06:18:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0047830520000000003 2024-09-14T06:18:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.972 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182202+0000", 2024-09-14T06:18:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0040949280000000003 2024-09-14T06:18:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182226+0000", 2024-09-14T06:18:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3535000000000001e-05 2024-09-14T06:18:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182243+0000", 2024-09-14T06:18:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7445e-05 2024-09-14T06:18:14.973 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13246 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:18:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.779054685000006, 2024-09-14T06:18:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081962276000000001, 2024-09-14T06:18:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.974 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13246 2024-09-14T06:18:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:18:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.975 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:18:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100364+0000", 2024-09-14T06:18:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0800000000000001e-07 2024-09-14T06:18:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100367+0000", 2024-09-14T06:18:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8439999999999998e-06 2024-09-14T06:18:14.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100372+0000", 2024-09-14T06:18:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4009999999999996e-06 2024-09-14T06:18:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100378+0000", 2024-09-14T06:18:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7180000000000001e-06 2024-09-14T06:18:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.977 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173780+0000", 2024-09-14T06:18:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.073401991 2024-09-14T06:18:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.173879+0000", 2024-09-14T06:18:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9449000000000003e-05 2024-09-14T06:18:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174240+0000", 2024-09-14T06:18:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036118100000000002 2024-09-14T06:18:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.978 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178109+0000", 2024-09-14T06:18:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0038690619999999999 2024-09-14T06:18:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182281+0000", 2024-09-14T06:18:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041715900000000002 2024-09-14T06:18:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182306+0000", 2024-09-14T06:18:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4477e-05 2024-09-14T06:18:14.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182326+0000", 2024-09-14T06:18:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0254999999999998e-05 2024-09-14T06:18:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13247 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:18:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.778957508000005, 2024-09-14T06:18:14.980 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966593000000004, 2024-09-14T06:18:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13247 2024-09-14T06:18:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:18:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100461+0000", 2024-09-14T06:18:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100462+0000", 2024-09-14T06:18:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9800000000000002e-07 2024-09-14T06:18:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.982 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100466+0000", 2024-09-14T06:18:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.4039999999999997e-06 2024-09-14T06:18:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100475+0000", 2024-09-14T06:18:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.3680000000000008e-06 2024-09-14T06:18:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100482+0000", 2024-09-14T06:18:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8630000000000002e-06 2024-09-14T06:18:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.983 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174669+0000", 2024-09-14T06:18:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074186849999999999 2024-09-14T06:18:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.174758+0000", 2024-09-14T06:18:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9021e-05 2024-09-14T06:18:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175137+0000", 2024-09-14T06:18:14.984 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378511 2024-09-14T06:18:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178158+0000", 2024-09-14T06:18:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.003020956 2024-09-14T06:18:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182386+0000", 2024-09-14T06:18:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042278840000000003 2024-09-14T06:18:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.985 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182410+0000", 2024-09-14T06:18:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4026e-05 2024-09-14T06:18:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182427+0000", 2024-09-14T06:18:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7711999999999999e-05 2024-09-14T06:18:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13248 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:18:14.986 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.778875427000003, 2024-09-14T06:18:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081966527999999997, 2024-09-14T06:18:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13248 2024-09-14T06:18:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:18:14.987 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:18:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100543+0000", 2024-09-14T06:18:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9299999999999997e-07 2024-09-14T06:18:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100548+0000", 2024-09-14T06:18:14.988 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:18:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100557+0000", 2024-09-14T06:18:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.02e-06 2024-09-14T06:18:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100563+0000", 2024-09-14T06:18:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2519999999999996e-06 2024-09-14T06:18:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.989 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175572+0000", 2024-09-14T06:18:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075008459 2024-09-14T06:18:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.175656+0000", 2024-09-14T06:18:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.3906999999999995e-05 2024-09-14T06:18:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176010+0000", 2024-09-14T06:18:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00035447499999999998 2024-09-14T06:18:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.990 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178304+0000", 2024-09-14T06:18:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0022941379999999998 2024-09-14T06:18:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182467+0000", 2024-09-14T06:18:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0041632020000000004 2024-09-14T06:18:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182491+0000", 2024-09-14T06:18:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3733000000000001e-05 2024-09-14T06:18:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.991 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182509+0000", 2024-09-14T06:18:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8113000000000001e-05 2024-09-14T06:18:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13249 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:18:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.778840493000004, 2024-09-14T06:18:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.082037225000000005, 2024-09-14T06:18:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.992 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13249 2024-09-14T06:18:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:18:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:18:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.993 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100578+0000", 2024-09-14T06:18:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4600000000000005e-07 2024-09-14T06:18:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100584+0000", 2024-09-14T06:18:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.947e-06 2024-09-14T06:18:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100593+0000", 2024-09-14T06:18:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.9760000000000001e-06 2024-09-14T06:18:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.994 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100597+0000", 2024-09-14T06:18:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9689999999999996e-06 2024-09-14T06:18:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176461+0000", 2024-09-14T06:18:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075864132000000001 2024-09-14T06:18:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176516+0000", 2024-09-14T06:18:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4416999999999998e-05 2024-09-14T06:18:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.995 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.176734+0000", 2024-09-14T06:18:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021859399999999999 2024-09-14T06:18:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178308+0000", 2024-09-14T06:18:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015733660000000001 2024-09-14T06:18:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182573+0000", 2024-09-14T06:18:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0042649200000000002 2024-09-14T06:18:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.996 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182597+0000", 2024-09-14T06:18:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4355e-05 2024-09-14T06:18:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182615+0000", 2024-09-14T06:18:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8003e-05 2024-09-14T06:18:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.997 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:13250 3.19 3:995e6f38:::rbd_object_map.115223d7013d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:18:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 98.778675887999995, 2024-09-14T06:18:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.081983890000000004, 2024-09-14T06:18:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 13250 2024-09-14T06:18:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.998 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:18:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100742+0000", 2024-09-14T06:18:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100743+0000", 2024-09-14T06:18:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:18:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:14.999 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100753+0000", 2024-09-14T06:18:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9979999999999998e-06 2024-09-14T06:18:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100772+0000", 2024-09-14T06:18:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8664e-05 2024-09-14T06:18:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.100779+0000", 2024-09-14T06:18:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.046e-06 2024-09-14T06:18:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.000 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177049+0000", 2024-09-14T06:18:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076270819000000004 2024-09-14T06:18:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177140+0000", 2024-09-14T06:18:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0746000000000002e-05 2024-09-14T06:18:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 1", 2024-09-14T06:18:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.177519+0000", 2024-09-14T06:18:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378656 2024-09-14T06:18:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.001 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.178349+0000", 2024-09-14T06:18:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00083061599999999995 2024-09-14T06:18:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182679+0000", 2024-09-14T06:18:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0043298379999999999 2024-09-14T06:18:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182705+0000", 2024-09-14T06:18:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5602999999999999e-05 2024-09-14T06:18:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.002 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.182726+0000", 2024-09-14T06:18:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1297e-05 2024-09-14T06:18:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.003 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.003 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:15.005 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:18:15.005 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:15.001+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:15.106 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:18:15.101+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:18:15.130 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:15.130 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:15.130 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:18:15.130 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:15.144 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:18:15.207 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:15.203+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:15.282 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:15.282 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:15.282 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:18:15.282 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:18:15.282 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:15.299 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:18:15.340 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:15.339+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:15.408 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:15.407+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:15.459 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:15.459 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:18:15.459 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:18:15.459 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:18:15.460 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.460 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:30547 3.1f 3:f84dfb53:::rbd_data.11529eb229bd.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write 16226533~50642] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T06:18:15.460 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:18:15.460 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 76.268296946999996, 2024-09-14T06:18:15.460 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0079770020000001, 2024-09-14T06:18:15.460 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.460 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.460 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:15.460 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.460 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.460 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.460 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 30547 2024-09-14T06:18:15.460 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.461 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:18:15.461 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189809+0000", 2024-09-14T06:18:15.461 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.461 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.461 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.461 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189811+0000", 2024-09-14T06:18:15.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2060000000000001e-06 2024-09-14T06:18:15.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.462 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.462 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189871+0000", 2024-09-14T06:18:15.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.9862999999999999e-05 2024-09-14T06:18:15.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.462 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.462 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189899+0000", 2024-09-14T06:18:15.462 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7849000000000001e-05 2024-09-14T06:18:15.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.462 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.462 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.462 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.189915+0000", 2024-09-14T06:18:15.463 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:18:15.463 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.463 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.463 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.463 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190024+0000", 2024-09-14T06:18:15.463 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000109202 2024-09-14T06:18:15.463 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.463 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.463 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:18:15.463 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:59.190041+0000", 2024-09-14T06:18:15.463 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7665000000000001e-05 2024-09-14T06:18:15.463 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.463 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.463 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.463 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188512+0000", 2024-09-14T06:18:15.464 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.99847100700000002 2024-09-14T06:18:15.464 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.464 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.464 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for missing object", 2024-09-14T06:18:15.464 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.188646+0000", 2024-09-14T06:18:15.464 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000133986 2024-09-14T06:18:15.464 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.464 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.464 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.464 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193097+0000", 2024-09-14T06:18:15.464 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0044507059999999996 2024-09-14T06:18:15.464 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.464 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.464 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.464 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193183+0000", 2024-09-14T06:18:15.464 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5496999999999998e-05 2024-09-14T06:18:15.465 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.465 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.465 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:18:15.465 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.193307+0000", 2024-09-14T06:18:15.465 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000124611 2024-09-14T06:18:15.465 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.465 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.465 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:15.465 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.194318+0000", 2024-09-14T06:18:15.465 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0010106 2024-09-14T06:18:15.465 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.465 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.465 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:15.465 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197670+0000", 2024-09-14T06:18:15.465 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0033523170000000001 2024-09-14T06:18:15.466 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.466 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.466 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:15.466 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197712+0000", 2024-09-14T06:18:15.466 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1681000000000003e-05 2024-09-14T06:18:15.466 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.466 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.466 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.466 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:00.197786+0000", 2024-09-14T06:18:15.466 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4148999999999998e-05 2024-09-14T06:18:15.466 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.466 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.467 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.467 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.467 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.467 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32022 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:15.467 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:18:15.467 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 71.589916736000006, 2024-09-14T06:18:15.467 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095598104000000003, 2024-09-14T06:18:15.467 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.467 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.467 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.467 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.467 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.467 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.467 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32022 2024-09-14T06:18:15.467 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.467 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.468 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.468 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.468 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:18:15.468 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.468 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.468 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.468 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.468 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868189+0000", 2024-09-14T06:18:15.468 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.468 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.468 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.468 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.468 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868190+0000", 2024-09-14T06:18:15.468 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6000000000000003e-07 2024-09-14T06:18:15.469 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.469 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.469 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.469 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868243+0000", 2024-09-14T06:18:15.469 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.3443999999999999e-05 2024-09-14T06:18:15.469 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.469 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.469 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.469 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868260+0000", 2024-09-14T06:18:15.469 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6799e-05 2024-09-14T06:18:15.469 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.469 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.469 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.469 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868271+0000", 2024-09-14T06:18:15.469 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0523000000000001e-05 2024-09-14T06:18:15.469 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.470 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.470 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.470 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963502+0000", 2024-09-14T06:18:15.470 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095231466000000001 2024-09-14T06:18:15.470 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.470 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.470 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.470 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963636+0000", 2024-09-14T06:18:15.470 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013388699999999999 2024-09-14T06:18:15.470 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.470 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.470 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.470 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963787+0000", 2024-09-14T06:18:15.470 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000151225 2024-09-14T06:18:15.470 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.471 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.471 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.471 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.471 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.471 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32023 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5816320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:15.471 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:18:15.471 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 71.589821610000001, 2024-09-14T06:18:15.471 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095879753999999998, 2024-09-14T06:18:15.471 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.472 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.472 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.472 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.472 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.472 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.472 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32023 2024-09-14T06:18:15.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.472 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.472 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.472 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.472 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:18:15.472 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.472 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.473 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.473 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868284+0000", 2024-09-14T06:18:15.473 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.473 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.473 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.473 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.473 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868285+0000", 2024-09-14T06:18:15.473 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6299999999999993e-07 2024-09-14T06:18:15.473 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.473 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.473 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.473 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868288+0000", 2024-09-14T06:18:15.473 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.18e-06 2024-09-14T06:18:15.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.474 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.474 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.474 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868298+0000", 2024-09-14T06:18:15.474 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0029e-05 2024-09-14T06:18:15.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.474 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.474 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.474 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868307+0000", 2024-09-14T06:18:15.474 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7229999999999993e-06 2024-09-14T06:18:15.474 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.474 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.474 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.474 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.963871+0000", 2024-09-14T06:18:15.475 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095563876000000006 2024-09-14T06:18:15.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.475 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.475 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.475 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964015+0000", 2024-09-14T06:18:15.475 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143573 2024-09-14T06:18:15.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.475 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.475 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.475 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964164+0000", 2024-09-14T06:18:15.475 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014940999999999999 2024-09-14T06:18:15.475 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.475 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.475 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.476 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.476 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32024 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:15.476 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:18:15.476 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 71.589788466000002, 2024-09-14T06:18:15.476 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096230231999999999, 2024-09-14T06:18:15.476 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.476 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.476 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.476 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.476 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.476 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.476 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32024 2024-09-14T06:18:15.476 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.476 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:18:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868317+0000", 2024-09-14T06:18:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868318+0000", 2024-09-14T06:18:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1699999999999997e-07 2024-09-14T06:18:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.478 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.478 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868321+0000", 2024-09-14T06:18:15.478 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6769999999999999e-06 2024-09-14T06:18:15.478 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.478 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.478 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.478 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868334+0000", 2024-09-14T06:18:15.478 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2928999999999999e-05 2024-09-14T06:18:15.478 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.478 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.478 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.478 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868341+0000", 2024-09-14T06:18:15.478 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9800000000000001e-06 2024-09-14T06:18:15.478 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.479 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.479 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.479 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964258+0000", 2024-09-14T06:18:15.479 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095917099000000006 2024-09-14T06:18:15.479 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.479 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.479 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.479 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964392+0000", 2024-09-14T06:18:15.479 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013396999999999999 2024-09-14T06:18:15.479 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.479 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.479 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.479 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964548+0000", 2024-09-14T06:18:15.479 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015585999999999999 2024-09-14T06:18:15.479 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.479 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.480 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.480 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.480 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.480 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32025 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:15.480 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:18:15.480 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 71.58975581, 2024-09-14T06:18:15.480 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096576591000000003, 2024-09-14T06:18:15.480 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.480 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.480 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.480 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.480 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.480 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.480 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32025 2024-09-14T06:18:15.480 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.481 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.481 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.481 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.481 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:18:15.481 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.481 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.481 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.481 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.481 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868350+0000", 2024-09-14T06:18:15.481 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.481 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.481 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.481 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.481 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868351+0000", 2024-09-14T06:18:15.481 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.1600000000000001e-07 2024-09-14T06:18:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868353+0000", 2024-09-14T06:18:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.627e-06 2024-09-14T06:18:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868364+0000", 2024-09-14T06:18:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0893000000000001e-05 2024-09-14T06:18:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868381+0000", 2024-09-14T06:18:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6769000000000002e-05 2024-09-14T06:18:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.483 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.483 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.483 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964632+0000", 2024-09-14T06:18:15.483 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096250589999999997 2024-09-14T06:18:15.483 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.483 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.483 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.483 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964772+0000", 2024-09-14T06:18:15.483 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014045299999999999 2024-09-14T06:18:15.483 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.483 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.483 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.483 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.964927+0000", 2024-09-14T06:18:15.484 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015454299999999999 2024-09-14T06:18:15.484 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.484 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.484 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.484 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.484 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.484 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32026 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:15.484 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:18:15.484 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 71.589714858999997, 2024-09-14T06:18:15.484 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096901622000000007, 2024-09-14T06:18:15.484 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.484 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.484 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.484 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.484 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32026 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868391+0000", 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.485 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868392+0000", 2024-09-14T06:18:15.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.0699999999999996e-07 2024-09-14T06:18:15.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868395+0000", 2024-09-14T06:18:15.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9859999999999999e-06 2024-09-14T06:18:15.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868405+0000", 2024-09-14T06:18:15.486 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9229999999999997e-06 2024-09-14T06:18:15.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.486 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.486 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.486 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868412+0000", 2024-09-14T06:18:15.487 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9670000000000004e-06 2024-09-14T06:18:15.487 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.487 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.487 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.487 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965010+0000", 2024-09-14T06:18:15.487 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096598562999999998 2024-09-14T06:18:15.487 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.487 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.487 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.487 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965143+0000", 2024-09-14T06:18:15.487 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013247700000000001 2024-09-14T06:18:15.487 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.487 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.487 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.487 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965293+0000", 2024-09-14T06:18:15.488 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000149899 2024-09-14T06:18:15.488 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.488 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.488 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.488 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.488 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.488 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32027 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:15.488 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:18:15.488 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 71.589646521999995, 2024-09-14T06:18:15.488 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097211306999999997, 2024-09-14T06:18:15.488 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.488 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.488 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.488 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.488 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.488 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.489 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32027 2024-09-14T06:18:15.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.489 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.489 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.489 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.489 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:18:15.489 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.489 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.489 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.489 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868459+0000", 2024-09-14T06:18:15.489 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.490 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.490 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868461+0000", 2024-09-14T06:18:15.490 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.296e-06 2024-09-14T06:18:15.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.490 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.490 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868465+0000", 2024-09-14T06:18:15.490 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.9570000000000004e-06 2024-09-14T06:18:15.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.490 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.490 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868476+0000", 2024-09-14T06:18:15.490 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1365999999999999e-05 2024-09-14T06:18:15.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.491 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.491 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.491 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868483+0000", 2024-09-14T06:18:15.491 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2529999999999998e-06 2024-09-14T06:18:15.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.491 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.491 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.491 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965375+0000", 2024-09-14T06:18:15.491 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096891973000000006 2024-09-14T06:18:15.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.491 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.491 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.491 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965518+0000", 2024-09-14T06:18:15.491 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000143017 2024-09-14T06:18:15.491 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.492 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.492 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.492 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965671+0000", 2024-09-14T06:18:15.492 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015244499999999999 2024-09-14T06:18:15.492 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.492 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.492 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.492 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.492 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.492 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32028 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:15.492 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:18:15.492 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 71.589611693999998, 2024-09-14T06:18:15.492 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097841480999999994, 2024-09-14T06:18:15.492 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.492 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.493 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.493 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.493 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.493 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.493 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32028 2024-09-14T06:18:15.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.493 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.493 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.493 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.493 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:18:15.493 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.493 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.493 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.493 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868494+0000", 2024-09-14T06:18:15.493 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.494 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.494 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.494 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.494 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868495+0000", 2024-09-14T06:18:15.494 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6400000000000001e-07 2024-09-14T06:18:15.494 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.494 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.494 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.494 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868498+0000", 2024-09-14T06:18:15.494 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7760000000000002e-06 2024-09-14T06:18:15.494 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.494 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.494 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.494 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868507+0000", 2024-09-14T06:18:15.494 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1530000000000003e-06 2024-09-14T06:18:15.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868516+0000", 2024-09-14T06:18:15.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.8929999999999994e-06 2024-09-14T06:18:15.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965753+0000", 2024-09-14T06:18:15.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097236970000000006 2024-09-14T06:18:15.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.495 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.495 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.495 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.965890+0000", 2024-09-14T06:18:15.495 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013677599999999999 2024-09-14T06:18:15.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.496 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.496 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966336+0000", 2024-09-14T06:18:15.496 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00044614900000000001 2024-09-14T06:18:15.496 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.496 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.496 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.496 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.496 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32029 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:15.496 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:18:15.496 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 71.589580619000003, 2024-09-14T06:18:15.496 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098577779000000004, 2024-09-14T06:18:15.496 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.496 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.496 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.497 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.497 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.497 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.497 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32029 2024-09-14T06:18:15.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.497 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.497 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.497 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:18:15.497 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.497 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.497 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.497 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868525+0000", 2024-09-14T06:18:15.497 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.498 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868526+0000", 2024-09-14T06:18:15.498 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9300000000000006e-07 2024-09-14T06:18:15.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.498 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868529+0000", 2024-09-14T06:18:15.498 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.5969999999999999e-06 2024-09-14T06:18:15.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.498 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.498 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.498 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868538+0000", 2024-09-14T06:18:15.499 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.1700000000000003e-06 2024-09-14T06:18:15.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.499 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.499 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868560+0000", 2024-09-14T06:18:15.499 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1596000000000001e-05 2024-09-14T06:18:15.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.499 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.499 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966423+0000", 2024-09-14T06:18:15.499 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.097863396000000005 2024-09-14T06:18:15.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.499 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.499 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.499 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.966578+0000", 2024-09-14T06:18:15.500 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000154792 2024-09-14T06:18:15.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.500 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.500 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967103+0000", 2024-09-14T06:18:15.500 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00052523499999999998 2024-09-14T06:18:15.500 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.500 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.500 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.500 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.500 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32030 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:15.500 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:18:15.500 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 71.589535701000003, 2024-09-14T06:18:15.501 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099324030999999993, 2024-09-14T06:18:15.501 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.501 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.501 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.501 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.501 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.501 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.501 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32030 2024-09-14T06:18:15.501 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.501 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.501 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.501 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.501 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:18:15.501 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868570+0000", 2024-09-14T06:18:15.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868571+0000", 2024-09-14T06:18:15.502 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.9699999999999995e-07 2024-09-14T06:18:15.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.502 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.502 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.502 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868574+0000", 2024-09-14T06:18:15.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9529999999999999e-06 2024-09-14T06:18:15.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.503 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.503 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868584+0000", 2024-09-14T06:18:15.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7860000000000008e-06 2024-09-14T06:18:15.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.503 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.503 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868591+0000", 2024-09-14T06:18:15.503 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0500000000000003e-06 2024-09-14T06:18:15.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.503 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.503 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.504 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967191+0000", 2024-09-14T06:18:15.504 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.098600035000000003 2024-09-14T06:18:15.504 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.504 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.504 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.504 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967333+0000", 2024-09-14T06:18:15.504 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00014190599999999999 2024-09-14T06:18:15.504 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.504 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.504 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.504 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967894+0000", 2024-09-14T06:18:15.505 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000561504 2024-09-14T06:18:15.505 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.505 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.505 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.505 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.505 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.505 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32031 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:15.505 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:18:15.505 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 71.589506138999994, 2024-09-14T06:18:15.505 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.10010279, 2024-09-14T06:18:15.505 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.505 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.505 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.505 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.505 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.506 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.506 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32031 2024-09-14T06:18:15.506 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.506 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.506 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.506 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.506 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:18:15.506 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.506 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.506 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.506 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.506 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:18:15.506 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.506 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.507 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.507 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.507 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868600+0000", 2024-09-14T06:18:15.507 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.3399999999999998e-07 2024-09-14T06:18:15.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.507 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868603+0000", 2024-09-14T06:18:15.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.599e-06 2024-09-14T06:18:15.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868612+0000", 2024-09-14T06:18:15.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.986e-06 2024-09-14T06:18:15.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.508 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.508 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.508 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868619+0000", 2024-09-14T06:18:15.508 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8900000000000001e-06 2024-09-14T06:18:15.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.967983+0000", 2024-09-14T06:18:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.099363612000000004 2024-09-14T06:18:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968140+0000", 2024-09-14T06:18:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00015788600000000001 2024-09-14T06:18:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968703+0000", 2024-09-14T06:18:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00056208300000000001 2024-09-14T06:18:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.509 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.510 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.510 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.510 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32032 3.1e 3:7c827c05:::rbd_data.11529eb229bd.0000000000000000:13 [read 6258688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:15.510 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:18:15.510 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 71.589477664, 2024-09-14T06:18:15.510 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100488088, 2024-09-14T06:18:15.510 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.510 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.510 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.510 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.510 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.510 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.510 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32032 2024-09-14T06:18:15.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.511 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.511 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.511 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:18:15.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.511 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.511 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868628+0000", 2024-09-14T06:18:15.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.511 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.511 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.511 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868629+0000", 2024-09-14T06:18:15.511 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.8299999999999996e-07 2024-09-14T06:18:15.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.512 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.512 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868631+0000", 2024-09-14T06:18:15.512 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.4099999999999998e-06 2024-09-14T06:18:15.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.512 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.512 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868640+0000", 2024-09-14T06:18:15.512 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0939999999999997e-06 2024-09-14T06:18:15.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.512 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.512 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.512 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.868647+0000", 2024-09-14T06:18:15.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.9460000000000001e-06 2024-09-14T06:18:15.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.513 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.513 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968797+0000", 2024-09-14T06:18:15.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.100149867 2024-09-14T06:18:15.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.513 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.513 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.513 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.968937+0000", 2024-09-14T06:18:15.513 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013981399999999999 2024-09-14T06:18:15.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.514 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.514 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969116+0000", 2024-09-14T06:18:15.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00017927399999999999 2024-09-14T06:18:15.514 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.514 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.514 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.514 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:32044 3.1e 3:7e6619d4:::rbd_data.115274ed0aa8.0000000000000068:head [set-alloc-hint object_size 1048576 write_size 1048576,write 835584~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T06:18:15.514 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:18:15.514 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 71.581788580999998, 2024-09-14T06:18:15.514 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096039367, 2024-09-14T06:18:15.514 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.514 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.514 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:18:15.515 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.515 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.515 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.515 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 32044 2024-09-14T06:18:15.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.515 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:18:15.515 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.515 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.515 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.515 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.515 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876317+0000", 2024-09-14T06:18:15.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.516 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.516 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876320+0000", 2024-09-14T06:18:15.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8270000000000002e-06 2024-09-14T06:18:15.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.516 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.516 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876397+0000", 2024-09-14T06:18:15.516 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6985999999999998e-05 2024-09-14T06:18:15.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.516 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.516 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.517 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876431+0000", 2024-09-14T06:18:15.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.3719e-05 2024-09-14T06:18:15.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.517 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.517 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.517 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.876452+0000", 2024-09-14T06:18:15.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0891999999999999e-05 2024-09-14T06:18:15.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.517 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.517 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.517 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969186+0000", 2024-09-14T06:18:15.517 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.092734681999999999 2024-09-14T06:18:15.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.517 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.518 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.518 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969550+0000", 2024-09-14T06:18:15.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00036402199999999998 2024-09-14T06:18:15.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.518 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 2", 2024-09-14T06:18:15.518 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.969987+0000", 2024-09-14T06:18:15.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000436759 2024-09-14T06:18:15.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.518 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:18:15.518 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971569+0000", 2024-09-14T06:18:15.518 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.001581821 2024-09-14T06:18:15.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.518 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:15.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.971947+0000", 2024-09-14T06:18:15.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000378446 2024-09-14T06:18:15.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "commit_sent", 2024-09-14T06:18:15.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972146+0000", 2024-09-14T06:18:15.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019819 2024-09-14T06:18:15.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.519 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.519 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.519 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:03.972357+0000", 2024-09-14T06:18:15.519 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00021102299999999999 2024-09-14T06:18:15.520 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.520 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.520 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.520 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.520 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.520 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40252 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 827392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:15.520 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:18:15.520 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.741152475000007, 2024-09-14T06:18:15.520 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095633474999999996, 2024-09-14T06:18:15.520 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.520 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.520 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.520 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.520 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.521 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.521 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40252 2024-09-14T06:18:15.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.521 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:18:15.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.521 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.521 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.716953+0000", 2024-09-14T06:18:15.521 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717376+0000", 2024-09-14T06:18:15.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000422216 2024-09-14T06:18:15.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717386+0000", 2024-09-14T06:18:15.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0396e-05 2024-09-14T06:18:15.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.522 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.522 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717395+0000", 2024-09-14T06:18:15.522 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7819999999999999e-06 2024-09-14T06:18:15.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717400+0000", 2024-09-14T06:18:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1599999999999997e-06 2024-09-14T06:18:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.811922+0000", 2024-09-14T06:18:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.094522018999999999 2024-09-14T06:18:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812007+0000", 2024-09-14T06:18:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.5328999999999996e-05 2024-09-14T06:18:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812587+0000", 2024-09-14T06:18:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000579573 2024-09-14T06:18:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40254 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:18:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.740694301999994, 2024-09-14T06:18:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095562731999999997, 2024-09-14T06:18:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40254 2024-09-14T06:18:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:18:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:18:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717412+0000", 2024-09-14T06:18:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.1699999999999999e-07 2024-09-14T06:18:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717414+0000", 2024-09-14T06:18:15.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:18:15.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.527 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717419+0000", 2024-09-14T06:18:15.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.2839999999999997e-06 2024-09-14T06:18:15.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.527 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.527 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717424+0000", 2024-09-14T06:18:15.527 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1549999999999998e-06 2024-09-14T06:18:15.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812619+0000", 2024-09-14T06:18:15.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095194940000000006 2024-09-14T06:18:15.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.528 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.528 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812717+0000", 2024-09-14T06:18:15.528 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.7647999999999997e-05 2024-09-14T06:18:15.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.529 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.529 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.812974+0000", 2024-09-14T06:18:15.529 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025752099999999998 2024-09-14T06:18:15.529 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.529 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.529 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.529 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40256 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 843776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:15.530 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:18:15.530 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.740671698, 2024-09-14T06:18:15.530 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095932066999999996, 2024-09-14T06:18:15.530 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.530 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.530 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.530 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.530 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.530 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.530 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40256 2024-09-14T06:18:15.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.530 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.530 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:18:15.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717434+0000", 2024-09-14T06:18:15.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.531 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.531 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717628+0000", 2024-09-14T06:18:15.531 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00019339499999999999 2024-09-14T06:18:15.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717632+0000", 2024-09-14T06:18:15.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.2230000000000001e-06 2024-09-14T06:18:15.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717641+0000", 2024-09-14T06:18:15.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6309999999999998e-06 2024-09-14T06:18:15.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.532 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.532 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717656+0000", 2024-09-14T06:18:15.532 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4724e-05 2024-09-14T06:18:15.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.533 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.533 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.533 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813006+0000", 2024-09-14T06:18:15.533 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095349801999999997 2024-09-14T06:18:15.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.533 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.533 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.533 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813107+0000", 2024-09-14T06:18:15.533 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000100973 2024-09-14T06:18:15.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.533 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.533 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.534 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813366+0000", 2024-09-14T06:18:15.534 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025931899999999999 2024-09-14T06:18:15.534 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.534 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.534 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.534 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40258 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 851968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:15.534 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:18:15.534 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.740435598000005, 2024-09-14T06:18:15.534 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096062543, 2024-09-14T06:18:15.534 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.534 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.534 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.534 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.535 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.535 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.535 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40258 2024-09-14T06:18:15.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.535 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.535 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:18:15.535 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.535 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717670+0000", 2024-09-14T06:18:15.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717671+0000", 2024-09-14T06:18:15.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8299999999999997e-07 2024-09-14T06:18:15.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.536 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.536 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717674+0000", 2024-09-14T06:18:15.536 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.7080000000000002e-06 2024-09-14T06:18:15.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717681+0000", 2024-09-14T06:18:15.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.7889999999999995e-06 2024-09-14T06:18:15.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717687+0000", 2024-09-14T06:18:15.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7529999999999998e-06 2024-09-14T06:18:15.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.537 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.537 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813408+0000", 2024-09-14T06:18:15.537 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095720451999999998 2024-09-14T06:18:15.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.538 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.538 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813499+0000", 2024-09-14T06:18:15.538 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0933000000000005e-05 2024-09-14T06:18:15.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.538 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.538 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813733+0000", 2024-09-14T06:18:15.538 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00023432499999999999 2024-09-14T06:18:15.538 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.538 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.538 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.538 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40260 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 860160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:15.539 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:18:15.539 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.740409748999994, 2024-09-14T06:18:15.539 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096337381999999999, 2024-09-14T06:18:15.539 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.539 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.539 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.539 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.539 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.540 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.540 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40260 2024-09-14T06:18:15.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.540 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.540 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.540 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.541 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:18:15.541 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.541 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.541 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.541 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.717696+0000", 2024-09-14T06:18:15.542 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.542 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.542 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.542 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.542 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718008+0000", 2024-09-14T06:18:15.542 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00031192699999999998 2024-09-14T06:18:15.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.543 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718013+0000", 2024-09-14T06:18:15.543 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.1830000000000002e-06 2024-09-14T06:18:15.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.543 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718036+0000", 2024-09-14T06:18:15.544 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2721000000000002e-05 2024-09-14T06:18:15.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.544 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.544 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.544 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718053+0000", 2024-09-14T06:18:15.545 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6736e-05 2024-09-14T06:18:15.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.545 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.545 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.545 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813767+0000", 2024-09-14T06:18:15.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095714658999999994 2024-09-14T06:18:15.546 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.546 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.546 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.546 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.813816+0000", 2024-09-14T06:18:15.546 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.8848999999999999e-05 2024-09-14T06:18:15.547 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.547 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.547 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.547 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814034+0000", 2024-09-14T06:18:15.547 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000217307 2024-09-14T06:18:15.547 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.547 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.548 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.548 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.548 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40262 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 868352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:15.548 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:18:15.548 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.740010065999996, 2024-09-14T06:18:15.548 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096354165000000006, 2024-09-14T06:18:15.548 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.548 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.548 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.548 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.548 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.549 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.549 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40262 2024-09-14T06:18:15.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.549 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:18:15.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.549 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.549 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.549 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718096+0000", 2024-09-14T06:18:15.549 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718281+0000", 2024-09-14T06:18:15.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000185069 2024-09-14T06:18:15.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718286+0000", 2024-09-14T06:18:15.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.118e-06 2024-09-14T06:18:15.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.550 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.550 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.550 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718295+0000", 2024-09-14T06:18:15.550 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0429999999999996e-06 2024-09-14T06:18:15.550 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.551 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.551 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.551 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718302+0000", 2024-09-14T06:18:15.551 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2790000000000001e-06 2024-09-14T06:18:15.551 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.551 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.551 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.551 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814076+0000", 2024-09-14T06:18:15.551 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095774076999999999 2024-09-14T06:18:15.551 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.551 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.551 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.551 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814191+0000", 2024-09-14T06:18:15.551 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00011500700000000001 2024-09-14T06:18:15.552 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.552 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.552 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.552 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814450+0000", 2024-09-14T06:18:15.552 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00025857199999999999 2024-09-14T06:18:15.552 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.552 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.552 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.552 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.552 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.552 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:40264 3.18 3:1af67dea:::rbd_data.115274ed0aa8.0000000000000009:14 [read 876544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T06:18:15.552 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:18:15.552 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 65.739652997999997, 2024-09-14T06:18:15.552 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.096372478999999997, 2024-09-14T06:18:15.553 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:15.553 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:15.553 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "started", 2024-09-14T06:18:15.553 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:15.553 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:15.553 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:15.553 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 40264 2024-09-14T06:18:15.553 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.553 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:15.553 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.553 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:15.553 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:18:15.553 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:15.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718453+0000", 2024-09-14T06:18:15.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:15.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:15.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718793+0000", 2024-09-14T06:18:15.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00033991099999999998 2024-09-14T06:18:15.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.554 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.554 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:15.554 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718805+0000", 2024-09-14T06:18:15.554 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2514000000000001e-05 2024-09-14T06:18:15.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.555 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.555 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:15.555 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718815+0000", 2024-09-14T06:18:15.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9590000000000007e-06 2024-09-14T06:18:15.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.555 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.555 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:15.555 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.718822+0000", 2024-09-14T06:18:15.555 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.2030000000000003e-06 2024-09-14T06:18:15.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.555 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.555 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:15.555 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814490+0000", 2024-09-14T06:18:15.556 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.095667346 2024-09-14T06:18:15.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.556 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:15.556 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814577+0000", 2024-09-14T06:18:15.556 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.6920000000000001e-05 2024-09-14T06:18:15.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:15.556 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:15.556 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:15.556 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:17:09.814825+0000", 2024-09-14T06:18:15.556 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000248626 2024-09-14T06:18:15.556 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.556 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.556 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.557 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:15.557 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:15.557 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:15.558 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:18:15.559 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:15.555+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:15.659 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:18:15.653+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:18:15.726 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:15.726 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:15.726 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:18:15.727 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:15.742 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:18:15.760 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:15.757+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:15.861 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-09-14T06:18:15.859+0000 7f318885d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26530) UID: 0 2024-09-14T06:18:15.865 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:15.865 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:15.865 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:18:15.865 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:18:15.865 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:15.880 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:18:15.961 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:15.957+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:16.049 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:16.049 INFO:teuthology.orchestra.run.smithi087.stdout: "size": 20, 2024-09-14T06:18:16.049 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 600, 2024-09-14T06:18:16.050 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [ 2024-09-14T06:18:16.050 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.050 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:3595 3.1 3:811c1c24:::rbd_data.1152bd92e04f.000000000000006e:head [set-alloc-hint object_size 2097152 write_size 2097152,write 440183~2185] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T06:18:16.050 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:18:16.050 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 109.825303926, 2024-09-14T06:18:16.050 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0365806689999999, 2024-09-14T06:18:16.050 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.050 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.050 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.050 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.050 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.050 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.051 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 3595 2024-09-14T06:18:16.051 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.051 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.051 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.051 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.051 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:18:16.051 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.051 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.051 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.051 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.051 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222820+0000", 2024-09-14T06:18:16.051 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.051 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.052 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.052 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.052 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222823+0000", 2024-09-14T06:18:16.052 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.8219999999999998e-06 2024-09-14T06:18:16.052 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.052 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.052 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.052 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222835+0000", 2024-09-14T06:18:16.052 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2052000000000001e-05 2024-09-14T06:18:16.052 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.052 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.052 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.052 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222859+0000", 2024-09-14T06:18:16.052 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.3694e-05 2024-09-14T06:18:16.053 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.053 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.053 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.053 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.222872+0000", 2024-09-14T06:18:16.053 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3203999999999999e-05 2024-09-14T06:18:16.053 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.053 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.053 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.053 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223068+0000", 2024-09-14T06:18:16.053 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.000195995 2024-09-14T06:18:16.053 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.053 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.054 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "started", 2024-09-14T06:18:16.054 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223312+0000", 2024-09-14T06:18:16.054 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00024424199999999998 2024-09-14T06:18:16.054 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.054 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.054 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for subops from 4", 2024-09-14T06:18:16.054 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.223786+0000", 2024-09-14T06:18:16.054 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00047356600000000003 2024-09-14T06:18:16.054 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.054 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.054 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "op_commit", 2024-09-14T06:18:16.054 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.225374+0000", 2024-09-14T06:18:16.054 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0015875150000000001 2024-09-14T06:18:16.054 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.054 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.054 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.055 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.233122+0000", 2024-09-14T06:18:16.055 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0077480700000000001 2024-09-14T06:18:16.055 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.055 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.055 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for peered", 2024-09-14T06:18:16.055 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:26.233157+0000", 2024-09-14T06:18:16.055 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5113000000000001e-05 2024-09-14T06:18:16.055 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.055 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.055 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.055 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.252998+0000", 2024-09-14T06:18:16.055 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.019841142 2024-09-14T06:18:16.055 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.055 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.056 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "waiting for degraded object", 2024-09-14T06:18:16.056 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.254121+0000", 2024-09-14T06:18:16.056 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0011227979999999999 2024-09-14T06:18:16.056 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.056 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.056 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.056 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.259263+0000", 2024-09-14T06:18:16.056 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.0051426700000000002 2024-09-14T06:18:16.056 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.056 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.056 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.056 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:27.259401+0000", 2024-09-14T06:18:16.056 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00013778600000000001 2024-09-14T06:18:16.056 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.056 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.057 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.057 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.057 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.057 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11421 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.057 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:18:16.057 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.06240874, 2024-09-14T06:18:16.057 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078553170000000005, 2024-09-14T06:18:16.057 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.057 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.057 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.057 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.057 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.057 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.057 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11421 2024-09-14T06:18:16.058 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.058 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.058 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.058 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.058 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:18:16.058 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.058 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.058 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.058 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.058 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985716+0000", 2024-09-14T06:18:16.058 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.058 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.058 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.058 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.059 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985718+0000", 2024-09-14T06:18:16.059 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8840000000000001e-06 2024-09-14T06:18:16.059 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.059 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.059 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.059 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985723+0000", 2024-09-14T06:18:16.059 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8799999999999996e-06 2024-09-14T06:18:16.059 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.059 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.059 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.059 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985742+0000", 2024-09-14T06:18:16.059 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8776000000000001e-05 2024-09-14T06:18:16.059 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.059 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.059 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.060 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985759+0000", 2024-09-14T06:18:16.060 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7119999999999999e-05 2024-09-14T06:18:16.060 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.060 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.060 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.060 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064259+0000", 2024-09-14T06:18:16.060 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078499418000000001 2024-09-14T06:18:16.060 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.060 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.060 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.060 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064269+0000", 2024-09-14T06:18:16.060 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:18:16.060 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.060 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.061 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.061 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.061 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.061 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11422 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.061 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:18:16.061 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.06233877299999, 2024-09-14T06:18:16.061 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078521540000000001, 2024-09-14T06:18:16.061 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.061 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.061 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.061 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.061 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.061 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.061 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11422 2024-09-14T06:18:16.062 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.062 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.062 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.062 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.062 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:18:16.062 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.062 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.062 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.062 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.062 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985786+0000", 2024-09-14T06:18:16.062 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.062 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.062 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.062 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.062 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985787+0000", 2024-09-14T06:18:16.062 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7630000000000001e-06 2024-09-14T06:18:16.063 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.063 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.063 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.063 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985842+0000", 2024-09-14T06:18:16.063 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4555999999999997e-05 2024-09-14T06:18:16.063 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.063 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.063 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.063 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985864+0000", 2024-09-14T06:18:16.063 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.2189999999999999e-05 2024-09-14T06:18:16.063 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.063 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.063 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.063 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985879+0000", 2024-09-14T06:18:16.063 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4382999999999999e-05 2024-09-14T06:18:16.064 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.064 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.064 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.064 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064298+0000", 2024-09-14T06:18:16.064 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078419121999999994 2024-09-14T06:18:16.064 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.064 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.064 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.064 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064307+0000", 2024-09-14T06:18:16.064 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.5259999999999999e-06 2024-09-14T06:18:16.064 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.064 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.064 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.064 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.064 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.065 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11423 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.065 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:18:16.065 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.062221401, 2024-09-14T06:18:16.065 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078437175999999997, 2024-09-14T06:18:16.065 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.065 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.065 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.065 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.065 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.065 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.065 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11423 2024-09-14T06:18:16.065 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.065 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.066 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.066 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.066 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:18:16.066 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.066 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.066 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.066 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.066 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985903+0000", 2024-09-14T06:18:16.066 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.066 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.066 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.066 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.066 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985905+0000", 2024-09-14T06:18:16.066 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8139999999999999e-06 2024-09-14T06:18:16.067 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.067 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.067 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.067 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985911+0000", 2024-09-14T06:18:16.067 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7230000000000001e-06 2024-09-14T06:18:16.067 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.067 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.067 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.067 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985930+0000", 2024-09-14T06:18:16.067 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9360000000000001e-05 2024-09-14T06:18:16.067 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.067 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.067 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.067 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.985944+0000", 2024-09-14T06:18:16.068 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4479e-05 2024-09-14T06:18:16.068 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.068 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.068 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.068 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064332+0000", 2024-09-14T06:18:16.068 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.078387686999999998 2024-09-14T06:18:16.068 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.068 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.068 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.068 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064340+0000", 2024-09-14T06:18:16.068 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1130000000000001e-06 2024-09-14T06:18:16.068 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.068 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.069 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.069 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.069 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.069 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11480 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.069 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:18:16.069 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.060870925, 2024-09-14T06:18:16.069 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077122298000000006, 2024-09-14T06:18:16.069 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.069 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.069 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.069 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.069 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.069 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.070 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11480 2024-09-14T06:18:16.070 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.070 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.070 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.070 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.070 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:18:16.070 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.070 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.070 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.070 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.070 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987253+0000", 2024-09-14T06:18:16.070 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.070 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.071 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.071 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.071 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987255+0000", 2024-09-14T06:18:16.071 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.767e-06 2024-09-14T06:18:16.071 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.071 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.071 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.071 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987263+0000", 2024-09-14T06:18:16.071 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.6499999999999996e-06 2024-09-14T06:18:16.071 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.071 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.071 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.071 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987285+0000", 2024-09-14T06:18:16.071 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.1860000000000001e-05 2024-09-14T06:18:16.072 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.072 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.072 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.072 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987341+0000", 2024-09-14T06:18:16.072 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6648999999999999e-05 2024-09-14T06:18:16.072 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.072 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.072 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.072 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064367+0000", 2024-09-14T06:18:16.072 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077025622000000002 2024-09-14T06:18:16.072 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.072 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.072 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.073 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064376+0000", 2024-09-14T06:18:16.073 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.7499999999999992e-06 2024-09-14T06:18:16.073 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.073 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.073 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.073 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.073 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.073 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11481 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.073 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:18:16.073 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.06073522299999, 2024-09-14T06:18:16.073 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.077022057000000005, 2024-09-14T06:18:16.073 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.073 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.073 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.073 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.074 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.074 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.074 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11481 2024-09-14T06:18:16.074 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.074 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.074 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.074 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.074 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:18:16.074 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.074 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.074 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.074 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.074 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987389+0000", 2024-09-14T06:18:16.074 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.074 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.075 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.075 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.075 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987391+0000", 2024-09-14T06:18:16.075 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4839999999999999e-06 2024-09-14T06:18:16.075 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.075 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.075 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.075 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987399+0000", 2024-09-14T06:18:16.075 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8660000000000006e-06 2024-09-14T06:18:16.075 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.075 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.075 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.075 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987419+0000", 2024-09-14T06:18:16.075 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9984e-05 2024-09-14T06:18:16.075 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.076 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.076 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.076 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987434+0000", 2024-09-14T06:18:16.076 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5396e-05 2024-09-14T06:18:16.076 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.076 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.076 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.076 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064403+0000", 2024-09-14T06:18:16.076 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076969212999999995 2024-09-14T06:18:16.076 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.076 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.076 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.076 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064411+0000", 2024-09-14T06:18:16.076 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 8.1140000000000006e-06 2024-09-14T06:18:16.076 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.077 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.077 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.077 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.077 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.077 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11482 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.077 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:18:16.077 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.06065036299999, 2024-09-14T06:18:16.077 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076971172000000004, 2024-09-14T06:18:16.077 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.077 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.077 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.077 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.077 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.077 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.077 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11482 2024-09-14T06:18:16.078 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.078 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.078 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.078 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.078 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:18:16.078 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.078 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.078 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.078 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.078 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987474+0000", 2024-09-14T06:18:16.078 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.078 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.078 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.078 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.078 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987476+0000", 2024-09-14T06:18:16.079 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.793e-06 2024-09-14T06:18:16.079 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.079 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.079 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.079 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987482+0000", 2024-09-14T06:18:16.079 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.8270000000000003e-06 2024-09-14T06:18:16.079 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.079 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.079 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.079 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987502+0000", 2024-09-14T06:18:16.079 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0656e-05 2024-09-14T06:18:16.079 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.079 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.079 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.079 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987518+0000", 2024-09-14T06:18:16.079 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5662999999999999e-05 2024-09-14T06:18:16.080 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.080 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.080 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.080 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064438+0000", 2024-09-14T06:18:16.080 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076919623000000006 2024-09-14T06:18:16.080 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.080 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.080 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.080 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064445+0000", 2024-09-14T06:18:16.080 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.61e-06 2024-09-14T06:18:16.080 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.080 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.080 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.080 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.081 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.081 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11483 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.081 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:18:16.081 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.060582514, 2024-09-14T06:18:16.081 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076936562999999999, 2024-09-14T06:18:16.081 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.081 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.081 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.081 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.081 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.081 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.081 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11483 2024-09-14T06:18:16.081 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.081 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.081 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.082 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.082 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:18:16.082 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.082 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.082 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.082 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987542+0000", 2024-09-14T06:18:16.082 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.082 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.082 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.082 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987544+0000", 2024-09-14T06:18:16.082 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.654e-06 2024-09-14T06:18:16.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.082 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.083 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.083 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987550+0000", 2024-09-14T06:18:16.083 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0299999999999999e-06 2024-09-14T06:18:16.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.083 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.083 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.083 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987570+0000", 2024-09-14T06:18:16.083 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0080000000000001e-05 2024-09-14T06:18:16.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.083 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.083 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.083 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987619+0000", 2024-09-14T06:18:16.083 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 4.9228999999999999e-05 2024-09-14T06:18:16.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.083 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.084 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.084 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064471+0000", 2024-09-14T06:18:16.084 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076852086999999999 2024-09-14T06:18:16.084 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.084 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.084 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.084 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064478+0000", 2024-09-14T06:18:16.084 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4830000000000002e-06 2024-09-14T06:18:16.084 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.084 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.084 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.084 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.084 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.084 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11485 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.085 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:18:16.085 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.06048402099999, 2024-09-14T06:18:16.085 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076868463999999997, 2024-09-14T06:18:16.085 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.085 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.085 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.085 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.085 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.085 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.085 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11485 2024-09-14T06:18:16.085 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.086 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.086 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.086 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.086 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:18:16.086 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.086 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.086 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.086 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.086 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987640+0000", 2024-09-14T06:18:16.086 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.086 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.086 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.086 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.086 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987642+0000", 2024-09-14T06:18:16.086 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4100000000000001e-06 2024-09-14T06:18:16.087 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.087 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.087 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.087 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987648+0000", 2024-09-14T06:18:16.087 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.2500000000000003e-06 2024-09-14T06:18:16.087 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.087 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.087 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.087 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987683+0000", 2024-09-14T06:18:16.087 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.5193e-05 2024-09-14T06:18:16.087 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.087 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.087 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.087 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987698+0000", 2024-09-14T06:18:16.087 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4893e-05 2024-09-14T06:18:16.088 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.088 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.088 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.088 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064501+0000", 2024-09-14T06:18:16.088 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076803263999999996 2024-09-14T06:18:16.088 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.088 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.088 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.088 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064509+0000", 2024-09-14T06:18:16.088 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4540000000000001e-06 2024-09-14T06:18:16.088 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.088 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.088 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.088 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.088 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.088 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11490 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.089 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:18:16.089 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.06033445200001, 2024-09-14T06:18:16.089 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076755043999999994, 2024-09-14T06:18:16.089 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.089 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.089 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.089 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.089 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.089 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.089 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11490 2024-09-14T06:18:16.089 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.089 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.089 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.089 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.090 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:18:16.090 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.090 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.090 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.090 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.090 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987790+0000", 2024-09-14T06:18:16.090 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.090 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.090 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.090 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.090 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987791+0000", 2024-09-14T06:18:16.090 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:18:16.090 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.090 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.090 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.091 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987797+0000", 2024-09-14T06:18:16.091 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7969999999999997e-06 2024-09-14T06:18:16.091 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.091 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.091 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.091 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987817+0000", 2024-09-14T06:18:16.091 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9746e-05 2024-09-14T06:18:16.091 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.091 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.091 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.091 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987832+0000", 2024-09-14T06:18:16.091 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.452e-05 2024-09-14T06:18:16.091 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.092 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.092 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.092 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064535+0000", 2024-09-14T06:18:16.092 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076703478000000005 2024-09-14T06:18:16.092 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.092 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.092 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.092 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064545+0000", 2024-09-14T06:18:16.092 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.9799999999999993e-06 2024-09-14T06:18:16.092 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.092 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.092 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.092 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.093 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.093 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11492 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.093 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:18:16.093 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.06020758699999, 2024-09-14T06:18:16.093 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076661759999999995, 2024-09-14T06:18:16.093 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.093 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.093 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.093 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.093 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.093 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.093 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11492 2024-09-14T06:18:16.094 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.094 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.094 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.094 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.094 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:18:16.094 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.094 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.094 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.094 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.094 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987917+0000", 2024-09-14T06:18:16.094 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.094 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.094 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.094 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.095 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987918+0000", 2024-09-14T06:18:16.095 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3239999999999999e-06 2024-09-14T06:18:16.095 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.095 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.095 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.095 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987924+0000", 2024-09-14T06:18:16.095 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6930000000000004e-06 2024-09-14T06:18:16.095 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.095 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.095 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.095 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987943+0000", 2024-09-14T06:18:16.095 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9487000000000002e-05 2024-09-14T06:18:16.095 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.095 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.096 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.096 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.987958+0000", 2024-09-14T06:18:16.096 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4453e-05 2024-09-14T06:18:16.096 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.096 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.096 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.096 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064571+0000", 2024-09-14T06:18:16.096 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076612988000000007 2024-09-14T06:18:16.096 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.096 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.096 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.096 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064579+0000", 2024-09-14T06:18:16.096 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8150000000000006e-06 2024-09-14T06:18:16.097 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.097 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.097 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.097 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.097 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.097 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11495 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.097 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:18:16.097 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.060038898, 2024-09-14T06:18:16.097 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076524746000000005, 2024-09-14T06:18:16.097 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.097 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.097 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.097 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.098 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.098 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.098 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11495 2024-09-14T06:18:16.098 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.098 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.098 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.098 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.098 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:18:16.098 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.098 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.098 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.098 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.098 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988086+0000", 2024-09-14T06:18:16.098 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.099 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.099 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.099 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.099 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988087+0000", 2024-09-14T06:18:16.099 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0499999999999999e-06 2024-09-14T06:18:16.099 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.099 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.099 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.099 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988118+0000", 2024-09-14T06:18:16.099 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.1900000000000003e-05 2024-09-14T06:18:16.099 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.099 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.099 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.099 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988138+0000", 2024-09-14T06:18:16.100 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.933e-05 2024-09-14T06:18:16.100 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.100 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.100 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.100 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988152+0000", 2024-09-14T06:18:16.100 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3889e-05 2024-09-14T06:18:16.100 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.100 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.100 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.100 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064603+0000", 2024-09-14T06:18:16.100 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076451082000000004 2024-09-14T06:18:16.100 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.100 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.100 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.101 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064610+0000", 2024-09-14T06:18:16.101 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.4950000000000002e-06 2024-09-14T06:18:16.101 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.101 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.101 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.101 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.101 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.101 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11498 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.101 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:18:16.101 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.059952822, 2024-09-14T06:18:16.101 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076471883000000004, 2024-09-14T06:18:16.101 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.101 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.101 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.102 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.102 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.102 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.102 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11498 2024-09-14T06:18:16.102 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.102 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.102 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.102 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.102 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:18:16.102 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.102 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.102 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.102 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.102 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988172+0000", 2024-09-14T06:18:16.102 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.102 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.103 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.103 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.103 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988173+0000", 2024-09-14T06:18:16.103 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.2300000000000001e-06 2024-09-14T06:18:16.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.103 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.103 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.103 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988178+0000", 2024-09-14T06:18:16.103 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.4229999999999996e-06 2024-09-14T06:18:16.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.103 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.103 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.103 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988235+0000", 2024-09-14T06:18:16.103 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6616999999999997e-05 2024-09-14T06:18:16.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.104 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.104 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.104 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988264+0000", 2024-09-14T06:18:16.104 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.9189000000000001e-05 2024-09-14T06:18:16.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.104 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.104 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.104 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064636+0000", 2024-09-14T06:18:16.104 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076371573999999998 2024-09-14T06:18:16.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.104 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.104 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.104 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064643+0000", 2024-09-14T06:18:16.105 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8499999999999994e-06 2024-09-14T06:18:16.105 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.105 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.105 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.105 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.105 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.105 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11523 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.105 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:18:16.105 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.059232674, 2024-09-14T06:18:16.105 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.076062454000000002, 2024-09-14T06:18:16.105 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.108 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.108 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.108 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.108 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.108 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.108 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11523 2024-09-14T06:18:16.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.108 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.108 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.108 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.108 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:18:16.109 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.109 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.109 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.109 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.109 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988892+0000", 2024-09-14T06:18:16.109 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.109 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.109 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.109 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.109 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988894+0000", 2024-09-14T06:18:16.109 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.843e-06 2024-09-14T06:18:16.109 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.109 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.110 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.110 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988903+0000", 2024-09-14T06:18:16.110 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.0440000000000001e-06 2024-09-14T06:18:16.110 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.110 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.110 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.110 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988929+0000", 2024-09-14T06:18:16.110 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.6679e-05 2024-09-14T06:18:16.110 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.110 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.110 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.110 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.988967+0000", 2024-09-14T06:18:16.110 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.7453000000000002e-05 2024-09-14T06:18:16.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.111 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.111 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.111 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064668+0000", 2024-09-14T06:18:16.111 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075701389999999993 2024-09-14T06:18:16.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.111 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.111 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.111 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064954+0000", 2024-09-14T06:18:16.111 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.00028604500000000002 2024-09-14T06:18:16.111 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.111 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.111 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.112 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.112 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11530 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.112 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:18:16.112 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.05874212400001, 2024-09-14T06:18:16.112 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075620674999999998, 2024-09-14T06:18:16.112 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.112 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.112 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.112 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.112 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.112 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.112 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11530 2024-09-14T06:18:16.112 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.112 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.112 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.113 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.113 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:18:16.113 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.113 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.113 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.113 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989382+0000", 2024-09-14T06:18:16.113 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.113 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.113 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.113 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989384+0000", 2024-09-14T06:18:16.113 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6160000000000001e-06 2024-09-14T06:18:16.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.114 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.114 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989390+0000", 2024-09-14T06:18:16.114 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7039999999999999e-06 2024-09-14T06:18:16.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.114 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.114 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989409+0000", 2024-09-14T06:18:16.114 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.95e-05 2024-09-14T06:18:16.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.114 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.114 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.114 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989429+0000", 2024-09-14T06:18:16.114 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 2.0173000000000002e-05 2024-09-14T06:18:16.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.115 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.115 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.115 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064992+0000", 2024-09-14T06:18:16.115 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075562764000000004 2024-09-14T06:18:16.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.115 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.115 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.115 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065003+0000", 2024-09-14T06:18:16.115 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.0917999999999999e-05 2024-09-14T06:18:16.115 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.116 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.116 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.116 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.116 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11532 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.116 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:18:16.116 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.05867172799999, 2024-09-14T06:18:16.116 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075604656000000006, 2024-09-14T06:18:16.116 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.116 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.116 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.116 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.116 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.116 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.117 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11532 2024-09-14T06:18:16.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.117 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.117 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.117 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.117 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:18:16.117 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.117 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.117 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.117 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989453+0000", 2024-09-14T06:18:16.117 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.117 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.118 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989454+0000", 2024-09-14T06:18:16.118 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.6500000000000001e-06 2024-09-14T06:18:16.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.118 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989460+0000", 2024-09-14T06:18:16.118 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7459999999999997e-06 2024-09-14T06:18:16.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.118 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989480+0000", 2024-09-14T06:18:16.118 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.948e-05 2024-09-14T06:18:16.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.118 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.118 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.119 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989497+0000", 2024-09-14T06:18:16.119 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.7153e-05 2024-09-14T06:18:16.119 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.119 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.119 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.119 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065046+0000", 2024-09-14T06:18:16.119 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075548905999999999 2024-09-14T06:18:16.119 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.119 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.119 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.119 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065057+0000", 2024-09-14T06:18:16.119 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1721e-05 2024-09-14T06:18:16.119 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.120 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.120 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.120 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.120 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.120 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11534 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.120 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:18:16.120 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.058605242, 2024-09-14T06:18:16.120 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075574350999999998, 2024-09-14T06:18:16.120 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.120 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.120 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.120 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.120 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.120 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.120 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11534 2024-09-14T06:18:16.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.121 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.121 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.121 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:18:16.121 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.121 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.121 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989519+0000", 2024-09-14T06:18:16.121 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.121 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.121 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989521+0000", 2024-09-14T06:18:16.122 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.533e-06 2024-09-14T06:18:16.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.122 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.122 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.122 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989527+0000", 2024-09-14T06:18:16.122 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.0700000000000003e-06 2024-09-14T06:18:16.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.122 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.122 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.122 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989546+0000", 2024-09-14T06:18:16.122 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.948e-05 2024-09-14T06:18:16.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.122 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.122 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.123 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989561+0000", 2024-09-14T06:18:16.123 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.4827000000000001e-05 2024-09-14T06:18:16.123 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.123 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.123 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.123 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065086+0000", 2024-09-14T06:18:16.123 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075524542 2024-09-14T06:18:16.123 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.123 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.123 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.123 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065094+0000", 2024-09-14T06:18:16.123 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.8990000000000001e-06 2024-09-14T06:18:16.123 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.124 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.124 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.124 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.124 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.124 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11535 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.124 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:18:16.124 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.058537879, 2024-09-14T06:18:16.124 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075542466000000003, 2024-09-14T06:18:16.124 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.124 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.124 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.124 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.124 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.124 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.125 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11535 2024-09-14T06:18:16.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.125 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.125 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.125 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.125 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:18:16.125 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.125 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.125 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.125 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989587+0000", 2024-09-14T06:18:16.125 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.125 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.126 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.126 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989588+0000", 2024-09-14T06:18:16.126 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.1659999999999999e-06 2024-09-14T06:18:16.126 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.126 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.126 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.126 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989593+0000", 2024-09-14T06:18:16.126 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.6230000000000002e-06 2024-09-14T06:18:16.126 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.126 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.126 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.126 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989613+0000", 2024-09-14T06:18:16.126 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.9604000000000001e-05 2024-09-14T06:18:16.126 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.127 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.127 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.127 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989643+0000", 2024-09-14T06:18:16.127 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 3.0450000000000002e-05 2024-09-14T06:18:16.127 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.127 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.127 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.127 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065119+0000", 2024-09-14T06:18:16.127 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075475930999999996 2024-09-14T06:18:16.127 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.127 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.127 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.127 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065129+0000", 2024-09-14T06:18:16.127 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 9.6919999999999997e-06 2024-09-14T06:18:16.127 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.128 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.128 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.128 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.128 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.128 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11537 3.5 3.9b430165 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.128 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:18:16.128 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.05845995599999, 2024-09-14T06:18:16.128 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075497868999999995, 2024-09-14T06:18:16.128 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.128 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.128 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.128 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.128 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.128 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.129 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11537 2024-09-14T06:18:16.129 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.129 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.129 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.129 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.129 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:18:16.129 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.129 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.129 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.129 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.129 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989664+0000", 2024-09-14T06:18:16.129 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.129 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.129 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.130 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.130 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989666+0000", 2024-09-14T06:18:16.130 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.48e-06 2024-09-14T06:18:16.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.130 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.130 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.130 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989672+0000", 2024-09-14T06:18:16.130 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 5.7960000000000001e-06 2024-09-14T06:18:16.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.130 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.130 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.130 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989690+0000", 2024-09-14T06:18:16.130 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8743e-05 2024-09-14T06:18:16.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.131 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.131 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.131 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989704+0000", 2024-09-14T06:18:16.131 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.3913e-05 2024-09-14T06:18:16.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.131 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.131 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.131 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065155+0000", 2024-09-14T06:18:16.131 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.075450351999999998 2024-09-14T06:18:16.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.131 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.131 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.131 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.065162+0000", 2024-09-14T06:18:16.131 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.5850000000000002e-06 2024-09-14T06:18:16.132 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.132 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.132 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.132 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.132 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.132 INFO:teuthology.orchestra.run.smithi087.stdout: "description": "osd_op(client.4434.0:11543 3.12 3.4d02a8b2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:18:16.132 INFO:teuthology.orchestra.run.smithi087.stdout: "initiated_at": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:18:16.132 INFO:teuthology.orchestra.run.smithi087.stdout: "age": 100.05818239200001, 2024-09-14T06:18:16.132 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074335152000000002, 2024-09-14T06:18:16.132 INFO:teuthology.orchestra.run.smithi087.stdout: "continuous": false, 2024-09-14T06:18:16.132 INFO:teuthology.orchestra.run.smithi087.stdout: "type_data": { 2024-09-14T06:18:16.132 INFO:teuthology.orchestra.run.smithi087.stdout: "flag_point": "reached pg", 2024-09-14T06:18:16.132 INFO:teuthology.orchestra.run.smithi087.stdout: "client_info": { 2024-09-14T06:18:16.132 INFO:teuthology.orchestra.run.smithi087.stdout: "client": "client.4434", 2024-09-14T06:18:16.133 INFO:teuthology.orchestra.run.smithi087.stdout: "client_addr": "172.21.15.39:0/2565080541", 2024-09-14T06:18:16.133 INFO:teuthology.orchestra.run.smithi087.stdout: "tid": 11543 2024-09-14T06:18:16.133 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.133 INFO:teuthology.orchestra.run.smithi087.stdout: "events": [ 2024-09-14T06:18:16.133 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.133 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "initiated", 2024-09-14T06:18:16.133 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:18:16.133 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.133 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.133 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.133 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "header_read", 2024-09-14T06:18:16.133 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989942+0000", 2024-09-14T06:18:16.133 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0 2024-09-14T06:18:16.133 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.134 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.134 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "throttled", 2024-09-14T06:18:16.134 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989944+0000", 2024-09-14T06:18:16.134 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.694e-06 2024-09-14T06:18:16.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.134 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.134 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "all_read", 2024-09-14T06:18:16.134 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989950+0000", 2024-09-14T06:18:16.134 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 6.3389999999999998e-06 2024-09-14T06:18:16.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.134 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.134 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "dispatched", 2024-09-14T06:18:16.134 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989969+0000", 2024-09-14T06:18:16.134 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.8963000000000001e-05 2024-09-14T06:18:16.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.135 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.135 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "queued_for_pg", 2024-09-14T06:18:16.135 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:35.989985+0000", 2024-09-14T06:18:16.135 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 1.5586999999999998e-05 2024-09-14T06:18:16.135 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.135 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.135 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "reached_pg", 2024-09-14T06:18:16.135 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064270+0000", 2024-09-14T06:18:16.135 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 0.074285552000000005 2024-09-14T06:18:16.135 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-14T06:18:16.135 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-14T06:18:16.135 INFO:teuthology.orchestra.run.smithi087.stdout: "event": "done", 2024-09-14T06:18:16.135 INFO:teuthology.orchestra.run.smithi087.stdout: "time": "2024-09-14T06:16:36.064277+0000", 2024-09-14T06:18:16.136 INFO:teuthology.orchestra.run.smithi087.stdout: "duration": 7.0169999999999999e-06 2024-09-14T06:18:16.136 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.136 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.136 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.136 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-14T06:18:16.136 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-14T06:18:16.136 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:16.138 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:18:16.138 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:16.135+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:16.210 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:18:16.238 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:16.235+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:16.276 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:18:16.324 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:16.325 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:16.325 INFO:teuthology.orchestra.run.smithi087.stdout: "num_ops": 0 2024-09-14T06:18:16.325 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:16.339 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-09-14T06:18:16.335+0000 7fc94553f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26525) UID: 0 2024-09-14T06:18:16.340 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:18:16.436 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-09-14T06:18:16.436 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:18:16.439 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:16.435+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:16.453 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-09-14T06:18:16.453 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:18:16.470 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-09-14T06:18:16.470 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:18:16.488 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-09-14T06:18:16.488 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:18:16.506 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-09-14T06:18:16.507 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:18:16.507 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-14T06:18:16.507 INFO:teuthology.orchestra.run.smithi087.stdout: "ops": [], 2024-09-14T06:18:16.507 INFO:teuthology.orchestra.run.smithi087.stdout: "complaint_time": 30, 2024-09-14T06:18:16.507 INFO:teuthology.orchestra.run.smithi087.stdout: "num_blocked_ops": 0 2024-09-14T06:18:16.507 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-14T06:18:16.526 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:18:16.540 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-09-14T06:18:16.540 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:18:16.540 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:16.537+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:16.565 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-09-14T06:18:16.565 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:18:16.583 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-09-14T06:18:16.584 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:18:16.641 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:16.639+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:16.691 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:16.692 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:16.707 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:18:16.752 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-09-14T06:18:16.751+0000 7f8b0ad39640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26532) UID: 0 2024-09-14T06:18:16.839 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:16.839 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:16.842 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:16.837+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:16.854 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:18:16.944 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:16.939+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:17.021 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:17.022 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:17.036 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:18:17.083 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-14T06:18:17.081+0000 7f45396b9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26366) UID: 0 2024-09-14T06:18:17.146 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-09-14T06:18:17.143+0000 7f88f2736640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26527) UID: 0 2024-09-14T06:18:17.164 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:17.164 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:17.179 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:18:17.223 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-09-14T06:18:17.254 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-14T06:18:17.249+0000 7fd9757ab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26368) UID: 0 2024-09-14T06:18:17.348 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-14T06:18:17.345+0000 7fded7af1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26369) UID: 0 2024-09-14T06:18:17.359 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:17.359 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:17.362 INFO:teuthology.orchestra.run.smithi039.stdout:All operations completed A-OK! 2024-09-14T06:18:17.375 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:18:17.410 DEBUG:teuthology.parallel:result is None 2024-09-14T06:18:17.411 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-14T06:18:17.425 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-14T06:18:17.479 INFO:tasks.thrashosds:joining thrashosds 2024-09-14T06:18:17.577 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:17.577 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:17.592 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:18:17.720 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:17.720 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:17.735 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:18:17.856 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:17.856 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:17.870 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:18:17.991 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:17.991 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:18.008 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:18:18.132 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:18.132 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:18.146 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:18:18.261 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:18.261 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:18.275 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:18:18.386 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:18.386 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:18.399 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:18:18.494 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-14T06:18:18.525 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:18.525 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:18.539 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:18:18.630 INFO:teuthology.orchestra.run.smithi039.stdout:mon.a: {} 2024-09-14T06:18:18.631 INFO:teuthology.orchestra.run.smithi039.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T06:18:18.643 INFO:teuthology.orchestra.run.smithi039.stdout:mon.b: {} 2024-09-14T06:18:18.643 INFO:teuthology.orchestra.run.smithi039.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T06:18:18.653 INFO:teuthology.orchestra.run.smithi039.stdout:mon.c: {} 2024-09-14T06:18:18.653 INFO:teuthology.orchestra.run.smithi039.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T06:18:18.670 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T06:18:18.671 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:18.671 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:18.685 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:18:18.802 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:18.802 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:18.816 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:18:18.934 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:18.934 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:18.948 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:18:19.065 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:19.066 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:19.081 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:18:19.199 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T06:18:19.200 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:19.200 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:19.215 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:18:19.223 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-14T06:18:19.254 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:18:19.337 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:19.337 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:19.352 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:18:19.476 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:19.477 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:19.491 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:18:19.613 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:19.613 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:19.627 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:18:19.749 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:19.749 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:19.763 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:18:19.880 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:19.880 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:19.894 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:18:20.010 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:20.011 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:20.025 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:18:20.141 INFO:teuthology.orchestra.run.smithi087.stdout:ERROR: (22) Invalid argument 2024-09-14T06:18:20.141 INFO:teuthology.orchestra.run.smithi087.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:18:20.258 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-14T06:18:20.259 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-09-14T06:18:20.287 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:18:20.289 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T06:18:21.279 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-09-14T06:18:21.292 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T06:18:21.316 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-14T06:18:22.282 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-14T06:18:22.307 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T06:18:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T06:18:23.338 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-14T06:18:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-14T06:18:24.220 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T06:18:25.218 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T06:18:25.246 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-14T06:18:26.377 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-14T06:18:26.403 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T06:18:27.394 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T06:18:27.420 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-14T06:18:28.412 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-14T06:18:28.439 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T06:18:29.438 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T06:18:29.462 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-14T06:18:30.439 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-14T06:18:30.463 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T06:18:31.444 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T06:18:31.474 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-14T06:18:32.457 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-14T06:18:32.488 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T06:18:33.476 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T06:18:33.504 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-14T06:18:34.516 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-14T06:18:34.543 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-14T06:18:34.543 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-14T06:18:34.543 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-14T06:18:34.543 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-14T06:18:34.543 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T06:18:34.543 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T06:18:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:18:34.858 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":139,"fsid":"038563b1-268f-461f-b6a6-4b4b41ad0236","created":"2024-09-14T06:15:49.493035+0000","modified":"2024-09-14T06:18:34.496136+0000","last_up_change":"2024-09-14T06:16:02.014989+0000","last_in_change":"2024-09-14T06:16:26.221200+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:16:02.429225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T06:16:08.129454+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T06:16:18.928556+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":123,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c669ea76-ed19-4c12-9e56-d79d62846f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6809","nonce":1157559201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6811","nonce":1157559201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6815","nonce":1157559201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6813","nonce":1157559201}]},"public_addr":"172.21.15.39:6809/1157559201","cluster_addr":"172.21.15.39:6811/1157559201","heartbeat_back_addr":"172.21.15.39:6815/1157559201","heartbeat_front_addr":"172.21.15.39:6813/1157559201","state":["exists","up"]},{"osd":1,"uuid":"7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6817","nonce":4062089212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6819","nonce":4062089212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6823","nonce":4062089212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6821","nonce":4062089212}]},"public_addr":"172.21.15.39:6817/4062089212","cluster_addr":"172.21.15.39:6819/4062089212","heartbeat_back_addr":"172.21.15.39:6823/4062089212","heartbeat_front_addr":"172.21.15.39:6821/4062089212","state":["exists","up"]},{"osd":2,"uuid":"9bda988f-e615-4545-849b-318c408c4c42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6825","nonce":1724773882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6827","nonce":1724773882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6831","nonce":1724773882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6829","nonce":1724773882}]},"public_addr":"172.21.15.39:6825/1724773882","cluster_addr":"172.21.15.39:6827/1724773882","heartbeat_back_addr":"172.21.15.39:6831/1724773882","heartbeat_front_addr":"172.21.15.39:6829/1724773882","state":["exists","up"]},{"osd":3,"uuid":"8ea6cd69-2ef1-42ea-b339-7394bf8f4a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6801","nonce":3775031293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6803","nonce":3775031293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6807","nonce":3775031293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6805","nonce":3775031293}]},"public_addr":"172.21.15.39:6801/3775031293","cluster_addr":"172.21.15.39:6803/3775031293","heartbeat_back_addr":"172.21.15.39:6807/3775031293","heartbeat_front_addr":"172.21.15.39:6805/3775031293","state":["exists","up"]},{"osd":4,"uuid":"fea45e8a-e8ea-4963-b93f-0651b84c9e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6825","nonce":2632317380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6827","nonce":2632317380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6831","nonce":2632317380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6829","nonce":2632317380}]},"public_addr":"172.21.15.87:6825/2632317380","cluster_addr":"172.21.15.87:6827/2632317380","heartbeat_back_addr":"172.21.15.87:6831/2632317380","heartbeat_front_addr":"172.21.15.87:6829/2632317380","state":["exists","up"]},{"osd":5,"uuid":"b601e852-a180-477f-86fc-6a213fa866f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6817","nonce":3501664149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6819","nonce":3501664149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6823","nonce":3501664149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6821","nonce":3501664149}]},"public_addr":"172.21.15.87:6817/3501664149","cluster_addr":"172.21.15.87:6819/3501664149","heartbeat_back_addr":"172.21.15.87:6823/3501664149","heartbeat_front_addr":"172.21.15.87:6821/3501664149","state":["exists","up"]},{"osd":6,"uuid":"1113afe0-f524-41b1-8e51-eb5164602d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6801","nonce":1395936437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6803","nonce":1395936437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6807","nonce":1395936437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6805","nonce":1395936437}]},"public_addr":"172.21.15.87:6801/1395936437","cluster_addr":"172.21.15.87:6803/1395936437","heartbeat_back_addr":"172.21.15.87:6807/1395936437","heartbeat_front_addr":"172.21.15.87:6805/1395936437","state":["exists","up"]},{"osd":7,"uuid":"c9162545-318b-4591-be64-cc8ace9097fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6809","nonce":711112216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6811","nonce":711112216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6815","nonce":711112216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6813","nonce":711112216}]},"public_addr":"172.21.15.87:6809/711112216","cluster_addr":"172.21.15.87:6811/711112216","heartbeat_back_addr":"172.21.15.87:6815/711112216","heartbeat_front_addr":"172.21.15.87:6813/711112216","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.705575+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.806946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.853831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.690315+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.494526+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.546122+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.173691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.148509+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.0","osds":[4,5]},{"pgid":"3.1c","osds":[4,6]}],"pg_upmap_items":[{"pgid":"3.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:18:34.878 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T06:18:34.878 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T06:18:35.196 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:18:35.196 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":139,"fsid":"038563b1-268f-461f-b6a6-4b4b41ad0236","created":"2024-09-14T06:15:49.493035+0000","modified":"2024-09-14T06:18:34.496136+0000","last_up_change":"2024-09-14T06:16:02.014989+0000","last_in_change":"2024-09-14T06:16:26.221200+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:16:02.429225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T06:16:08.129454+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T06:16:18.928556+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":123,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c669ea76-ed19-4c12-9e56-d79d62846f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6809","nonce":1157559201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6811","nonce":1157559201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6815","nonce":1157559201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6813","nonce":1157559201}]},"public_addr":"172.21.15.39:6809/1157559201","cluster_addr":"172.21.15.39:6811/1157559201","heartbeat_back_addr":"172.21.15.39:6815/1157559201","heartbeat_front_addr":"172.21.15.39:6813/1157559201","state":["exists","up"]},{"osd":1,"uuid":"7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6817","nonce":4062089212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6819","nonce":4062089212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6823","nonce":4062089212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6821","nonce":4062089212}]},"public_addr":"172.21.15.39:6817/4062089212","cluster_addr":"172.21.15.39:6819/4062089212","heartbeat_back_addr":"172.21.15.39:6823/4062089212","heartbeat_front_addr":"172.21.15.39:6821/4062089212","state":["exists","up"]},{"osd":2,"uuid":"9bda988f-e615-4545-849b-318c408c4c42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6825","nonce":1724773882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6827","nonce":1724773882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6831","nonce":1724773882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6829","nonce":1724773882}]},"public_addr":"172.21.15.39:6825/1724773882","cluster_addr":"172.21.15.39:6827/1724773882","heartbeat_back_addr":"172.21.15.39:6831/1724773882","heartbeat_front_addr":"172.21.15.39:6829/1724773882","state":["exists","up"]},{"osd":3,"uuid":"8ea6cd69-2ef1-42ea-b339-7394bf8f4a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6801","nonce":3775031293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6803","nonce":3775031293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6807","nonce":3775031293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6805","nonce":3775031293}]},"public_addr":"172.21.15.39:6801/3775031293","cluster_addr":"172.21.15.39:6803/3775031293","heartbeat_back_addr":"172.21.15.39:6807/3775031293","heartbeat_front_addr":"172.21.15.39:6805/3775031293","state":["exists","up"]},{"osd":4,"uuid":"fea45e8a-e8ea-4963-b93f-0651b84c9e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6825","nonce":2632317380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6827","nonce":2632317380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6831","nonce":2632317380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6829","nonce":2632317380}]},"public_addr":"172.21.15.87:6825/2632317380","cluster_addr":"172.21.15.87:6827/2632317380","heartbeat_back_addr":"172.21.15.87:6831/2632317380","heartbeat_front_addr":"172.21.15.87:6829/2632317380","state":["exists","up"]},{"osd":5,"uuid":"b601e852-a180-477f-86fc-6a213fa866f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6817","nonce":3501664149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6819","nonce":3501664149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6823","nonce":3501664149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6821","nonce":3501664149}]},"public_addr":"172.21.15.87:6817/3501664149","cluster_addr":"172.21.15.87:6819/3501664149","heartbeat_back_addr":"172.21.15.87:6823/3501664149","heartbeat_front_addr":"172.21.15.87:6821/3501664149","state":["exists","up"]},{"osd":6,"uuid":"1113afe0-f524-41b1-8e51-eb5164602d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6801","nonce":1395936437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6803","nonce":1395936437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6807","nonce":1395936437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6805","nonce":1395936437}]},"public_addr":"172.21.15.87:6801/1395936437","cluster_addr":"172.21.15.87:6803/1395936437","heartbeat_back_addr":"172.21.15.87:6807/1395936437","heartbeat_front_addr":"172.21.15.87:6805/1395936437","state":["exists","up"]},{"osd":7,"uuid":"c9162545-318b-4591-be64-cc8ace9097fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6809","nonce":711112216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6811","nonce":711112216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6815","nonce":711112216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6813","nonce":711112216}]},"public_addr":"172.21.15.87:6809/711112216","cluster_addr":"172.21.15.87:6811/711112216","heartbeat_back_addr":"172.21.15.87:6815/711112216","heartbeat_front_addr":"172.21.15.87:6813/711112216","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.705575+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.806946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.853831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.690315+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.494526+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.546122+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.173691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.148509+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.0","osds":[4,5]},{"pgid":"3.1c","osds":[4,6]}],"pg_upmap_items":[{"pgid":"3.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:18:35.217 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T06:18:35.217 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T06:18:35.217 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T06:18:35.218 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T06:18:35.218 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T06:18:35.219 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T06:18:35.219 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-14T06:18:35.220 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-14T06:18:35.484 INFO:teuthology.orchestra.run.smithi039.stdout:60129542178 2024-09-14T06:18:35.485 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T06:18:35.499 INFO:teuthology.orchestra.run.smithi039.stdout:64424509476 2024-09-14T06:18:35.499 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T06:18:35.500 INFO:teuthology.orchestra.run.smithi039.stdout:64424509474 2024-09-14T06:18:35.501 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T06:18:35.503 INFO:teuthology.orchestra.run.smithi039.stdout:64424509475 2024-09-14T06:18:35.504 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T06:18:35.509 INFO:teuthology.orchestra.run.smithi039.stdout:64424509474 2024-09-14T06:18:35.509 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T06:18:35.512 INFO:teuthology.orchestra.run.smithi039.stdout:60129542178 2024-09-14T06:18:35.513 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T06:18:35.519 INFO:teuthology.orchestra.run.smithi039.stdout:60129542179 2024-09-14T06:18:35.520 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T06:18:35.526 INFO:teuthology.orchestra.run.smithi039.stdout:60129542177 2024-09-14T06:18:35.526 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T06:18:36.023 INFO:teuthology.orchestra.run.smithi039.stdout:64424509473 2024-09-14T06:18:36.047 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509474 got 64424509473 for osd.2 2024-09-14T06:18:36.048 INFO:teuthology.orchestra.run.smithi039.stdout:64424509474 2024-09-14T06:18:36.049 INFO:teuthology.orchestra.run.smithi039.stdout:64424509475 2024-09-14T06:18:36.051 INFO:teuthology.orchestra.run.smithi039.stdout:60129542177 2024-09-14T06:18:36.060 INFO:teuthology.orchestra.run.smithi039.stdout:64424509473 2024-09-14T06:18:36.066 INFO:teuthology.orchestra.run.smithi039.stdout:60129542177 2024-09-14T06:18:36.084 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509474 for osd.0 2024-09-14T06:18:36.087 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542178 got 60129542177 for osd.6 2024-09-14T06:18:36.088 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509474 got 64424509473 for osd.3 2024-09-14T06:18:36.089 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509476 got 64424509475 for osd.1 2024-09-14T06:18:36.096 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542178 got 60129542177 for osd.7 2024-09-14T06:18:36.096 INFO:teuthology.orchestra.run.smithi039.stdout:60129542176 2024-09-14T06:18:36.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542177 got 60129542176 for osd.5 2024-09-14T06:18:36.116 INFO:teuthology.orchestra.run.smithi039.stdout:60129542178 2024-09-14T06:18:36.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542179 got 60129542178 for osd.4 2024-09-14T06:18:37.048 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T06:18:37.084 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T06:18:37.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T06:18:37.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T06:18:37.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T06:18:37.096 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T06:18:37.115 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T06:18:37.134 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T06:18:37.578 INFO:teuthology.orchestra.run.smithi039.stdout:64424509474 2024-09-14T06:18:37.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509474 got 64424509474 for osd.2 2024-09-14T06:18:37.602 DEBUG:teuthology.parallel:result is None 2024-09-14T06:18:37.616 INFO:teuthology.orchestra.run.smithi039.stdout:64424509476 2024-09-14T06:18:37.627 INFO:teuthology.orchestra.run.smithi039.stdout:64424509474 2024-09-14T06:18:37.636 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509476 got 64424509476 for osd.1 2024-09-14T06:18:37.636 DEBUG:teuthology.parallel:result is None 2024-09-14T06:18:37.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509474 got 64424509474 for osd.3 2024-09-14T06:18:37.656 DEBUG:teuthology.parallel:result is None 2024-09-14T06:18:37.662 INFO:teuthology.orchestra.run.smithi039.stdout:64424509475 2024-09-14T06:18:37.666 INFO:teuthology.orchestra.run.smithi039.stdout:60129542178 2024-09-14T06:18:37.684 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509475 for osd.0 2024-09-14T06:18:37.684 DEBUG:teuthology.parallel:result is None 2024-09-14T06:18:37.685 INFO:teuthology.orchestra.run.smithi039.stdout:60129542178 2024-09-14T06:18:37.686 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542178 got 60129542178 for osd.6 2024-09-14T06:18:37.686 DEBUG:teuthology.parallel:result is None 2024-09-14T06:18:37.697 INFO:teuthology.orchestra.run.smithi039.stdout:60129542177 2024-09-14T06:18:37.710 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542178 got 60129542178 for osd.7 2024-09-14T06:18:37.710 DEBUG:teuthology.parallel:result is None 2024-09-14T06:18:37.713 INFO:teuthology.orchestra.run.smithi039.stdout:60129542179 2024-09-14T06:18:37.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542177 got 60129542177 for osd.5 2024-09-14T06:18:37.716 DEBUG:teuthology.parallel:result is None 2024-09-14T06:18:37.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542179 got 60129542179 for osd.4 2024-09-14T06:18:37.731 DEBUG:teuthology.parallel:result is None 2024-09-14T06:18:37.731 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T06:18:37.731 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:18:38.063 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:18:38.064 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:18:38.081 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2024-09-14T06:18:36.419497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":214,"num_object_clones":0,"num_object_copies":428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":214,"num_whiteouts":206,"num_read":107322,"num_read_kb":751209,"num_write":17568,"num_write_kb":430901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1270,"num_bytes_recovered":144403222,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15645,"ondisk_log_size":15645,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":347276,"kb_used_data":13072,"kb_used_omap":12,"kb_used_meta":334003,"kb_avail":754627444,"statfs":{"total":773094113280,"available":772738502656,"internally_reserved":0,"allocated":13385728,"data_stored":7485092,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":342019632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.973991"},"pg_stats":[{"pgid":"3.1f","version":"123'604","reported_seq":3721,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740380+0000","last_change":"2024-09-14T06:18:17.225123+0000","last_active":"2024-09-14T06:18:34.740380+0000","last_peered":"2024-09-14T06:18:34.740380+0000","last_clean":"2024-09-14T06:18:34.740380+0000","last_became_active":"2024-09-14T06:17:02.193313+0000","last_became_peered":"2024-09-14T06:17:02.193313+0000","last_unstale":"2024-09-14T06:18:34.740380+0000","last_undegraded":"2024-09-14T06:18:34.740380+0000","last_fullsized":"2024-09-14T06:18:34.740380+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:43:51.113692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002155396,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4617,"num_read_kb":53605,"num_write":714,"num_write_kb":19355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"122'407","reported_seq":4554,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739967+0000","last_change":"2024-09-14T06:18:17.224253+0000","last_active":"2024-09-14T06:18:34.739967+0000","last_peered":"2024-09-14T06:18:34.739967+0000","last_clean":"2024-09-14T06:18:34.739967+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:18:34.739967+0000","last_undegraded":"2024-09-14T06:18:34.739967+0000","last_fullsized":"2024-09-14T06:18:34.739967+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:46:26.694964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019300890000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5022,"num_read_kb":43064,"num_write":350,"num_write_kb":15485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"123'1132","reported_seq":2556,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522291+0000","last_change":"2024-09-14T06:18:17.337001+0000","last_active":"2024-09-14T06:18:34.522291+0000","last_peered":"2024-09-14T06:18:34.522291+0000","last_clean":"2024-09-14T06:18:34.522291+0000","last_became_active":"2024-09-14T06:17:38.226838+0000","last_became_peered":"2024-09-14T06:17:38.226838+0000","last_unstale":"2024-09-14T06:18:34.522291+0000","last_undegraded":"2024-09-14T06:18:34.522291+0000","last_fullsized":"2024-09-14T06:18:34.522291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":1132,"log_dups_size":0,"ondisk_log_size":1132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:01.176538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062318570000000004,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5125,"num_read_kb":14378,"num_write":1577,"num_write_kb":13994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10785266,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"123'934","reported_seq":3133,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521839+0000","last_change":"2024-09-14T06:18:17.228603+0000","last_active":"2024-09-14T06:18:34.521839+0000","last_peered":"2024-09-14T06:18:34.521839+0000","last_clean":"2024-09-14T06:18:34.521839+0000","last_became_active":"2024-09-14T06:17:48.737020+0000","last_became_peered":"2024-09-14T06:17:48.737020+0000","last_unstale":"2024-09-14T06:18:34.521839+0000","last_undegraded":"2024-09-14T06:18:34.521839+0000","last_fullsized":"2024-09-14T06:18:34.521839+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:04:33.297278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024240709999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":6011,"num_read_kb":10015,"num_write":1345,"num_write_kb":6524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":3869232,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"123'349","reported_seq":3834,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521767+0000","last_change":"2024-09-14T06:18:17.228692+0000","last_active":"2024-09-14T06:18:34.521767+0000","last_peered":"2024-09-14T06:18:34.521767+0000","last_clean":"2024-09-14T06:18:34.521767+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:18:34.521767+0000","last_undegraded":"2024-09-14T06:18:34.521767+0000","last_fullsized":"2024-09-14T06:18:34.521767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:40:01.686387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024977549999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3342,"num_read_kb":39056,"num_write":366,"num_write_kb":25723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"122'539","reported_seq":3387,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738796+0000","last_change":"2024-09-14T06:18:17.221967+0000","last_active":"2024-09-14T06:18:34.738796+0000","last_peered":"2024-09-14T06:18:34.738796+0000","last_clean":"2024-09-14T06:18:34.738796+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:18:34.738796+0000","last_undegraded":"2024-09-14T06:18:34.738796+0000","last_fullsized":"2024-09-14T06:18:34.738796+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:05:11.528071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087057599999999997,"stat_sum":{"num_bytes":19,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":4088,"num_read_kb":13128,"num_write":658,"num_write_kb":7511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"122'532","reported_seq":2187,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521641+0000","last_change":"2024-09-14T06:18:17.227798+0000","last_active":"2024-09-14T06:18:34.521641+0000","last_peered":"2024-09-14T06:18:34.521641+0000","last_clean":"2024-09-14T06:18:34.521641+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:18:34.521641+0000","last_undegraded":"2024-09-14T06:18:34.521641+0000","last_fullsized":"2024-09-14T06:18:34.521641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:09:43.212330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001716539,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2883,"num_read_kb":19234,"num_write":704,"num_write_kb":12873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"123'400","reported_seq":3103,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740079+0000","last_change":"2024-09-14T06:18:17.225653+0000","last_active":"2024-09-14T06:18:34.740079+0000","last_peered":"2024-09-14T06:18:34.740079+0000","last_clean":"2024-09-14T06:18:34.740079+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:18:34.740079+0000","last_undegraded":"2024-09-14T06:18:34.740079+0000","last_fullsized":"2024-09-14T06:18:34.740079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:38:39.952754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002524938,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":3795,"num_read_kb":37772,"num_write":360,"num_write_kb":32941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"119'395","reported_seq":1793,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740431+0000","last_change":"2024-09-14T06:18:17.224547+0000","last_active":"2024-09-14T06:18:34.740431+0000","last_peered":"2024-09-14T06:18:34.740431+0000","last_clean":"2024-09-14T06:18:34.740431+0000","last_became_active":"2024-09-14T06:17:37.233410+0000","last_became_peered":"2024-09-14T06:17:37.233410+0000","last_unstale":"2024-09-14T06:18:34.740431+0000","last_undegraded":"2024-09-14T06:18:34.740431+0000","last_fullsized":"2024-09-14T06:18:34.740431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:33.122557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019943040000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1416,"num_read_kb":17477,"num_write":336,"num_write_kb":17193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":23467377,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"122'436","reported_seq":2420,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521785+0000","last_change":"2024-09-14T06:18:25.223056+0000","last_active":"2024-09-14T06:18:34.521785+0000","last_peered":"2024-09-14T06:18:34.521785+0000","last_clean":"2024-09-14T06:18:34.521785+0000","last_became_active":"2024-09-14T06:18:25.219200+0000","last_became_peered":"2024-09-14T06:18:25.219200+0000","last_unstale":"2024-09-14T06:18:34.521785+0000","last_undegraded":"2024-09-14T06:18:34.521785+0000","last_fullsized":"2024-09-14T06:18:34.521785+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:26:30.150015+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.047023011000000003,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2483,"num_read_kb":16488,"num_write":510,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":6785399,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"122'432","reported_seq":3823,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739366+0000","last_change":"2024-09-14T06:18:17.226299+0000","last_active":"2024-09-14T06:18:34.739366+0000","last_peered":"2024-09-14T06:18:34.739366+0000","last_clean":"2024-09-14T06:18:34.739366+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:18:34.739366+0000","last_undegraded":"2024-09-14T06:18:34.739366+0000","last_fullsized":"2024-09-14T06:18:34.739366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:42:58.536316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094053699999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4134,"num_read_kb":64629,"num_write":506,"num_write_kb":22219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"123'308","reported_seq":2112,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522107+0000","last_change":"2024-09-14T06:18:17.333655+0000","last_active":"2024-09-14T06:18:34.522107+0000","last_peered":"2024-09-14T06:18:34.522107+0000","last_clean":"2024-09-14T06:18:34.522107+0000","last_became_active":"2024-09-14T06:17:37.231244+0000","last_became_peered":"2024-09-14T06:17:37.231244+0000","last_unstale":"2024-09-14T06:18:34.522107+0000","last_undegraded":"2024-09-14T06:18:34.522107+0000","last_fullsized":"2024-09-14T06:18:34.522107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:10.356294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044577879999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2916,"num_read_kb":27195,"num_write":273,"num_write_kb":15734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":32506492,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":123,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522222+0000","last_change":"2024-09-14T06:17:34.200709+0000","last_active":"2024-09-14T06:18:34.522222+0000","last_peered":"2024-09-14T06:18:34.522222+0000","last_clean":"2024-09-14T06:18:34.522222+0000","last_became_active":"2024-09-14T06:17:34.200238+0000","last_became_peered":"2024-09-14T06:17:34.200238+0000","last_unstale":"2024-09-14T06:18:34.522222+0000","last_undegraded":"2024-09-14T06:18:34.522222+0000","last_fullsized":"2024-09-14T06:18:34.522222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:34.864695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"123'487","reported_seq":1380,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521933+0000","last_change":"2024-09-14T06:18:17.229782+0000","last_active":"2024-09-14T06:18:34.521933+0000","last_peered":"2024-09-14T06:18:34.521933+0000","last_clean":"2024-09-14T06:18:34.521933+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:18:34.521933+0000","last_undegraded":"2024-09-14T06:18:34.521933+0000","last_fullsized":"2024-09-14T06:18:34.521933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:58:18.826180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030371690000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1562,"num_read_kb":9726,"num_write":495,"num_write_kb":9169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":253,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521995+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:18:34.521995+0000","last_peered":"2024-09-14T06:18:34.521995+0000","last_clean":"2024-09-14T06:18:34.521995+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:18:34.521995+0000","last_undegraded":"2024-09-14T06:18:34.521995+0000","last_fullsized":"2024-09-14T06:18:34.521995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"123'382","reported_seq":3317,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522136+0000","last_change":"2024-09-14T06:18:17.229810+0000","last_active":"2024-09-14T06:18:34.522136+0000","last_peered":"2024-09-14T06:18:34.522136+0000","last_clean":"2024-09-14T06:18:34.522136+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:18:34.522136+0000","last_undegraded":"2024-09-14T06:18:34.522136+0000","last_fullsized":"2024-09-14T06:18:34.522136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:45:12.946377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030864040000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3293,"num_read_kb":28583,"num_write":338,"num_write_kb":15237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":123,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522548+0000","last_change":"2024-09-14T06:17:34.196668+0000","last_active":"2024-09-14T06:18:34.522548+0000","last_peered":"2024-09-14T06:18:34.522548+0000","last_clean":"2024-09-14T06:18:34.522548+0000","last_became_active":"2024-09-14T06:17:34.196072+0000","last_became_peered":"2024-09-14T06:17:34.196072+0000","last_unstale":"2024-09-14T06:18:34.522548+0000","last_undegraded":"2024-09-14T06:18:34.522548+0000","last_fullsized":"2024-09-14T06:18:34.522548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:18:45.616834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"123'369","reported_seq":2035,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739803+0000","last_change":"2024-09-14T06:18:17.300971+0000","last_active":"2024-09-14T06:18:34.739803+0000","last_peered":"2024-09-14T06:18:34.739803+0000","last_clean":"2024-09-14T06:18:34.739803+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:18:34.739803+0000","last_undegraded":"2024-09-14T06:18:34.739803+0000","last_fullsized":"2024-09-14T06:18:34.739803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:05.035879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018818730000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1780,"num_read_kb":12934,"num_write":307,"num_write_kb":11582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":255,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740225+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:18:34.740225+0000","last_peered":"2024-09-14T06:18:34.740225+0000","last_clean":"2024-09-14T06:18:34.740225+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:18:34.740225+0000","last_undegraded":"2024-09-14T06:18:34.740225+0000","last_fullsized":"2024-09-14T06:18:34.740225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"123'272","reported_seq":1766,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521495+0000","last_change":"2024-09-14T06:18:25.223649+0000","last_active":"2024-09-14T06:18:34.521495+0000","last_peered":"2024-09-14T06:18:34.521495+0000","last_clean":"2024-09-14T06:18:34.521495+0000","last_became_active":"2024-09-14T06:18:25.221823+0000","last_became_peered":"2024-09-14T06:18:25.221823+0000","last_unstale":"2024-09-14T06:18:34.521495+0000","last_undegraded":"2024-09-14T06:18:34.521495+0000","last_fullsized":"2024-09-14T06:18:34.521495+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:35:13.386184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019482589999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1362,"num_read_kb":21261,"num_write":264,"num_write_kb":9954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"123'394","reported_seq":1248,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739512+0000","last_change":"2024-09-14T06:18:17.226968+0000","last_active":"2024-09-14T06:18:34.739512+0000","last_peered":"2024-09-14T06:18:34.739512+0000","last_clean":"2024-09-14T06:18:34.739512+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:18:34.739512+0000","last_undegraded":"2024-09-14T06:18:34.739512+0000","last_fullsized":"2024-09-14T06:18:34.739512+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:39:35.192039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001556469,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":547,"num_read_kb":4255,"num_write":315,"num_write_kb":7133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":127,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520864+0000","last_change":"2024-09-14T06:17:34.203977+0000","last_active":"2024-09-14T06:18:34.520864+0000","last_peered":"2024-09-14T06:18:34.520864+0000","last_clean":"2024-09-14T06:18:34.520864+0000","last_became_active":"2024-09-14T06:17:34.199244+0000","last_became_peered":"2024-09-14T06:17:34.199244+0000","last_unstale":"2024-09-14T06:18:34.520864+0000","last_undegraded":"2024-09-14T06:18:34.520864+0000","last_fullsized":"2024-09-14T06:18:34.520864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:21.934094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"123'407","reported_seq":2036,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521548+0000","last_change":"2024-09-14T06:18:17.238754+0000","last_active":"2024-09-14T06:18:34.521548+0000","last_peered":"2024-09-14T06:18:34.521548+0000","last_clean":"2024-09-14T06:18:34.521548+0000","last_became_active":"2024-09-14T06:18:17.230545+0000","last_became_peered":"2024-09-14T06:18:17.230545+0000","last_unstale":"2024-09-14T06:18:34.521548+0000","last_undegraded":"2024-09-14T06:18:34.521548+0000","last_fullsized":"2024-09-14T06:18:34.521548+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:17:11.336110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048632429999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2097,"num_read_kb":18969,"num_write":521,"num_write_kb":11051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":36437,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":254,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739790+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:18:34.739790+0000","last_peered":"2024-09-14T06:18:34.739790+0000","last_clean":"2024-09-14T06:18:34.739790+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:18:34.739790+0000","last_undegraded":"2024-09-14T06:18:34.739790+0000","last_fullsized":"2024-09-14T06:18:34.739790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"117'540","reported_seq":2241,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739370+0000","last_change":"2024-09-14T06:18:17.300145+0000","last_active":"2024-09-14T06:18:34.739370+0000","last_peered":"2024-09-14T06:18:34.739370+0000","last_clean":"2024-09-14T06:18:34.739370+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:18:34.739370+0000","last_undegraded":"2024-09-14T06:18:34.739370+0000","last_fullsized":"2024-09-14T06:18:34.739370+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:25.627665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001640601,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2885,"num_read_kb":8812,"num_write":721,"num_write_kb":8784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":253,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520702+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:18:34.520702+0000","last_peered":"2024-09-14T06:18:34.520702+0000","last_clean":"2024-09-14T06:18:34.520702+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:18:34.520702+0000","last_undegraded":"2024-09-14T06:18:34.520702+0000","last_fullsized":"2024-09-14T06:18:34.520702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"122'471","reported_seq":2078,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521629+0000","last_change":"2024-09-14T06:18:17.227668+0000","last_active":"2024-09-14T06:18:34.521629+0000","last_peered":"2024-09-14T06:18:34.521629+0000","last_clean":"2024-09-14T06:18:34.521629+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:18:34.521629+0000","last_undegraded":"2024-09-14T06:18:34.521629+0000","last_fullsized":"2024-09-14T06:18:34.521629+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:19:40.041408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001487162,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2220,"num_read_kb":23013,"num_write":375,"num_write_kb":14523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":274,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522087+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:18:34.522087+0000","last_peered":"2024-09-14T06:18:34.522087+0000","last_clean":"2024-09-14T06:18:34.522087+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:18:34.522087+0000","last_undegraded":"2024-09-14T06:18:34.522087+0000","last_fullsized":"2024-09-14T06:18:34.522087+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":345,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738574+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:18:34.738574+0000","last_peered":"2024-09-14T06:18:34.738574+0000","last_clean":"2024-09-14T06:18:34.738574+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:18:34.738574+0000","last_undegraded":"2024-09-14T06:18:34.738574+0000","last_fullsized":"2024-09-14T06:18:34.738574+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"122'442","reported_seq":1709,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521572+0000","last_change":"2024-09-14T06:18:17.230742+0000","last_active":"2024-09-14T06:18:34.521572+0000","last_peered":"2024-09-14T06:18:34.521572+0000","last_clean":"2024-09-14T06:18:34.521572+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:18:34.521572+0000","last_undegraded":"2024-09-14T06:18:34.521572+0000","last_fullsized":"2024-09-14T06:18:34.521572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:58:35.960390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00348241,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2133,"num_read_kb":6715,"num_write":589,"num_write_kb":6849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"122'398","reported_seq":1904,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520702+0000","last_change":"2024-09-14T06:18:17.336435+0000","last_active":"2024-09-14T06:18:34.520702+0000","last_peered":"2024-09-14T06:18:34.520702+0000","last_clean":"2024-09-14T06:18:34.520702+0000","last_became_active":"2024-09-14T06:17:38.224238+0000","last_became_peered":"2024-09-14T06:17:38.224238+0000","last_unstale":"2024-09-14T06:18:34.520702+0000","last_undegraded":"2024-09-14T06:18:34.520702+0000","last_fullsized":"2024-09-14T06:18:34.520702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:37.628302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064732729999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2223,"num_read_kb":13651,"num_write":549,"num_write_kb":14353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8477378,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"123'384","reported_seq":2444,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521434+0000","last_change":"2024-09-14T06:18:17.336082+0000","last_active":"2024-09-14T06:18:34.521434+0000","last_peered":"2024-09-14T06:18:34.521434+0000","last_clean":"2024-09-14T06:18:34.521434+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:18:34.521434+0000","last_undegraded":"2024-09-14T06:18:34.521434+0000","last_fullsized":"2024-09-14T06:18:34.521434+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:58.737457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005800637,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":2425,"num_read_kb":16285,"num_write":322,"num_write_kb":13807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"123'318","reported_seq":2937,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739738+0000","last_change":"2024-09-14T06:18:17.236905+0000","last_active":"2024-09-14T06:18:34.739738+0000","last_peered":"2024-09-14T06:18:34.739738+0000","last_clean":"2024-09-14T06:18:34.739738+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:18:34.739738+0000","last_undegraded":"2024-09-14T06:18:34.739738+0000","last_fullsized":"2024-09-14T06:18:34.739738+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:57:26.637373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013408000000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2870,"num_read_kb":16793,"num_write":272,"num_write_kb":9901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"123'189","reported_seq":1698,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739303+0000","last_change":"2024-09-14T06:18:17.226333+0000","last_active":"2024-09-14T06:18:34.739303+0000","last_peered":"2024-09-14T06:18:34.739303+0000","last_clean":"2024-09-14T06:18:34.739303+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:18:34.739303+0000","last_undegraded":"2024-09-14T06:18:34.739303+0000","last_fullsized":"2024-09-14T06:18:34.739303+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:57:04.903011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010333339999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1287,"num_read_kb":29081,"num_write":248,"num_write_kb":11577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"122'687","reported_seq":2640,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522423+0000","last_change":"2024-09-14T06:18:17.337571+0000","last_active":"2024-09-14T06:18:34.522423+0000","last_peered":"2024-09-14T06:18:34.522423+0000","last_clean":"2024-09-14T06:18:34.522423+0000","last_became_active":"2024-09-14T06:17:38.227106+0000","last_became_peered":"2024-09-14T06:17:38.227106+0000","last_unstale":"2024-09-14T06:18:34.522423+0000","last_undegraded":"2024-09-14T06:18:34.522423+0000","last_fullsized":"2024-09-14T06:18:34.522423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:27:58.009450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006423425,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3768,"num_read_kb":26949,"num_write":994,"num_write_kb":20276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":29390214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"123'762","reported_seq":5455,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521683+0000","last_change":"2024-09-14T06:18:25.224296+0000","last_active":"2024-09-14T06:18:34.521683+0000","last_peered":"2024-09-14T06:18:34.521683+0000","last_clean":"2024-09-14T06:18:34.521683+0000","last_became_active":"2024-09-14T06:18:25.223198+0000","last_became_peered":"2024-09-14T06:18:25.223198+0000","last_unstale":"2024-09-14T06:18:34.521683+0000","last_undegraded":"2024-09-14T06:18:34.521683+0000","last_fullsized":"2024-09-14T06:18:34.521683+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:33:35.589616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021565500000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10255,"num_read_kb":51706,"num_write":697,"num_write_kb":16134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"123'661","reported_seq":2987,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739412+0000","last_change":"2024-09-14T06:18:17.226919+0000","last_active":"2024-09-14T06:18:34.739412+0000","last_peered":"2024-09-14T06:18:34.739412+0000","last_clean":"2024-09-14T06:18:34.739412+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:18:34.739412+0000","last_undegraded":"2024-09-14T06:18:34.739412+0000","last_fullsized":"2024-09-14T06:18:34.739412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:55:17.915614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001432902,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4425,"num_read_kb":16166,"num_write":768,"num_write_kb":12749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"122'360","reported_seq":2605,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738517+0000","last_change":"2024-09-14T06:18:17.221410+0000","last_active":"2024-09-14T06:18:34.738517+0000","last_peered":"2024-09-14T06:18:34.738517+0000","last_clean":"2024-09-14T06:18:34.738517+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:18:34.738517+0000","last_undegraded":"2024-09-14T06:18:34.738517+0000","last_fullsized":"2024-09-14T06:18:34.738517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:24:43.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068876799999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2476,"num_read_kb":47684,"num_write":341,"num_write_kb":14643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"123'556","reported_seq":3001,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521007+0000","last_change":"2024-09-14T06:18:17.332968+0000","last_active":"2024-09-14T06:18:34.521007+0000","last_peered":"2024-09-14T06:18:34.521007+0000","last_clean":"2024-09-14T06:18:34.521007+0000","last_became_active":"2024-09-14T06:17:38.224251+0000","last_became_peered":"2024-09-14T06:17:38.224251+0000","last_unstale":"2024-09-14T06:18:34.521007+0000","last_undegraded":"2024-09-14T06:18:34.521007+0000","last_fullsized":"2024-09-14T06:18:34.521007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:32:17.880108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033873589999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4434,"num_read_kb":14632,"num_write":613,"num_write_kb":7670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":12239008,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"122'518","reported_seq":3082,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739545+0000","last_change":"2024-09-14T06:18:17.300159+0000","last_active":"2024-09-14T06:18:34.739545+0000","last_peered":"2024-09-14T06:18:34.739545+0000","last_clean":"2024-09-14T06:18:34.739545+0000","last_became_active":"2024-09-14T06:17:37.232145+0000","last_became_peered":"2024-09-14T06:17:37.232145+0000","last_unstale":"2024-09-14T06:18:34.739545+0000","last_undegraded":"2024-09-14T06:18:34.739545+0000","last_fullsized":"2024-09-14T06:18:34.739545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:12:47.470446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016513039999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3345,"num_read_kb":11048,"num_write":606,"num_write_kb":8456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":10176554,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"122'505","reported_seq":4220,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521410+0000","last_change":"2024-09-14T06:18:17.333610+0000","last_active":"2024-09-14T06:18:34.521410+0000","last_peered":"2024-09-14T06:18:34.521410+0000","last_clean":"2024-09-14T06:18:34.521410+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:18:34.521410+0000","last_undegraded":"2024-09-14T06:18:34.521410+0000","last_fullsized":"2024-09-14T06:18:34.521410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:51:31.054844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039616110000000003,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6037,"num_read_kb":16850,"num_write":431,"num_write_kb":5588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":210,"num_object_clones":0,"num_object_copies":420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210,"num_whiteouts":206,"num_read":107256,"num_read_kb":751154,"num_write":17465,"num_write_kb":429271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1267,"num_bytes_recovered":144403203,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15570,"ondisk_log_size":15570,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509474,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55028,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316812,"statfs":{"total":96636764160,"available":96580415488,"internally_reserved":0,"allocated":1662976,"data_stored":789529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509474,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36972,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334868,"statfs":{"total":96636764160,"available":96598904832,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509476,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73132,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298708,"statfs":{"total":96636764160,"available":96561876992,"internally_reserved":0,"allocated":1724416,"data_stored":1379897,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59168,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312672,"statfs":{"total":96636764160,"available":96576176128,"internally_reserved":0,"allocated":1646592,"data_stored":783610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542178,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36956,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334884,"statfs":{"total":96636764160,"available":96598921216,"internally_reserved":0,"allocated":1646592,"data_stored":783610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542178,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28652,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343188,"statfs":{"total":96636764160,"available":96607424512,"internally_reserved":0,"allocated":1662976,"data_stored":789529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542177,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28716,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343124,"statfs":{"total":96636764160,"available":96607358976,"internally_reserved":0,"allocated":1724416,"data_stored":1379897,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28652,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343188,"statfs":{"total":96636764160,"available":96607424512,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:18:38.083 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:18:38.369 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:18:38.370 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:18:38.387 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2024-09-14T06:18:36.419497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":214,"num_object_clones":0,"num_object_copies":428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":214,"num_whiteouts":206,"num_read":107322,"num_read_kb":751209,"num_write":17568,"num_write_kb":430901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1270,"num_bytes_recovered":144403222,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15645,"ondisk_log_size":15645,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":347276,"kb_used_data":13072,"kb_used_omap":12,"kb_used_meta":334003,"kb_avail":754627444,"statfs":{"total":773094113280,"available":772738502656,"internally_reserved":0,"allocated":13385728,"data_stored":7485092,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":342019632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.973991"},"pg_stats":[{"pgid":"3.1f","version":"123'604","reported_seq":3721,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740380+0000","last_change":"2024-09-14T06:18:17.225123+0000","last_active":"2024-09-14T06:18:34.740380+0000","last_peered":"2024-09-14T06:18:34.740380+0000","last_clean":"2024-09-14T06:18:34.740380+0000","last_became_active":"2024-09-14T06:17:02.193313+0000","last_became_peered":"2024-09-14T06:17:02.193313+0000","last_unstale":"2024-09-14T06:18:34.740380+0000","last_undegraded":"2024-09-14T06:18:34.740380+0000","last_fullsized":"2024-09-14T06:18:34.740380+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:43:51.113692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002155396,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4617,"num_read_kb":53605,"num_write":714,"num_write_kb":19355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"122'407","reported_seq":4554,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739967+0000","last_change":"2024-09-14T06:18:17.224253+0000","last_active":"2024-09-14T06:18:34.739967+0000","last_peered":"2024-09-14T06:18:34.739967+0000","last_clean":"2024-09-14T06:18:34.739967+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:18:34.739967+0000","last_undegraded":"2024-09-14T06:18:34.739967+0000","last_fullsized":"2024-09-14T06:18:34.739967+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:46:26.694964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019300890000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5022,"num_read_kb":43064,"num_write":350,"num_write_kb":15485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"123'1132","reported_seq":2556,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522291+0000","last_change":"2024-09-14T06:18:17.337001+0000","last_active":"2024-09-14T06:18:34.522291+0000","last_peered":"2024-09-14T06:18:34.522291+0000","last_clean":"2024-09-14T06:18:34.522291+0000","last_became_active":"2024-09-14T06:17:38.226838+0000","last_became_peered":"2024-09-14T06:17:38.226838+0000","last_unstale":"2024-09-14T06:18:34.522291+0000","last_undegraded":"2024-09-14T06:18:34.522291+0000","last_fullsized":"2024-09-14T06:18:34.522291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":1132,"log_dups_size":0,"ondisk_log_size":1132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:01.176538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062318570000000004,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5125,"num_read_kb":14378,"num_write":1577,"num_write_kb":13994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10785266,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"123'934","reported_seq":3133,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521839+0000","last_change":"2024-09-14T06:18:17.228603+0000","last_active":"2024-09-14T06:18:34.521839+0000","last_peered":"2024-09-14T06:18:34.521839+0000","last_clean":"2024-09-14T06:18:34.521839+0000","last_became_active":"2024-09-14T06:17:48.737020+0000","last_became_peered":"2024-09-14T06:17:48.737020+0000","last_unstale":"2024-09-14T06:18:34.521839+0000","last_undegraded":"2024-09-14T06:18:34.521839+0000","last_fullsized":"2024-09-14T06:18:34.521839+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:04:33.297278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024240709999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":6011,"num_read_kb":10015,"num_write":1345,"num_write_kb":6524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":3869232,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"123'349","reported_seq":3834,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521767+0000","last_change":"2024-09-14T06:18:17.228692+0000","last_active":"2024-09-14T06:18:34.521767+0000","last_peered":"2024-09-14T06:18:34.521767+0000","last_clean":"2024-09-14T06:18:34.521767+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:18:34.521767+0000","last_undegraded":"2024-09-14T06:18:34.521767+0000","last_fullsized":"2024-09-14T06:18:34.521767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:40:01.686387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024977549999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3342,"num_read_kb":39056,"num_write":366,"num_write_kb":25723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"122'539","reported_seq":3387,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738796+0000","last_change":"2024-09-14T06:18:17.221967+0000","last_active":"2024-09-14T06:18:34.738796+0000","last_peered":"2024-09-14T06:18:34.738796+0000","last_clean":"2024-09-14T06:18:34.738796+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:18:34.738796+0000","last_undegraded":"2024-09-14T06:18:34.738796+0000","last_fullsized":"2024-09-14T06:18:34.738796+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:05:11.528071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087057599999999997,"stat_sum":{"num_bytes":19,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":4088,"num_read_kb":13128,"num_write":658,"num_write_kb":7511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"122'532","reported_seq":2187,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521641+0000","last_change":"2024-09-14T06:18:17.227798+0000","last_active":"2024-09-14T06:18:34.521641+0000","last_peered":"2024-09-14T06:18:34.521641+0000","last_clean":"2024-09-14T06:18:34.521641+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:18:34.521641+0000","last_undegraded":"2024-09-14T06:18:34.521641+0000","last_fullsized":"2024-09-14T06:18:34.521641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:09:43.212330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001716539,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2883,"num_read_kb":19234,"num_write":704,"num_write_kb":12873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"123'400","reported_seq":3103,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740079+0000","last_change":"2024-09-14T06:18:17.225653+0000","last_active":"2024-09-14T06:18:34.740079+0000","last_peered":"2024-09-14T06:18:34.740079+0000","last_clean":"2024-09-14T06:18:34.740079+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:18:34.740079+0000","last_undegraded":"2024-09-14T06:18:34.740079+0000","last_fullsized":"2024-09-14T06:18:34.740079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:38:39.952754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002524938,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":3795,"num_read_kb":37772,"num_write":360,"num_write_kb":32941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"119'395","reported_seq":1793,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740431+0000","last_change":"2024-09-14T06:18:17.224547+0000","last_active":"2024-09-14T06:18:34.740431+0000","last_peered":"2024-09-14T06:18:34.740431+0000","last_clean":"2024-09-14T06:18:34.740431+0000","last_became_active":"2024-09-14T06:17:37.233410+0000","last_became_peered":"2024-09-14T06:17:37.233410+0000","last_unstale":"2024-09-14T06:18:34.740431+0000","last_undegraded":"2024-09-14T06:18:34.740431+0000","last_fullsized":"2024-09-14T06:18:34.740431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:33.122557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019943040000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1416,"num_read_kb":17477,"num_write":336,"num_write_kb":17193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":23467377,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"122'436","reported_seq":2420,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521785+0000","last_change":"2024-09-14T06:18:25.223056+0000","last_active":"2024-09-14T06:18:34.521785+0000","last_peered":"2024-09-14T06:18:34.521785+0000","last_clean":"2024-09-14T06:18:34.521785+0000","last_became_active":"2024-09-14T06:18:25.219200+0000","last_became_peered":"2024-09-14T06:18:25.219200+0000","last_unstale":"2024-09-14T06:18:34.521785+0000","last_undegraded":"2024-09-14T06:18:34.521785+0000","last_fullsized":"2024-09-14T06:18:34.521785+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:26:30.150015+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.047023011000000003,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2483,"num_read_kb":16488,"num_write":510,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":6785399,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"122'432","reported_seq":3823,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739366+0000","last_change":"2024-09-14T06:18:17.226299+0000","last_active":"2024-09-14T06:18:34.739366+0000","last_peered":"2024-09-14T06:18:34.739366+0000","last_clean":"2024-09-14T06:18:34.739366+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:18:34.739366+0000","last_undegraded":"2024-09-14T06:18:34.739366+0000","last_fullsized":"2024-09-14T06:18:34.739366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:42:58.536316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094053699999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4134,"num_read_kb":64629,"num_write":506,"num_write_kb":22219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"123'308","reported_seq":2112,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522107+0000","last_change":"2024-09-14T06:18:17.333655+0000","last_active":"2024-09-14T06:18:34.522107+0000","last_peered":"2024-09-14T06:18:34.522107+0000","last_clean":"2024-09-14T06:18:34.522107+0000","last_became_active":"2024-09-14T06:17:37.231244+0000","last_became_peered":"2024-09-14T06:17:37.231244+0000","last_unstale":"2024-09-14T06:18:34.522107+0000","last_undegraded":"2024-09-14T06:18:34.522107+0000","last_fullsized":"2024-09-14T06:18:34.522107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:10.356294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044577879999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2916,"num_read_kb":27195,"num_write":273,"num_write_kb":15734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":32506492,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":123,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522222+0000","last_change":"2024-09-14T06:17:34.200709+0000","last_active":"2024-09-14T06:18:34.522222+0000","last_peered":"2024-09-14T06:18:34.522222+0000","last_clean":"2024-09-14T06:18:34.522222+0000","last_became_active":"2024-09-14T06:17:34.200238+0000","last_became_peered":"2024-09-14T06:17:34.200238+0000","last_unstale":"2024-09-14T06:18:34.522222+0000","last_undegraded":"2024-09-14T06:18:34.522222+0000","last_fullsized":"2024-09-14T06:18:34.522222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:34.864695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"123'487","reported_seq":1380,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521933+0000","last_change":"2024-09-14T06:18:17.229782+0000","last_active":"2024-09-14T06:18:34.521933+0000","last_peered":"2024-09-14T06:18:34.521933+0000","last_clean":"2024-09-14T06:18:34.521933+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:18:34.521933+0000","last_undegraded":"2024-09-14T06:18:34.521933+0000","last_fullsized":"2024-09-14T06:18:34.521933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:58:18.826180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030371690000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1562,"num_read_kb":9726,"num_write":495,"num_write_kb":9169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":253,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521995+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:18:34.521995+0000","last_peered":"2024-09-14T06:18:34.521995+0000","last_clean":"2024-09-14T06:18:34.521995+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:18:34.521995+0000","last_undegraded":"2024-09-14T06:18:34.521995+0000","last_fullsized":"2024-09-14T06:18:34.521995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"123'382","reported_seq":3317,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522136+0000","last_change":"2024-09-14T06:18:17.229810+0000","last_active":"2024-09-14T06:18:34.522136+0000","last_peered":"2024-09-14T06:18:34.522136+0000","last_clean":"2024-09-14T06:18:34.522136+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:18:34.522136+0000","last_undegraded":"2024-09-14T06:18:34.522136+0000","last_fullsized":"2024-09-14T06:18:34.522136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:45:12.946377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030864040000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3293,"num_read_kb":28583,"num_write":338,"num_write_kb":15237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":123,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522548+0000","last_change":"2024-09-14T06:17:34.196668+0000","last_active":"2024-09-14T06:18:34.522548+0000","last_peered":"2024-09-14T06:18:34.522548+0000","last_clean":"2024-09-14T06:18:34.522548+0000","last_became_active":"2024-09-14T06:17:34.196072+0000","last_became_peered":"2024-09-14T06:17:34.196072+0000","last_unstale":"2024-09-14T06:18:34.522548+0000","last_undegraded":"2024-09-14T06:18:34.522548+0000","last_fullsized":"2024-09-14T06:18:34.522548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:18:45.616834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"123'369","reported_seq":2035,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739803+0000","last_change":"2024-09-14T06:18:17.300971+0000","last_active":"2024-09-14T06:18:34.739803+0000","last_peered":"2024-09-14T06:18:34.739803+0000","last_clean":"2024-09-14T06:18:34.739803+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:18:34.739803+0000","last_undegraded":"2024-09-14T06:18:34.739803+0000","last_fullsized":"2024-09-14T06:18:34.739803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:05.035879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018818730000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1780,"num_read_kb":12934,"num_write":307,"num_write_kb":11582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":255,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740225+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:18:34.740225+0000","last_peered":"2024-09-14T06:18:34.740225+0000","last_clean":"2024-09-14T06:18:34.740225+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:18:34.740225+0000","last_undegraded":"2024-09-14T06:18:34.740225+0000","last_fullsized":"2024-09-14T06:18:34.740225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"123'272","reported_seq":1766,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521495+0000","last_change":"2024-09-14T06:18:25.223649+0000","last_active":"2024-09-14T06:18:34.521495+0000","last_peered":"2024-09-14T06:18:34.521495+0000","last_clean":"2024-09-14T06:18:34.521495+0000","last_became_active":"2024-09-14T06:18:25.221823+0000","last_became_peered":"2024-09-14T06:18:25.221823+0000","last_unstale":"2024-09-14T06:18:34.521495+0000","last_undegraded":"2024-09-14T06:18:34.521495+0000","last_fullsized":"2024-09-14T06:18:34.521495+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:35:13.386184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019482589999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1362,"num_read_kb":21261,"num_write":264,"num_write_kb":9954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"123'394","reported_seq":1248,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739512+0000","last_change":"2024-09-14T06:18:17.226968+0000","last_active":"2024-09-14T06:18:34.739512+0000","last_peered":"2024-09-14T06:18:34.739512+0000","last_clean":"2024-09-14T06:18:34.739512+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:18:34.739512+0000","last_undegraded":"2024-09-14T06:18:34.739512+0000","last_fullsized":"2024-09-14T06:18:34.739512+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:39:35.192039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001556469,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":547,"num_read_kb":4255,"num_write":315,"num_write_kb":7133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":127,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520864+0000","last_change":"2024-09-14T06:17:34.203977+0000","last_active":"2024-09-14T06:18:34.520864+0000","last_peered":"2024-09-14T06:18:34.520864+0000","last_clean":"2024-09-14T06:18:34.520864+0000","last_became_active":"2024-09-14T06:17:34.199244+0000","last_became_peered":"2024-09-14T06:17:34.199244+0000","last_unstale":"2024-09-14T06:18:34.520864+0000","last_undegraded":"2024-09-14T06:18:34.520864+0000","last_fullsized":"2024-09-14T06:18:34.520864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:21.934094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"123'407","reported_seq":2036,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521548+0000","last_change":"2024-09-14T06:18:17.238754+0000","last_active":"2024-09-14T06:18:34.521548+0000","last_peered":"2024-09-14T06:18:34.521548+0000","last_clean":"2024-09-14T06:18:34.521548+0000","last_became_active":"2024-09-14T06:18:17.230545+0000","last_became_peered":"2024-09-14T06:18:17.230545+0000","last_unstale":"2024-09-14T06:18:34.521548+0000","last_undegraded":"2024-09-14T06:18:34.521548+0000","last_fullsized":"2024-09-14T06:18:34.521548+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:17:11.336110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048632429999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2097,"num_read_kb":18969,"num_write":521,"num_write_kb":11051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":36437,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":254,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739790+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:18:34.739790+0000","last_peered":"2024-09-14T06:18:34.739790+0000","last_clean":"2024-09-14T06:18:34.739790+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:18:34.739790+0000","last_undegraded":"2024-09-14T06:18:34.739790+0000","last_fullsized":"2024-09-14T06:18:34.739790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"117'540","reported_seq":2241,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739370+0000","last_change":"2024-09-14T06:18:17.300145+0000","last_active":"2024-09-14T06:18:34.739370+0000","last_peered":"2024-09-14T06:18:34.739370+0000","last_clean":"2024-09-14T06:18:34.739370+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:18:34.739370+0000","last_undegraded":"2024-09-14T06:18:34.739370+0000","last_fullsized":"2024-09-14T06:18:34.739370+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:25.627665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001640601,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2885,"num_read_kb":8812,"num_write":721,"num_write_kb":8784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":253,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520702+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:18:34.520702+0000","last_peered":"2024-09-14T06:18:34.520702+0000","last_clean":"2024-09-14T06:18:34.520702+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:18:34.520702+0000","last_undegraded":"2024-09-14T06:18:34.520702+0000","last_fullsized":"2024-09-14T06:18:34.520702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"122'471","reported_seq":2078,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521629+0000","last_change":"2024-09-14T06:18:17.227668+0000","last_active":"2024-09-14T06:18:34.521629+0000","last_peered":"2024-09-14T06:18:34.521629+0000","last_clean":"2024-09-14T06:18:34.521629+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:18:34.521629+0000","last_undegraded":"2024-09-14T06:18:34.521629+0000","last_fullsized":"2024-09-14T06:18:34.521629+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:19:40.041408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001487162,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2220,"num_read_kb":23013,"num_write":375,"num_write_kb":14523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":274,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522087+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:18:34.522087+0000","last_peered":"2024-09-14T06:18:34.522087+0000","last_clean":"2024-09-14T06:18:34.522087+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:18:34.522087+0000","last_undegraded":"2024-09-14T06:18:34.522087+0000","last_fullsized":"2024-09-14T06:18:34.522087+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":345,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738574+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:18:34.738574+0000","last_peered":"2024-09-14T06:18:34.738574+0000","last_clean":"2024-09-14T06:18:34.738574+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:18:34.738574+0000","last_undegraded":"2024-09-14T06:18:34.738574+0000","last_fullsized":"2024-09-14T06:18:34.738574+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"122'442","reported_seq":1709,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521572+0000","last_change":"2024-09-14T06:18:17.230742+0000","last_active":"2024-09-14T06:18:34.521572+0000","last_peered":"2024-09-14T06:18:34.521572+0000","last_clean":"2024-09-14T06:18:34.521572+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:18:34.521572+0000","last_undegraded":"2024-09-14T06:18:34.521572+0000","last_fullsized":"2024-09-14T06:18:34.521572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:58:35.960390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00348241,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2133,"num_read_kb":6715,"num_write":589,"num_write_kb":6849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"122'398","reported_seq":1904,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520702+0000","last_change":"2024-09-14T06:18:17.336435+0000","last_active":"2024-09-14T06:18:34.520702+0000","last_peered":"2024-09-14T06:18:34.520702+0000","last_clean":"2024-09-14T06:18:34.520702+0000","last_became_active":"2024-09-14T06:17:38.224238+0000","last_became_peered":"2024-09-14T06:17:38.224238+0000","last_unstale":"2024-09-14T06:18:34.520702+0000","last_undegraded":"2024-09-14T06:18:34.520702+0000","last_fullsized":"2024-09-14T06:18:34.520702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:37.628302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064732729999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2223,"num_read_kb":13651,"num_write":549,"num_write_kb":14353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8477378,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"123'384","reported_seq":2444,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521434+0000","last_change":"2024-09-14T06:18:17.336082+0000","last_active":"2024-09-14T06:18:34.521434+0000","last_peered":"2024-09-14T06:18:34.521434+0000","last_clean":"2024-09-14T06:18:34.521434+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:18:34.521434+0000","last_undegraded":"2024-09-14T06:18:34.521434+0000","last_fullsized":"2024-09-14T06:18:34.521434+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:58.737457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005800637,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":2425,"num_read_kb":16285,"num_write":322,"num_write_kb":13807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"123'318","reported_seq":2937,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739738+0000","last_change":"2024-09-14T06:18:17.236905+0000","last_active":"2024-09-14T06:18:34.739738+0000","last_peered":"2024-09-14T06:18:34.739738+0000","last_clean":"2024-09-14T06:18:34.739738+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:18:34.739738+0000","last_undegraded":"2024-09-14T06:18:34.739738+0000","last_fullsized":"2024-09-14T06:18:34.739738+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:57:26.637373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013408000000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2870,"num_read_kb":16793,"num_write":272,"num_write_kb":9901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"123'189","reported_seq":1698,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739303+0000","last_change":"2024-09-14T06:18:17.226333+0000","last_active":"2024-09-14T06:18:34.739303+0000","last_peered":"2024-09-14T06:18:34.739303+0000","last_clean":"2024-09-14T06:18:34.739303+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:18:34.739303+0000","last_undegraded":"2024-09-14T06:18:34.739303+0000","last_fullsized":"2024-09-14T06:18:34.739303+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:57:04.903011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010333339999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1287,"num_read_kb":29081,"num_write":248,"num_write_kb":11577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"122'687","reported_seq":2640,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522423+0000","last_change":"2024-09-14T06:18:17.337571+0000","last_active":"2024-09-14T06:18:34.522423+0000","last_peered":"2024-09-14T06:18:34.522423+0000","last_clean":"2024-09-14T06:18:34.522423+0000","last_became_active":"2024-09-14T06:17:38.227106+0000","last_became_peered":"2024-09-14T06:17:38.227106+0000","last_unstale":"2024-09-14T06:18:34.522423+0000","last_undegraded":"2024-09-14T06:18:34.522423+0000","last_fullsized":"2024-09-14T06:18:34.522423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:27:58.009450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006423425,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3768,"num_read_kb":26949,"num_write":994,"num_write_kb":20276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":29390214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"123'762","reported_seq":5455,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521683+0000","last_change":"2024-09-14T06:18:25.224296+0000","last_active":"2024-09-14T06:18:34.521683+0000","last_peered":"2024-09-14T06:18:34.521683+0000","last_clean":"2024-09-14T06:18:34.521683+0000","last_became_active":"2024-09-14T06:18:25.223198+0000","last_became_peered":"2024-09-14T06:18:25.223198+0000","last_unstale":"2024-09-14T06:18:34.521683+0000","last_undegraded":"2024-09-14T06:18:34.521683+0000","last_fullsized":"2024-09-14T06:18:34.521683+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:33:35.589616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021565500000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10255,"num_read_kb":51706,"num_write":697,"num_write_kb":16134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"123'661","reported_seq":2987,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739412+0000","last_change":"2024-09-14T06:18:17.226919+0000","last_active":"2024-09-14T06:18:34.739412+0000","last_peered":"2024-09-14T06:18:34.739412+0000","last_clean":"2024-09-14T06:18:34.739412+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:18:34.739412+0000","last_undegraded":"2024-09-14T06:18:34.739412+0000","last_fullsized":"2024-09-14T06:18:34.739412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:55:17.915614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001432902,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4425,"num_read_kb":16166,"num_write":768,"num_write_kb":12749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"122'360","reported_seq":2605,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738517+0000","last_change":"2024-09-14T06:18:17.221410+0000","last_active":"2024-09-14T06:18:34.738517+0000","last_peered":"2024-09-14T06:18:34.738517+0000","last_clean":"2024-09-14T06:18:34.738517+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:18:34.738517+0000","last_undegraded":"2024-09-14T06:18:34.738517+0000","last_fullsized":"2024-09-14T06:18:34.738517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:24:43.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068876799999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2476,"num_read_kb":47684,"num_write":341,"num_write_kb":14643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"123'556","reported_seq":3001,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521007+0000","last_change":"2024-09-14T06:18:17.332968+0000","last_active":"2024-09-14T06:18:34.521007+0000","last_peered":"2024-09-14T06:18:34.521007+0000","last_clean":"2024-09-14T06:18:34.521007+0000","last_became_active":"2024-09-14T06:17:38.224251+0000","last_became_peered":"2024-09-14T06:17:38.224251+0000","last_unstale":"2024-09-14T06:18:34.521007+0000","last_undegraded":"2024-09-14T06:18:34.521007+0000","last_fullsized":"2024-09-14T06:18:34.521007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:32:17.880108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033873589999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4434,"num_read_kb":14632,"num_write":613,"num_write_kb":7670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":12239008,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"122'518","reported_seq":3082,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739545+0000","last_change":"2024-09-14T06:18:17.300159+0000","last_active":"2024-09-14T06:18:34.739545+0000","last_peered":"2024-09-14T06:18:34.739545+0000","last_clean":"2024-09-14T06:18:34.739545+0000","last_became_active":"2024-09-14T06:17:37.232145+0000","last_became_peered":"2024-09-14T06:17:37.232145+0000","last_unstale":"2024-09-14T06:18:34.739545+0000","last_undegraded":"2024-09-14T06:18:34.739545+0000","last_fullsized":"2024-09-14T06:18:34.739545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:12:47.470446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016513039999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3345,"num_read_kb":11048,"num_write":606,"num_write_kb":8456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":10176554,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"122'505","reported_seq":4220,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521410+0000","last_change":"2024-09-14T06:18:17.333610+0000","last_active":"2024-09-14T06:18:34.521410+0000","last_peered":"2024-09-14T06:18:34.521410+0000","last_clean":"2024-09-14T06:18:34.521410+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:18:34.521410+0000","last_undegraded":"2024-09-14T06:18:34.521410+0000","last_fullsized":"2024-09-14T06:18:34.521410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:51:31.054844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039616110000000003,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6037,"num_read_kb":16850,"num_write":431,"num_write_kb":5588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":210,"num_object_clones":0,"num_object_copies":420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210,"num_whiteouts":206,"num_read":107256,"num_read_kb":751154,"num_write":17465,"num_write_kb":429271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1267,"num_bytes_recovered":144403203,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15570,"ondisk_log_size":15570,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509474,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55028,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316812,"statfs":{"total":96636764160,"available":96580415488,"internally_reserved":0,"allocated":1662976,"data_stored":789529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509474,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36972,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334868,"statfs":{"total":96636764160,"available":96598904832,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509476,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73132,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298708,"statfs":{"total":96636764160,"available":96561876992,"internally_reserved":0,"allocated":1724416,"data_stored":1379897,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59168,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312672,"statfs":{"total":96636764160,"available":96576176128,"internally_reserved":0,"allocated":1646592,"data_stored":783610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542178,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36956,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334884,"statfs":{"total":96636764160,"available":96598921216,"internally_reserved":0,"allocated":1646592,"data_stored":783610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542178,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28652,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343188,"statfs":{"total":96636764160,"available":96607424512,"internally_reserved":0,"allocated":1662976,"data_stored":789529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542177,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28716,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343124,"statfs":{"total":96636764160,"available":96607358976,"internally_reserved":0,"allocated":1724416,"data_stored":1379897,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28652,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343188,"statfs":{"total":96636764160,"available":96607424512,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:18:38.389 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T06:18:38.389 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-14T06:18:38.717 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-14T06:18:38.726 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T06:18:38.726 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:18:39.024 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:18:39.025 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:18:39.039 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-14T06:18:38.420127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":214,"num_object_clones":0,"num_object_copies":428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":214,"num_whiteouts":206,"num_read":107322,"num_read_kb":751209,"num_write":17568,"num_write_kb":430901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1270,"num_bytes_recovered":144403222,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15645,"ondisk_log_size":15645,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":347308,"kb_used_data":13096,"kb_used_omap":12,"kb_used_meta":334003,"kb_avail":754627412,"statfs":{"total":773094113280,"available":772738469888,"internally_reserved":0,"allocated":13410304,"data_stored":7496892,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":342019632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001830"},"pg_stats":[{"pgid":"3.1f","version":"123'604","reported_seq":3721,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740380+0000","last_change":"2024-09-14T06:18:17.225123+0000","last_active":"2024-09-14T06:18:34.740380+0000","last_peered":"2024-09-14T06:18:34.740380+0000","last_clean":"2024-09-14T06:18:34.740380+0000","last_became_active":"2024-09-14T06:17:02.193313+0000","last_became_peered":"2024-09-14T06:17:02.193313+0000","last_unstale":"2024-09-14T06:18:34.740380+0000","last_undegraded":"2024-09-14T06:18:34.740380+0000","last_fullsized":"2024-09-14T06:18:34.740380+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:43:51.113692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002155396,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4617,"num_read_kb":53605,"num_write":714,"num_write_kb":19355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"122'407","reported_seq":4554,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739967+0000","last_change":"2024-09-14T06:18:17.224253+0000","last_active":"2024-09-14T06:18:34.739967+0000","last_peered":"2024-09-14T06:18:34.739967+0000","last_clean":"2024-09-14T06:18:34.739967+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:18:34.739967+0000","last_undegraded":"2024-09-14T06:18:34.739967+0000","last_fullsized":"2024-09-14T06:18:34.739967+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:46:26.694964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019300890000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5022,"num_read_kb":43064,"num_write":350,"num_write_kb":15485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"123'1132","reported_seq":2556,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522291+0000","last_change":"2024-09-14T06:18:17.337001+0000","last_active":"2024-09-14T06:18:34.522291+0000","last_peered":"2024-09-14T06:18:34.522291+0000","last_clean":"2024-09-14T06:18:34.522291+0000","last_became_active":"2024-09-14T06:17:38.226838+0000","last_became_peered":"2024-09-14T06:17:38.226838+0000","last_unstale":"2024-09-14T06:18:34.522291+0000","last_undegraded":"2024-09-14T06:18:34.522291+0000","last_fullsized":"2024-09-14T06:18:34.522291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":1132,"log_dups_size":0,"ondisk_log_size":1132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:01.176538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062318570000000004,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5125,"num_read_kb":14378,"num_write":1577,"num_write_kb":13994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10785266,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"123'934","reported_seq":3133,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521839+0000","last_change":"2024-09-14T06:18:17.228603+0000","last_active":"2024-09-14T06:18:34.521839+0000","last_peered":"2024-09-14T06:18:34.521839+0000","last_clean":"2024-09-14T06:18:34.521839+0000","last_became_active":"2024-09-14T06:17:48.737020+0000","last_became_peered":"2024-09-14T06:17:48.737020+0000","last_unstale":"2024-09-14T06:18:34.521839+0000","last_undegraded":"2024-09-14T06:18:34.521839+0000","last_fullsized":"2024-09-14T06:18:34.521839+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:04:33.297278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024240709999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":6011,"num_read_kb":10015,"num_write":1345,"num_write_kb":6524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":3869232,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"123'349","reported_seq":3834,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521767+0000","last_change":"2024-09-14T06:18:17.228692+0000","last_active":"2024-09-14T06:18:34.521767+0000","last_peered":"2024-09-14T06:18:34.521767+0000","last_clean":"2024-09-14T06:18:34.521767+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:18:34.521767+0000","last_undegraded":"2024-09-14T06:18:34.521767+0000","last_fullsized":"2024-09-14T06:18:34.521767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:40:01.686387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024977549999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3342,"num_read_kb":39056,"num_write":366,"num_write_kb":25723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"122'539","reported_seq":3387,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738796+0000","last_change":"2024-09-14T06:18:17.221967+0000","last_active":"2024-09-14T06:18:34.738796+0000","last_peered":"2024-09-14T06:18:34.738796+0000","last_clean":"2024-09-14T06:18:34.738796+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:18:34.738796+0000","last_undegraded":"2024-09-14T06:18:34.738796+0000","last_fullsized":"2024-09-14T06:18:34.738796+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:05:11.528071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087057599999999997,"stat_sum":{"num_bytes":19,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":4088,"num_read_kb":13128,"num_write":658,"num_write_kb":7511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"122'532","reported_seq":2187,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521641+0000","last_change":"2024-09-14T06:18:17.227798+0000","last_active":"2024-09-14T06:18:34.521641+0000","last_peered":"2024-09-14T06:18:34.521641+0000","last_clean":"2024-09-14T06:18:34.521641+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:18:34.521641+0000","last_undegraded":"2024-09-14T06:18:34.521641+0000","last_fullsized":"2024-09-14T06:18:34.521641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:09:43.212330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001716539,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2883,"num_read_kb":19234,"num_write":704,"num_write_kb":12873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"123'400","reported_seq":3103,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740079+0000","last_change":"2024-09-14T06:18:17.225653+0000","last_active":"2024-09-14T06:18:34.740079+0000","last_peered":"2024-09-14T06:18:34.740079+0000","last_clean":"2024-09-14T06:18:34.740079+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:18:34.740079+0000","last_undegraded":"2024-09-14T06:18:34.740079+0000","last_fullsized":"2024-09-14T06:18:34.740079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:38:39.952754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002524938,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":3795,"num_read_kb":37772,"num_write":360,"num_write_kb":32941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"119'395","reported_seq":1793,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740431+0000","last_change":"2024-09-14T06:18:17.224547+0000","last_active":"2024-09-14T06:18:34.740431+0000","last_peered":"2024-09-14T06:18:34.740431+0000","last_clean":"2024-09-14T06:18:34.740431+0000","last_became_active":"2024-09-14T06:17:37.233410+0000","last_became_peered":"2024-09-14T06:17:37.233410+0000","last_unstale":"2024-09-14T06:18:34.740431+0000","last_undegraded":"2024-09-14T06:18:34.740431+0000","last_fullsized":"2024-09-14T06:18:34.740431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:33.122557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019943040000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1416,"num_read_kb":17477,"num_write":336,"num_write_kb":17193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":23467377,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"122'436","reported_seq":2420,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521785+0000","last_change":"2024-09-14T06:18:25.223056+0000","last_active":"2024-09-14T06:18:34.521785+0000","last_peered":"2024-09-14T06:18:34.521785+0000","last_clean":"2024-09-14T06:18:34.521785+0000","last_became_active":"2024-09-14T06:18:25.219200+0000","last_became_peered":"2024-09-14T06:18:25.219200+0000","last_unstale":"2024-09-14T06:18:34.521785+0000","last_undegraded":"2024-09-14T06:18:34.521785+0000","last_fullsized":"2024-09-14T06:18:34.521785+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:26:30.150015+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.047023011000000003,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2483,"num_read_kb":16488,"num_write":510,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":6785399,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"122'432","reported_seq":3823,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739366+0000","last_change":"2024-09-14T06:18:17.226299+0000","last_active":"2024-09-14T06:18:34.739366+0000","last_peered":"2024-09-14T06:18:34.739366+0000","last_clean":"2024-09-14T06:18:34.739366+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:18:34.739366+0000","last_undegraded":"2024-09-14T06:18:34.739366+0000","last_fullsized":"2024-09-14T06:18:34.739366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:42:58.536316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094053699999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4134,"num_read_kb":64629,"num_write":506,"num_write_kb":22219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"123'308","reported_seq":2112,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522107+0000","last_change":"2024-09-14T06:18:17.333655+0000","last_active":"2024-09-14T06:18:34.522107+0000","last_peered":"2024-09-14T06:18:34.522107+0000","last_clean":"2024-09-14T06:18:34.522107+0000","last_became_active":"2024-09-14T06:17:37.231244+0000","last_became_peered":"2024-09-14T06:17:37.231244+0000","last_unstale":"2024-09-14T06:18:34.522107+0000","last_undegraded":"2024-09-14T06:18:34.522107+0000","last_fullsized":"2024-09-14T06:18:34.522107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:10.356294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044577879999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2916,"num_read_kb":27195,"num_write":273,"num_write_kb":15734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":32506492,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":123,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522222+0000","last_change":"2024-09-14T06:17:34.200709+0000","last_active":"2024-09-14T06:18:34.522222+0000","last_peered":"2024-09-14T06:18:34.522222+0000","last_clean":"2024-09-14T06:18:34.522222+0000","last_became_active":"2024-09-14T06:17:34.200238+0000","last_became_peered":"2024-09-14T06:17:34.200238+0000","last_unstale":"2024-09-14T06:18:34.522222+0000","last_undegraded":"2024-09-14T06:18:34.522222+0000","last_fullsized":"2024-09-14T06:18:34.522222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:34.864695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"123'487","reported_seq":1380,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521933+0000","last_change":"2024-09-14T06:18:17.229782+0000","last_active":"2024-09-14T06:18:34.521933+0000","last_peered":"2024-09-14T06:18:34.521933+0000","last_clean":"2024-09-14T06:18:34.521933+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:18:34.521933+0000","last_undegraded":"2024-09-14T06:18:34.521933+0000","last_fullsized":"2024-09-14T06:18:34.521933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:58:18.826180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030371690000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1562,"num_read_kb":9726,"num_write":495,"num_write_kb":9169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":253,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521995+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:18:34.521995+0000","last_peered":"2024-09-14T06:18:34.521995+0000","last_clean":"2024-09-14T06:18:34.521995+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:18:34.521995+0000","last_undegraded":"2024-09-14T06:18:34.521995+0000","last_fullsized":"2024-09-14T06:18:34.521995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"123'382","reported_seq":3317,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522136+0000","last_change":"2024-09-14T06:18:17.229810+0000","last_active":"2024-09-14T06:18:34.522136+0000","last_peered":"2024-09-14T06:18:34.522136+0000","last_clean":"2024-09-14T06:18:34.522136+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:18:34.522136+0000","last_undegraded":"2024-09-14T06:18:34.522136+0000","last_fullsized":"2024-09-14T06:18:34.522136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:45:12.946377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030864040000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3293,"num_read_kb":28583,"num_write":338,"num_write_kb":15237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":123,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522548+0000","last_change":"2024-09-14T06:17:34.196668+0000","last_active":"2024-09-14T06:18:34.522548+0000","last_peered":"2024-09-14T06:18:34.522548+0000","last_clean":"2024-09-14T06:18:34.522548+0000","last_became_active":"2024-09-14T06:17:34.196072+0000","last_became_peered":"2024-09-14T06:17:34.196072+0000","last_unstale":"2024-09-14T06:18:34.522548+0000","last_undegraded":"2024-09-14T06:18:34.522548+0000","last_fullsized":"2024-09-14T06:18:34.522548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:18:45.616834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"123'369","reported_seq":2035,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739803+0000","last_change":"2024-09-14T06:18:17.300971+0000","last_active":"2024-09-14T06:18:34.739803+0000","last_peered":"2024-09-14T06:18:34.739803+0000","last_clean":"2024-09-14T06:18:34.739803+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:18:34.739803+0000","last_undegraded":"2024-09-14T06:18:34.739803+0000","last_fullsized":"2024-09-14T06:18:34.739803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:05.035879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018818730000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1780,"num_read_kb":12934,"num_write":307,"num_write_kb":11582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":255,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740225+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:18:34.740225+0000","last_peered":"2024-09-14T06:18:34.740225+0000","last_clean":"2024-09-14T06:18:34.740225+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:18:34.740225+0000","last_undegraded":"2024-09-14T06:18:34.740225+0000","last_fullsized":"2024-09-14T06:18:34.740225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"123'272","reported_seq":1766,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521495+0000","last_change":"2024-09-14T06:18:25.223649+0000","last_active":"2024-09-14T06:18:34.521495+0000","last_peered":"2024-09-14T06:18:34.521495+0000","last_clean":"2024-09-14T06:18:34.521495+0000","last_became_active":"2024-09-14T06:18:25.221823+0000","last_became_peered":"2024-09-14T06:18:25.221823+0000","last_unstale":"2024-09-14T06:18:34.521495+0000","last_undegraded":"2024-09-14T06:18:34.521495+0000","last_fullsized":"2024-09-14T06:18:34.521495+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:35:13.386184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019482589999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1362,"num_read_kb":21261,"num_write":264,"num_write_kb":9954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"123'394","reported_seq":1248,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739512+0000","last_change":"2024-09-14T06:18:17.226968+0000","last_active":"2024-09-14T06:18:34.739512+0000","last_peered":"2024-09-14T06:18:34.739512+0000","last_clean":"2024-09-14T06:18:34.739512+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:18:34.739512+0000","last_undegraded":"2024-09-14T06:18:34.739512+0000","last_fullsized":"2024-09-14T06:18:34.739512+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:39:35.192039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001556469,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":547,"num_read_kb":4255,"num_write":315,"num_write_kb":7133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":127,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520864+0000","last_change":"2024-09-14T06:17:34.203977+0000","last_active":"2024-09-14T06:18:34.520864+0000","last_peered":"2024-09-14T06:18:34.520864+0000","last_clean":"2024-09-14T06:18:34.520864+0000","last_became_active":"2024-09-14T06:17:34.199244+0000","last_became_peered":"2024-09-14T06:17:34.199244+0000","last_unstale":"2024-09-14T06:18:34.520864+0000","last_undegraded":"2024-09-14T06:18:34.520864+0000","last_fullsized":"2024-09-14T06:18:34.520864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:21.934094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"123'407","reported_seq":2036,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521548+0000","last_change":"2024-09-14T06:18:17.238754+0000","last_active":"2024-09-14T06:18:34.521548+0000","last_peered":"2024-09-14T06:18:34.521548+0000","last_clean":"2024-09-14T06:18:34.521548+0000","last_became_active":"2024-09-14T06:18:17.230545+0000","last_became_peered":"2024-09-14T06:18:17.230545+0000","last_unstale":"2024-09-14T06:18:34.521548+0000","last_undegraded":"2024-09-14T06:18:34.521548+0000","last_fullsized":"2024-09-14T06:18:34.521548+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:17:11.336110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048632429999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2097,"num_read_kb":18969,"num_write":521,"num_write_kb":11051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":36437,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":254,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739790+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:18:34.739790+0000","last_peered":"2024-09-14T06:18:34.739790+0000","last_clean":"2024-09-14T06:18:34.739790+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:18:34.739790+0000","last_undegraded":"2024-09-14T06:18:34.739790+0000","last_fullsized":"2024-09-14T06:18:34.739790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"117'540","reported_seq":2241,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739370+0000","last_change":"2024-09-14T06:18:17.300145+0000","last_active":"2024-09-14T06:18:34.739370+0000","last_peered":"2024-09-14T06:18:34.739370+0000","last_clean":"2024-09-14T06:18:34.739370+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:18:34.739370+0000","last_undegraded":"2024-09-14T06:18:34.739370+0000","last_fullsized":"2024-09-14T06:18:34.739370+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:25.627665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001640601,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2885,"num_read_kb":8812,"num_write":721,"num_write_kb":8784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":253,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520702+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:18:34.520702+0000","last_peered":"2024-09-14T06:18:34.520702+0000","last_clean":"2024-09-14T06:18:34.520702+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:18:34.520702+0000","last_undegraded":"2024-09-14T06:18:34.520702+0000","last_fullsized":"2024-09-14T06:18:34.520702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"122'471","reported_seq":2078,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521629+0000","last_change":"2024-09-14T06:18:17.227668+0000","last_active":"2024-09-14T06:18:34.521629+0000","last_peered":"2024-09-14T06:18:34.521629+0000","last_clean":"2024-09-14T06:18:34.521629+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:18:34.521629+0000","last_undegraded":"2024-09-14T06:18:34.521629+0000","last_fullsized":"2024-09-14T06:18:34.521629+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:19:40.041408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001487162,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2220,"num_read_kb":23013,"num_write":375,"num_write_kb":14523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":274,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522087+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:18:34.522087+0000","last_peered":"2024-09-14T06:18:34.522087+0000","last_clean":"2024-09-14T06:18:34.522087+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:18:34.522087+0000","last_undegraded":"2024-09-14T06:18:34.522087+0000","last_fullsized":"2024-09-14T06:18:34.522087+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":345,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738574+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:18:34.738574+0000","last_peered":"2024-09-14T06:18:34.738574+0000","last_clean":"2024-09-14T06:18:34.738574+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:18:34.738574+0000","last_undegraded":"2024-09-14T06:18:34.738574+0000","last_fullsized":"2024-09-14T06:18:34.738574+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"122'442","reported_seq":1709,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521572+0000","last_change":"2024-09-14T06:18:17.230742+0000","last_active":"2024-09-14T06:18:34.521572+0000","last_peered":"2024-09-14T06:18:34.521572+0000","last_clean":"2024-09-14T06:18:34.521572+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:18:34.521572+0000","last_undegraded":"2024-09-14T06:18:34.521572+0000","last_fullsized":"2024-09-14T06:18:34.521572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:58:35.960390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00348241,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2133,"num_read_kb":6715,"num_write":589,"num_write_kb":6849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"122'398","reported_seq":1904,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520702+0000","last_change":"2024-09-14T06:18:17.336435+0000","last_active":"2024-09-14T06:18:34.520702+0000","last_peered":"2024-09-14T06:18:34.520702+0000","last_clean":"2024-09-14T06:18:34.520702+0000","last_became_active":"2024-09-14T06:17:38.224238+0000","last_became_peered":"2024-09-14T06:17:38.224238+0000","last_unstale":"2024-09-14T06:18:34.520702+0000","last_undegraded":"2024-09-14T06:18:34.520702+0000","last_fullsized":"2024-09-14T06:18:34.520702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:37.628302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064732729999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2223,"num_read_kb":13651,"num_write":549,"num_write_kb":14353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8477378,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"123'384","reported_seq":2444,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521434+0000","last_change":"2024-09-14T06:18:17.336082+0000","last_active":"2024-09-14T06:18:34.521434+0000","last_peered":"2024-09-14T06:18:34.521434+0000","last_clean":"2024-09-14T06:18:34.521434+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:18:34.521434+0000","last_undegraded":"2024-09-14T06:18:34.521434+0000","last_fullsized":"2024-09-14T06:18:34.521434+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:58.737457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005800637,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":2425,"num_read_kb":16285,"num_write":322,"num_write_kb":13807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"123'318","reported_seq":2937,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739738+0000","last_change":"2024-09-14T06:18:17.236905+0000","last_active":"2024-09-14T06:18:34.739738+0000","last_peered":"2024-09-14T06:18:34.739738+0000","last_clean":"2024-09-14T06:18:34.739738+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:18:34.739738+0000","last_undegraded":"2024-09-14T06:18:34.739738+0000","last_fullsized":"2024-09-14T06:18:34.739738+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:57:26.637373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013408000000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2870,"num_read_kb":16793,"num_write":272,"num_write_kb":9901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"123'189","reported_seq":1698,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739303+0000","last_change":"2024-09-14T06:18:17.226333+0000","last_active":"2024-09-14T06:18:34.739303+0000","last_peered":"2024-09-14T06:18:34.739303+0000","last_clean":"2024-09-14T06:18:34.739303+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:18:34.739303+0000","last_undegraded":"2024-09-14T06:18:34.739303+0000","last_fullsized":"2024-09-14T06:18:34.739303+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:57:04.903011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010333339999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1287,"num_read_kb":29081,"num_write":248,"num_write_kb":11577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"122'687","reported_seq":2640,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522423+0000","last_change":"2024-09-14T06:18:17.337571+0000","last_active":"2024-09-14T06:18:34.522423+0000","last_peered":"2024-09-14T06:18:34.522423+0000","last_clean":"2024-09-14T06:18:34.522423+0000","last_became_active":"2024-09-14T06:17:38.227106+0000","last_became_peered":"2024-09-14T06:17:38.227106+0000","last_unstale":"2024-09-14T06:18:34.522423+0000","last_undegraded":"2024-09-14T06:18:34.522423+0000","last_fullsized":"2024-09-14T06:18:34.522423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:27:58.009450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006423425,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3768,"num_read_kb":26949,"num_write":994,"num_write_kb":20276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":29390214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"123'762","reported_seq":5455,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521683+0000","last_change":"2024-09-14T06:18:25.224296+0000","last_active":"2024-09-14T06:18:34.521683+0000","last_peered":"2024-09-14T06:18:34.521683+0000","last_clean":"2024-09-14T06:18:34.521683+0000","last_became_active":"2024-09-14T06:18:25.223198+0000","last_became_peered":"2024-09-14T06:18:25.223198+0000","last_unstale":"2024-09-14T06:18:34.521683+0000","last_undegraded":"2024-09-14T06:18:34.521683+0000","last_fullsized":"2024-09-14T06:18:34.521683+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:33:35.589616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021565500000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10255,"num_read_kb":51706,"num_write":697,"num_write_kb":16134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"123'661","reported_seq":2988,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739487+0000","last_change":"2024-09-14T06:18:17.226919+0000","last_active":"2024-09-14T06:18:34.739487+0000","last_peered":"2024-09-14T06:18:34.739487+0000","last_clean":"2024-09-14T06:18:34.739487+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:18:34.739487+0000","last_undegraded":"2024-09-14T06:18:34.739487+0000","last_fullsized":"2024-09-14T06:18:34.739487+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:55:17.915614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001432902,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4425,"num_read_kb":16166,"num_write":768,"num_write_kb":12749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"122'360","reported_seq":2605,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738517+0000","last_change":"2024-09-14T06:18:17.221410+0000","last_active":"2024-09-14T06:18:34.738517+0000","last_peered":"2024-09-14T06:18:34.738517+0000","last_clean":"2024-09-14T06:18:34.738517+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:18:34.738517+0000","last_undegraded":"2024-09-14T06:18:34.738517+0000","last_fullsized":"2024-09-14T06:18:34.738517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:24:43.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068876799999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2476,"num_read_kb":47684,"num_write":341,"num_write_kb":14643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"123'556","reported_seq":3001,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521007+0000","last_change":"2024-09-14T06:18:17.332968+0000","last_active":"2024-09-14T06:18:34.521007+0000","last_peered":"2024-09-14T06:18:34.521007+0000","last_clean":"2024-09-14T06:18:34.521007+0000","last_became_active":"2024-09-14T06:17:38.224251+0000","last_became_peered":"2024-09-14T06:17:38.224251+0000","last_unstale":"2024-09-14T06:18:34.521007+0000","last_undegraded":"2024-09-14T06:18:34.521007+0000","last_fullsized":"2024-09-14T06:18:34.521007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:32:17.880108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033873589999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4434,"num_read_kb":14632,"num_write":613,"num_write_kb":7670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":12239008,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"122'518","reported_seq":3082,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739545+0000","last_change":"2024-09-14T06:18:17.300159+0000","last_active":"2024-09-14T06:18:34.739545+0000","last_peered":"2024-09-14T06:18:34.739545+0000","last_clean":"2024-09-14T06:18:34.739545+0000","last_became_active":"2024-09-14T06:17:37.232145+0000","last_became_peered":"2024-09-14T06:17:37.232145+0000","last_unstale":"2024-09-14T06:18:34.739545+0000","last_undegraded":"2024-09-14T06:18:34.739545+0000","last_fullsized":"2024-09-14T06:18:34.739545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:12:47.470446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016513039999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3345,"num_read_kb":11048,"num_write":606,"num_write_kb":8456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":10176554,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"122'505","reported_seq":4220,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521410+0000","last_change":"2024-09-14T06:18:17.333610+0000","last_active":"2024-09-14T06:18:34.521410+0000","last_peered":"2024-09-14T06:18:34.521410+0000","last_clean":"2024-09-14T06:18:34.521410+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:18:34.521410+0000","last_undegraded":"2024-09-14T06:18:34.521410+0000","last_fullsized":"2024-09-14T06:18:34.521410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:51:31.054844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039616110000000003,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6037,"num_read_kb":16850,"num_write":431,"num_write_kb":5588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":210,"num_object_clones":0,"num_object_copies":420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210,"num_whiteouts":206,"num_read":107256,"num_read_kb":751154,"num_write":17465,"num_write_kb":429271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1267,"num_bytes_recovered":144403203,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15570,"ondisk_log_size":15570,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509475,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55028,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316812,"statfs":{"total":96636764160,"available":96580415488,"internally_reserved":0,"allocated":1662976,"data_stored":789529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36972,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334868,"statfs":{"total":96636764160,"available":96598904832,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509477,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73132,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298708,"statfs":{"total":96636764160,"available":96561876992,"internally_reserved":0,"allocated":1724416,"data_stored":1379897,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509476,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59184,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312656,"statfs":{"total":96636764160,"available":96576159744,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542179,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36972,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334868,"statfs":{"total":96636764160,"available":96598904832,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28652,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343188,"statfs":{"total":96636764160,"available":96607424512,"internally_reserved":0,"allocated":1662976,"data_stored":789529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542178,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28716,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343124,"statfs":{"total":96636764160,"available":96607358976,"internally_reserved":0,"allocated":1724416,"data_stored":1379897,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542180,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28652,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343188,"statfs":{"total":96636764160,"available":96607424512,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:18:39.041 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:18:39.337 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:18:39.338 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:18:39.354 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-14T06:18:38.420127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":214,"num_object_clones":0,"num_object_copies":428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":214,"num_whiteouts":206,"num_read":107322,"num_read_kb":751209,"num_write":17568,"num_write_kb":430901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1270,"num_bytes_recovered":144403222,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15645,"ondisk_log_size":15645,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":347308,"kb_used_data":13096,"kb_used_omap":12,"kb_used_meta":334003,"kb_avail":754627412,"statfs":{"total":773094113280,"available":772738469888,"internally_reserved":0,"allocated":13410304,"data_stored":7496892,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":342019632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001830"},"pg_stats":[{"pgid":"3.1f","version":"123'604","reported_seq":3721,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740380+0000","last_change":"2024-09-14T06:18:17.225123+0000","last_active":"2024-09-14T06:18:34.740380+0000","last_peered":"2024-09-14T06:18:34.740380+0000","last_clean":"2024-09-14T06:18:34.740380+0000","last_became_active":"2024-09-14T06:17:02.193313+0000","last_became_peered":"2024-09-14T06:17:02.193313+0000","last_unstale":"2024-09-14T06:18:34.740380+0000","last_undegraded":"2024-09-14T06:18:34.740380+0000","last_fullsized":"2024-09-14T06:18:34.740380+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:43:51.113692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002155396,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4617,"num_read_kb":53605,"num_write":714,"num_write_kb":19355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"122'407","reported_seq":4554,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739967+0000","last_change":"2024-09-14T06:18:17.224253+0000","last_active":"2024-09-14T06:18:34.739967+0000","last_peered":"2024-09-14T06:18:34.739967+0000","last_clean":"2024-09-14T06:18:34.739967+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:18:34.739967+0000","last_undegraded":"2024-09-14T06:18:34.739967+0000","last_fullsized":"2024-09-14T06:18:34.739967+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:46:26.694964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019300890000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5022,"num_read_kb":43064,"num_write":350,"num_write_kb":15485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"123'1132","reported_seq":2556,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522291+0000","last_change":"2024-09-14T06:18:17.337001+0000","last_active":"2024-09-14T06:18:34.522291+0000","last_peered":"2024-09-14T06:18:34.522291+0000","last_clean":"2024-09-14T06:18:34.522291+0000","last_became_active":"2024-09-14T06:17:38.226838+0000","last_became_peered":"2024-09-14T06:17:38.226838+0000","last_unstale":"2024-09-14T06:18:34.522291+0000","last_undegraded":"2024-09-14T06:18:34.522291+0000","last_fullsized":"2024-09-14T06:18:34.522291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":1132,"log_dups_size":0,"ondisk_log_size":1132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:01.176538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062318570000000004,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5125,"num_read_kb":14378,"num_write":1577,"num_write_kb":13994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10785266,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"123'934","reported_seq":3133,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521839+0000","last_change":"2024-09-14T06:18:17.228603+0000","last_active":"2024-09-14T06:18:34.521839+0000","last_peered":"2024-09-14T06:18:34.521839+0000","last_clean":"2024-09-14T06:18:34.521839+0000","last_became_active":"2024-09-14T06:17:48.737020+0000","last_became_peered":"2024-09-14T06:17:48.737020+0000","last_unstale":"2024-09-14T06:18:34.521839+0000","last_undegraded":"2024-09-14T06:18:34.521839+0000","last_fullsized":"2024-09-14T06:18:34.521839+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:04:33.297278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024240709999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":6011,"num_read_kb":10015,"num_write":1345,"num_write_kb":6524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":3869232,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"123'349","reported_seq":3834,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521767+0000","last_change":"2024-09-14T06:18:17.228692+0000","last_active":"2024-09-14T06:18:34.521767+0000","last_peered":"2024-09-14T06:18:34.521767+0000","last_clean":"2024-09-14T06:18:34.521767+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:18:34.521767+0000","last_undegraded":"2024-09-14T06:18:34.521767+0000","last_fullsized":"2024-09-14T06:18:34.521767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:40:01.686387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024977549999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3342,"num_read_kb":39056,"num_write":366,"num_write_kb":25723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"122'539","reported_seq":3387,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738796+0000","last_change":"2024-09-14T06:18:17.221967+0000","last_active":"2024-09-14T06:18:34.738796+0000","last_peered":"2024-09-14T06:18:34.738796+0000","last_clean":"2024-09-14T06:18:34.738796+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:18:34.738796+0000","last_undegraded":"2024-09-14T06:18:34.738796+0000","last_fullsized":"2024-09-14T06:18:34.738796+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:05:11.528071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087057599999999997,"stat_sum":{"num_bytes":19,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":4088,"num_read_kb":13128,"num_write":658,"num_write_kb":7511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"122'532","reported_seq":2187,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521641+0000","last_change":"2024-09-14T06:18:17.227798+0000","last_active":"2024-09-14T06:18:34.521641+0000","last_peered":"2024-09-14T06:18:34.521641+0000","last_clean":"2024-09-14T06:18:34.521641+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:18:34.521641+0000","last_undegraded":"2024-09-14T06:18:34.521641+0000","last_fullsized":"2024-09-14T06:18:34.521641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:09:43.212330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001716539,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2883,"num_read_kb":19234,"num_write":704,"num_write_kb":12873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"123'400","reported_seq":3103,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740079+0000","last_change":"2024-09-14T06:18:17.225653+0000","last_active":"2024-09-14T06:18:34.740079+0000","last_peered":"2024-09-14T06:18:34.740079+0000","last_clean":"2024-09-14T06:18:34.740079+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:18:34.740079+0000","last_undegraded":"2024-09-14T06:18:34.740079+0000","last_fullsized":"2024-09-14T06:18:34.740079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:38:39.952754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002524938,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":3795,"num_read_kb":37772,"num_write":360,"num_write_kb":32941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"119'395","reported_seq":1793,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740431+0000","last_change":"2024-09-14T06:18:17.224547+0000","last_active":"2024-09-14T06:18:34.740431+0000","last_peered":"2024-09-14T06:18:34.740431+0000","last_clean":"2024-09-14T06:18:34.740431+0000","last_became_active":"2024-09-14T06:17:37.233410+0000","last_became_peered":"2024-09-14T06:17:37.233410+0000","last_unstale":"2024-09-14T06:18:34.740431+0000","last_undegraded":"2024-09-14T06:18:34.740431+0000","last_fullsized":"2024-09-14T06:18:34.740431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:33.122557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019943040000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1416,"num_read_kb":17477,"num_write":336,"num_write_kb":17193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":23467377,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"122'436","reported_seq":2420,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521785+0000","last_change":"2024-09-14T06:18:25.223056+0000","last_active":"2024-09-14T06:18:34.521785+0000","last_peered":"2024-09-14T06:18:34.521785+0000","last_clean":"2024-09-14T06:18:34.521785+0000","last_became_active":"2024-09-14T06:18:25.219200+0000","last_became_peered":"2024-09-14T06:18:25.219200+0000","last_unstale":"2024-09-14T06:18:34.521785+0000","last_undegraded":"2024-09-14T06:18:34.521785+0000","last_fullsized":"2024-09-14T06:18:34.521785+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:26:30.150015+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.047023011000000003,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2483,"num_read_kb":16488,"num_write":510,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":6785399,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"122'432","reported_seq":3823,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739366+0000","last_change":"2024-09-14T06:18:17.226299+0000","last_active":"2024-09-14T06:18:34.739366+0000","last_peered":"2024-09-14T06:18:34.739366+0000","last_clean":"2024-09-14T06:18:34.739366+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:18:34.739366+0000","last_undegraded":"2024-09-14T06:18:34.739366+0000","last_fullsized":"2024-09-14T06:18:34.739366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:42:58.536316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094053699999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4134,"num_read_kb":64629,"num_write":506,"num_write_kb":22219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"123'308","reported_seq":2112,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522107+0000","last_change":"2024-09-14T06:18:17.333655+0000","last_active":"2024-09-14T06:18:34.522107+0000","last_peered":"2024-09-14T06:18:34.522107+0000","last_clean":"2024-09-14T06:18:34.522107+0000","last_became_active":"2024-09-14T06:17:37.231244+0000","last_became_peered":"2024-09-14T06:17:37.231244+0000","last_unstale":"2024-09-14T06:18:34.522107+0000","last_undegraded":"2024-09-14T06:18:34.522107+0000","last_fullsized":"2024-09-14T06:18:34.522107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:10.356294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044577879999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2916,"num_read_kb":27195,"num_write":273,"num_write_kb":15734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":32506492,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":123,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522222+0000","last_change":"2024-09-14T06:17:34.200709+0000","last_active":"2024-09-14T06:18:34.522222+0000","last_peered":"2024-09-14T06:18:34.522222+0000","last_clean":"2024-09-14T06:18:34.522222+0000","last_became_active":"2024-09-14T06:17:34.200238+0000","last_became_peered":"2024-09-14T06:17:34.200238+0000","last_unstale":"2024-09-14T06:18:34.522222+0000","last_undegraded":"2024-09-14T06:18:34.522222+0000","last_fullsized":"2024-09-14T06:18:34.522222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:34.864695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"123'487","reported_seq":1380,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521933+0000","last_change":"2024-09-14T06:18:17.229782+0000","last_active":"2024-09-14T06:18:34.521933+0000","last_peered":"2024-09-14T06:18:34.521933+0000","last_clean":"2024-09-14T06:18:34.521933+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:18:34.521933+0000","last_undegraded":"2024-09-14T06:18:34.521933+0000","last_fullsized":"2024-09-14T06:18:34.521933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:58:18.826180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030371690000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1562,"num_read_kb":9726,"num_write":495,"num_write_kb":9169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":253,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521995+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:18:34.521995+0000","last_peered":"2024-09-14T06:18:34.521995+0000","last_clean":"2024-09-14T06:18:34.521995+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:18:34.521995+0000","last_undegraded":"2024-09-14T06:18:34.521995+0000","last_fullsized":"2024-09-14T06:18:34.521995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"123'382","reported_seq":3317,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522136+0000","last_change":"2024-09-14T06:18:17.229810+0000","last_active":"2024-09-14T06:18:34.522136+0000","last_peered":"2024-09-14T06:18:34.522136+0000","last_clean":"2024-09-14T06:18:34.522136+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:18:34.522136+0000","last_undegraded":"2024-09-14T06:18:34.522136+0000","last_fullsized":"2024-09-14T06:18:34.522136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:45:12.946377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030864040000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3293,"num_read_kb":28583,"num_write":338,"num_write_kb":15237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":123,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522548+0000","last_change":"2024-09-14T06:17:34.196668+0000","last_active":"2024-09-14T06:18:34.522548+0000","last_peered":"2024-09-14T06:18:34.522548+0000","last_clean":"2024-09-14T06:18:34.522548+0000","last_became_active":"2024-09-14T06:17:34.196072+0000","last_became_peered":"2024-09-14T06:17:34.196072+0000","last_unstale":"2024-09-14T06:18:34.522548+0000","last_undegraded":"2024-09-14T06:18:34.522548+0000","last_fullsized":"2024-09-14T06:18:34.522548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:18:45.616834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"123'369","reported_seq":2035,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739803+0000","last_change":"2024-09-14T06:18:17.300971+0000","last_active":"2024-09-14T06:18:34.739803+0000","last_peered":"2024-09-14T06:18:34.739803+0000","last_clean":"2024-09-14T06:18:34.739803+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:18:34.739803+0000","last_undegraded":"2024-09-14T06:18:34.739803+0000","last_fullsized":"2024-09-14T06:18:34.739803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:05.035879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018818730000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1780,"num_read_kb":12934,"num_write":307,"num_write_kb":11582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":255,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740225+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:18:34.740225+0000","last_peered":"2024-09-14T06:18:34.740225+0000","last_clean":"2024-09-14T06:18:34.740225+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:18:34.740225+0000","last_undegraded":"2024-09-14T06:18:34.740225+0000","last_fullsized":"2024-09-14T06:18:34.740225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"123'272","reported_seq":1766,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521495+0000","last_change":"2024-09-14T06:18:25.223649+0000","last_active":"2024-09-14T06:18:34.521495+0000","last_peered":"2024-09-14T06:18:34.521495+0000","last_clean":"2024-09-14T06:18:34.521495+0000","last_became_active":"2024-09-14T06:18:25.221823+0000","last_became_peered":"2024-09-14T06:18:25.221823+0000","last_unstale":"2024-09-14T06:18:34.521495+0000","last_undegraded":"2024-09-14T06:18:34.521495+0000","last_fullsized":"2024-09-14T06:18:34.521495+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:35:13.386184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019482589999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1362,"num_read_kb":21261,"num_write":264,"num_write_kb":9954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"123'394","reported_seq":1248,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739512+0000","last_change":"2024-09-14T06:18:17.226968+0000","last_active":"2024-09-14T06:18:34.739512+0000","last_peered":"2024-09-14T06:18:34.739512+0000","last_clean":"2024-09-14T06:18:34.739512+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:18:34.739512+0000","last_undegraded":"2024-09-14T06:18:34.739512+0000","last_fullsized":"2024-09-14T06:18:34.739512+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:39:35.192039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001556469,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":547,"num_read_kb":4255,"num_write":315,"num_write_kb":7133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":127,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520864+0000","last_change":"2024-09-14T06:17:34.203977+0000","last_active":"2024-09-14T06:18:34.520864+0000","last_peered":"2024-09-14T06:18:34.520864+0000","last_clean":"2024-09-14T06:18:34.520864+0000","last_became_active":"2024-09-14T06:17:34.199244+0000","last_became_peered":"2024-09-14T06:17:34.199244+0000","last_unstale":"2024-09-14T06:18:34.520864+0000","last_undegraded":"2024-09-14T06:18:34.520864+0000","last_fullsized":"2024-09-14T06:18:34.520864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:21.934094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"123'407","reported_seq":2036,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521548+0000","last_change":"2024-09-14T06:18:17.238754+0000","last_active":"2024-09-14T06:18:34.521548+0000","last_peered":"2024-09-14T06:18:34.521548+0000","last_clean":"2024-09-14T06:18:34.521548+0000","last_became_active":"2024-09-14T06:18:17.230545+0000","last_became_peered":"2024-09-14T06:18:17.230545+0000","last_unstale":"2024-09-14T06:18:34.521548+0000","last_undegraded":"2024-09-14T06:18:34.521548+0000","last_fullsized":"2024-09-14T06:18:34.521548+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:17:11.336110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048632429999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2097,"num_read_kb":18969,"num_write":521,"num_write_kb":11051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":36437,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":254,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739790+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:18:34.739790+0000","last_peered":"2024-09-14T06:18:34.739790+0000","last_clean":"2024-09-14T06:18:34.739790+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:18:34.739790+0000","last_undegraded":"2024-09-14T06:18:34.739790+0000","last_fullsized":"2024-09-14T06:18:34.739790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"117'540","reported_seq":2241,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739370+0000","last_change":"2024-09-14T06:18:17.300145+0000","last_active":"2024-09-14T06:18:34.739370+0000","last_peered":"2024-09-14T06:18:34.739370+0000","last_clean":"2024-09-14T06:18:34.739370+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:18:34.739370+0000","last_undegraded":"2024-09-14T06:18:34.739370+0000","last_fullsized":"2024-09-14T06:18:34.739370+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:25.627665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001640601,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2885,"num_read_kb":8812,"num_write":721,"num_write_kb":8784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":253,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520702+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:18:34.520702+0000","last_peered":"2024-09-14T06:18:34.520702+0000","last_clean":"2024-09-14T06:18:34.520702+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:18:34.520702+0000","last_undegraded":"2024-09-14T06:18:34.520702+0000","last_fullsized":"2024-09-14T06:18:34.520702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"122'471","reported_seq":2078,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521629+0000","last_change":"2024-09-14T06:18:17.227668+0000","last_active":"2024-09-14T06:18:34.521629+0000","last_peered":"2024-09-14T06:18:34.521629+0000","last_clean":"2024-09-14T06:18:34.521629+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:18:34.521629+0000","last_undegraded":"2024-09-14T06:18:34.521629+0000","last_fullsized":"2024-09-14T06:18:34.521629+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:19:40.041408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001487162,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2220,"num_read_kb":23013,"num_write":375,"num_write_kb":14523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":274,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522087+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:18:34.522087+0000","last_peered":"2024-09-14T06:18:34.522087+0000","last_clean":"2024-09-14T06:18:34.522087+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:18:34.522087+0000","last_undegraded":"2024-09-14T06:18:34.522087+0000","last_fullsized":"2024-09-14T06:18:34.522087+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":345,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738574+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:18:34.738574+0000","last_peered":"2024-09-14T06:18:34.738574+0000","last_clean":"2024-09-14T06:18:34.738574+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:18:34.738574+0000","last_undegraded":"2024-09-14T06:18:34.738574+0000","last_fullsized":"2024-09-14T06:18:34.738574+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"122'442","reported_seq":1709,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521572+0000","last_change":"2024-09-14T06:18:17.230742+0000","last_active":"2024-09-14T06:18:34.521572+0000","last_peered":"2024-09-14T06:18:34.521572+0000","last_clean":"2024-09-14T06:18:34.521572+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:18:34.521572+0000","last_undegraded":"2024-09-14T06:18:34.521572+0000","last_fullsized":"2024-09-14T06:18:34.521572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:58:35.960390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00348241,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2133,"num_read_kb":6715,"num_write":589,"num_write_kb":6849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"122'398","reported_seq":1904,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520702+0000","last_change":"2024-09-14T06:18:17.336435+0000","last_active":"2024-09-14T06:18:34.520702+0000","last_peered":"2024-09-14T06:18:34.520702+0000","last_clean":"2024-09-14T06:18:34.520702+0000","last_became_active":"2024-09-14T06:17:38.224238+0000","last_became_peered":"2024-09-14T06:17:38.224238+0000","last_unstale":"2024-09-14T06:18:34.520702+0000","last_undegraded":"2024-09-14T06:18:34.520702+0000","last_fullsized":"2024-09-14T06:18:34.520702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:37.628302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064732729999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2223,"num_read_kb":13651,"num_write":549,"num_write_kb":14353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8477378,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"123'384","reported_seq":2444,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521434+0000","last_change":"2024-09-14T06:18:17.336082+0000","last_active":"2024-09-14T06:18:34.521434+0000","last_peered":"2024-09-14T06:18:34.521434+0000","last_clean":"2024-09-14T06:18:34.521434+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:18:34.521434+0000","last_undegraded":"2024-09-14T06:18:34.521434+0000","last_fullsized":"2024-09-14T06:18:34.521434+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:58.737457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005800637,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":2425,"num_read_kb":16285,"num_write":322,"num_write_kb":13807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"123'318","reported_seq":2937,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739738+0000","last_change":"2024-09-14T06:18:17.236905+0000","last_active":"2024-09-14T06:18:34.739738+0000","last_peered":"2024-09-14T06:18:34.739738+0000","last_clean":"2024-09-14T06:18:34.739738+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:18:34.739738+0000","last_undegraded":"2024-09-14T06:18:34.739738+0000","last_fullsized":"2024-09-14T06:18:34.739738+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:57:26.637373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013408000000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2870,"num_read_kb":16793,"num_write":272,"num_write_kb":9901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"123'189","reported_seq":1698,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739303+0000","last_change":"2024-09-14T06:18:17.226333+0000","last_active":"2024-09-14T06:18:34.739303+0000","last_peered":"2024-09-14T06:18:34.739303+0000","last_clean":"2024-09-14T06:18:34.739303+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:18:34.739303+0000","last_undegraded":"2024-09-14T06:18:34.739303+0000","last_fullsized":"2024-09-14T06:18:34.739303+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:57:04.903011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010333339999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1287,"num_read_kb":29081,"num_write":248,"num_write_kb":11577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"122'687","reported_seq":2640,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522423+0000","last_change":"2024-09-14T06:18:17.337571+0000","last_active":"2024-09-14T06:18:34.522423+0000","last_peered":"2024-09-14T06:18:34.522423+0000","last_clean":"2024-09-14T06:18:34.522423+0000","last_became_active":"2024-09-14T06:17:38.227106+0000","last_became_peered":"2024-09-14T06:17:38.227106+0000","last_unstale":"2024-09-14T06:18:34.522423+0000","last_undegraded":"2024-09-14T06:18:34.522423+0000","last_fullsized":"2024-09-14T06:18:34.522423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:27:58.009450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006423425,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3768,"num_read_kb":26949,"num_write":994,"num_write_kb":20276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":29390214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"123'762","reported_seq":5455,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521683+0000","last_change":"2024-09-14T06:18:25.224296+0000","last_active":"2024-09-14T06:18:34.521683+0000","last_peered":"2024-09-14T06:18:34.521683+0000","last_clean":"2024-09-14T06:18:34.521683+0000","last_became_active":"2024-09-14T06:18:25.223198+0000","last_became_peered":"2024-09-14T06:18:25.223198+0000","last_unstale":"2024-09-14T06:18:34.521683+0000","last_undegraded":"2024-09-14T06:18:34.521683+0000","last_fullsized":"2024-09-14T06:18:34.521683+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:33:35.589616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021565500000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10255,"num_read_kb":51706,"num_write":697,"num_write_kb":16134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"123'661","reported_seq":2988,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739487+0000","last_change":"2024-09-14T06:18:17.226919+0000","last_active":"2024-09-14T06:18:34.739487+0000","last_peered":"2024-09-14T06:18:34.739487+0000","last_clean":"2024-09-14T06:18:34.739487+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:18:34.739487+0000","last_undegraded":"2024-09-14T06:18:34.739487+0000","last_fullsized":"2024-09-14T06:18:34.739487+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:55:17.915614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001432902,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4425,"num_read_kb":16166,"num_write":768,"num_write_kb":12749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"122'360","reported_seq":2605,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738517+0000","last_change":"2024-09-14T06:18:17.221410+0000","last_active":"2024-09-14T06:18:34.738517+0000","last_peered":"2024-09-14T06:18:34.738517+0000","last_clean":"2024-09-14T06:18:34.738517+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:18:34.738517+0000","last_undegraded":"2024-09-14T06:18:34.738517+0000","last_fullsized":"2024-09-14T06:18:34.738517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:24:43.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068876799999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2476,"num_read_kb":47684,"num_write":341,"num_write_kb":14643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"123'556","reported_seq":3001,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521007+0000","last_change":"2024-09-14T06:18:17.332968+0000","last_active":"2024-09-14T06:18:34.521007+0000","last_peered":"2024-09-14T06:18:34.521007+0000","last_clean":"2024-09-14T06:18:34.521007+0000","last_became_active":"2024-09-14T06:17:38.224251+0000","last_became_peered":"2024-09-14T06:17:38.224251+0000","last_unstale":"2024-09-14T06:18:34.521007+0000","last_undegraded":"2024-09-14T06:18:34.521007+0000","last_fullsized":"2024-09-14T06:18:34.521007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:32:17.880108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033873589999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4434,"num_read_kb":14632,"num_write":613,"num_write_kb":7670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":12239008,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"122'518","reported_seq":3082,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739545+0000","last_change":"2024-09-14T06:18:17.300159+0000","last_active":"2024-09-14T06:18:34.739545+0000","last_peered":"2024-09-14T06:18:34.739545+0000","last_clean":"2024-09-14T06:18:34.739545+0000","last_became_active":"2024-09-14T06:17:37.232145+0000","last_became_peered":"2024-09-14T06:17:37.232145+0000","last_unstale":"2024-09-14T06:18:34.739545+0000","last_undegraded":"2024-09-14T06:18:34.739545+0000","last_fullsized":"2024-09-14T06:18:34.739545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:12:47.470446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016513039999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3345,"num_read_kb":11048,"num_write":606,"num_write_kb":8456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":10176554,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"122'505","reported_seq":4220,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521410+0000","last_change":"2024-09-14T06:18:17.333610+0000","last_active":"2024-09-14T06:18:34.521410+0000","last_peered":"2024-09-14T06:18:34.521410+0000","last_clean":"2024-09-14T06:18:34.521410+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:18:34.521410+0000","last_undegraded":"2024-09-14T06:18:34.521410+0000","last_fullsized":"2024-09-14T06:18:34.521410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:51:31.054844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039616110000000003,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6037,"num_read_kb":16850,"num_write":431,"num_write_kb":5588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":210,"num_object_clones":0,"num_object_copies":420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210,"num_whiteouts":206,"num_read":107256,"num_read_kb":751154,"num_write":17465,"num_write_kb":429271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1267,"num_bytes_recovered":144403203,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15570,"ondisk_log_size":15570,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509475,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55028,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316812,"statfs":{"total":96636764160,"available":96580415488,"internally_reserved":0,"allocated":1662976,"data_stored":789529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36972,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334868,"statfs":{"total":96636764160,"available":96598904832,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509477,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73132,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298708,"statfs":{"total":96636764160,"available":96561876992,"internally_reserved":0,"allocated":1724416,"data_stored":1379897,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509476,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59184,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312656,"statfs":{"total":96636764160,"available":96576159744,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542179,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36972,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334868,"statfs":{"total":96636764160,"available":96598904832,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28652,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343188,"statfs":{"total":96636764160,"available":96607424512,"internally_reserved":0,"allocated":1662976,"data_stored":789529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542178,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28716,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343124,"statfs":{"total":96636764160,"available":96607358976,"internally_reserved":0,"allocated":1724416,"data_stored":1379897,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542180,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28652,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343188,"statfs":{"total":96636764160,"available":96607424512,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:18:39.357 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T06:18:39.357 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:18:39.657 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:18:39.657 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:18:39.674 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-14T06:18:38.420127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":214,"num_object_clones":0,"num_object_copies":428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":214,"num_whiteouts":206,"num_read":107322,"num_read_kb":751209,"num_write":17568,"num_write_kb":430901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1270,"num_bytes_recovered":144403222,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15645,"ondisk_log_size":15645,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":347308,"kb_used_data":13096,"kb_used_omap":12,"kb_used_meta":334003,"kb_avail":754627412,"statfs":{"total":773094113280,"available":772738469888,"internally_reserved":0,"allocated":13410304,"data_stored":7496892,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":342019632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001830"},"pg_stats":[{"pgid":"3.1f","version":"123'604","reported_seq":3721,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740380+0000","last_change":"2024-09-14T06:18:17.225123+0000","last_active":"2024-09-14T06:18:34.740380+0000","last_peered":"2024-09-14T06:18:34.740380+0000","last_clean":"2024-09-14T06:18:34.740380+0000","last_became_active":"2024-09-14T06:17:02.193313+0000","last_became_peered":"2024-09-14T06:17:02.193313+0000","last_unstale":"2024-09-14T06:18:34.740380+0000","last_undegraded":"2024-09-14T06:18:34.740380+0000","last_fullsized":"2024-09-14T06:18:34.740380+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:43:51.113692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002155396,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4617,"num_read_kb":53605,"num_write":714,"num_write_kb":19355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"122'407","reported_seq":4554,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739967+0000","last_change":"2024-09-14T06:18:17.224253+0000","last_active":"2024-09-14T06:18:34.739967+0000","last_peered":"2024-09-14T06:18:34.739967+0000","last_clean":"2024-09-14T06:18:34.739967+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:18:34.739967+0000","last_undegraded":"2024-09-14T06:18:34.739967+0000","last_fullsized":"2024-09-14T06:18:34.739967+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:46:26.694964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019300890000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5022,"num_read_kb":43064,"num_write":350,"num_write_kb":15485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"123'1132","reported_seq":2556,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522291+0000","last_change":"2024-09-14T06:18:17.337001+0000","last_active":"2024-09-14T06:18:34.522291+0000","last_peered":"2024-09-14T06:18:34.522291+0000","last_clean":"2024-09-14T06:18:34.522291+0000","last_became_active":"2024-09-14T06:17:38.226838+0000","last_became_peered":"2024-09-14T06:17:38.226838+0000","last_unstale":"2024-09-14T06:18:34.522291+0000","last_undegraded":"2024-09-14T06:18:34.522291+0000","last_fullsized":"2024-09-14T06:18:34.522291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":1132,"log_dups_size":0,"ondisk_log_size":1132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:01.176538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062318570000000004,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5125,"num_read_kb":14378,"num_write":1577,"num_write_kb":13994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10785266,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"123'934","reported_seq":3133,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521839+0000","last_change":"2024-09-14T06:18:17.228603+0000","last_active":"2024-09-14T06:18:34.521839+0000","last_peered":"2024-09-14T06:18:34.521839+0000","last_clean":"2024-09-14T06:18:34.521839+0000","last_became_active":"2024-09-14T06:17:48.737020+0000","last_became_peered":"2024-09-14T06:17:48.737020+0000","last_unstale":"2024-09-14T06:18:34.521839+0000","last_undegraded":"2024-09-14T06:18:34.521839+0000","last_fullsized":"2024-09-14T06:18:34.521839+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:04:33.297278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024240709999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":6011,"num_read_kb":10015,"num_write":1345,"num_write_kb":6524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":3869232,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"123'349","reported_seq":3834,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521767+0000","last_change":"2024-09-14T06:18:17.228692+0000","last_active":"2024-09-14T06:18:34.521767+0000","last_peered":"2024-09-14T06:18:34.521767+0000","last_clean":"2024-09-14T06:18:34.521767+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:18:34.521767+0000","last_undegraded":"2024-09-14T06:18:34.521767+0000","last_fullsized":"2024-09-14T06:18:34.521767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:40:01.686387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024977549999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3342,"num_read_kb":39056,"num_write":366,"num_write_kb":25723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"122'539","reported_seq":3387,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738796+0000","last_change":"2024-09-14T06:18:17.221967+0000","last_active":"2024-09-14T06:18:34.738796+0000","last_peered":"2024-09-14T06:18:34.738796+0000","last_clean":"2024-09-14T06:18:34.738796+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:18:34.738796+0000","last_undegraded":"2024-09-14T06:18:34.738796+0000","last_fullsized":"2024-09-14T06:18:34.738796+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:05:11.528071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087057599999999997,"stat_sum":{"num_bytes":19,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":4088,"num_read_kb":13128,"num_write":658,"num_write_kb":7511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"122'532","reported_seq":2187,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521641+0000","last_change":"2024-09-14T06:18:17.227798+0000","last_active":"2024-09-14T06:18:34.521641+0000","last_peered":"2024-09-14T06:18:34.521641+0000","last_clean":"2024-09-14T06:18:34.521641+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:18:34.521641+0000","last_undegraded":"2024-09-14T06:18:34.521641+0000","last_fullsized":"2024-09-14T06:18:34.521641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:09:43.212330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001716539,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2883,"num_read_kb":19234,"num_write":704,"num_write_kb":12873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"123'400","reported_seq":3103,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740079+0000","last_change":"2024-09-14T06:18:17.225653+0000","last_active":"2024-09-14T06:18:34.740079+0000","last_peered":"2024-09-14T06:18:34.740079+0000","last_clean":"2024-09-14T06:18:34.740079+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:18:34.740079+0000","last_undegraded":"2024-09-14T06:18:34.740079+0000","last_fullsized":"2024-09-14T06:18:34.740079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:38:39.952754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002524938,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":3795,"num_read_kb":37772,"num_write":360,"num_write_kb":32941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"119'395","reported_seq":1793,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740431+0000","last_change":"2024-09-14T06:18:17.224547+0000","last_active":"2024-09-14T06:18:34.740431+0000","last_peered":"2024-09-14T06:18:34.740431+0000","last_clean":"2024-09-14T06:18:34.740431+0000","last_became_active":"2024-09-14T06:17:37.233410+0000","last_became_peered":"2024-09-14T06:17:37.233410+0000","last_unstale":"2024-09-14T06:18:34.740431+0000","last_undegraded":"2024-09-14T06:18:34.740431+0000","last_fullsized":"2024-09-14T06:18:34.740431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:33.122557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019943040000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1416,"num_read_kb":17477,"num_write":336,"num_write_kb":17193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":23467377,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"122'436","reported_seq":2420,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521785+0000","last_change":"2024-09-14T06:18:25.223056+0000","last_active":"2024-09-14T06:18:34.521785+0000","last_peered":"2024-09-14T06:18:34.521785+0000","last_clean":"2024-09-14T06:18:34.521785+0000","last_became_active":"2024-09-14T06:18:25.219200+0000","last_became_peered":"2024-09-14T06:18:25.219200+0000","last_unstale":"2024-09-14T06:18:34.521785+0000","last_undegraded":"2024-09-14T06:18:34.521785+0000","last_fullsized":"2024-09-14T06:18:34.521785+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:26:30.150015+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.047023011000000003,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2483,"num_read_kb":16488,"num_write":510,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":6785399,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"122'432","reported_seq":3823,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739366+0000","last_change":"2024-09-14T06:18:17.226299+0000","last_active":"2024-09-14T06:18:34.739366+0000","last_peered":"2024-09-14T06:18:34.739366+0000","last_clean":"2024-09-14T06:18:34.739366+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:18:34.739366+0000","last_undegraded":"2024-09-14T06:18:34.739366+0000","last_fullsized":"2024-09-14T06:18:34.739366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:42:58.536316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094053699999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4134,"num_read_kb":64629,"num_write":506,"num_write_kb":22219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"123'308","reported_seq":2112,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522107+0000","last_change":"2024-09-14T06:18:17.333655+0000","last_active":"2024-09-14T06:18:34.522107+0000","last_peered":"2024-09-14T06:18:34.522107+0000","last_clean":"2024-09-14T06:18:34.522107+0000","last_became_active":"2024-09-14T06:17:37.231244+0000","last_became_peered":"2024-09-14T06:17:37.231244+0000","last_unstale":"2024-09-14T06:18:34.522107+0000","last_undegraded":"2024-09-14T06:18:34.522107+0000","last_fullsized":"2024-09-14T06:18:34.522107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:10.356294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044577879999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2916,"num_read_kb":27195,"num_write":273,"num_write_kb":15734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":32506492,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":123,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522222+0000","last_change":"2024-09-14T06:17:34.200709+0000","last_active":"2024-09-14T06:18:34.522222+0000","last_peered":"2024-09-14T06:18:34.522222+0000","last_clean":"2024-09-14T06:18:34.522222+0000","last_became_active":"2024-09-14T06:17:34.200238+0000","last_became_peered":"2024-09-14T06:17:34.200238+0000","last_unstale":"2024-09-14T06:18:34.522222+0000","last_undegraded":"2024-09-14T06:18:34.522222+0000","last_fullsized":"2024-09-14T06:18:34.522222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:34.864695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"123'487","reported_seq":1380,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521933+0000","last_change":"2024-09-14T06:18:17.229782+0000","last_active":"2024-09-14T06:18:34.521933+0000","last_peered":"2024-09-14T06:18:34.521933+0000","last_clean":"2024-09-14T06:18:34.521933+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:18:34.521933+0000","last_undegraded":"2024-09-14T06:18:34.521933+0000","last_fullsized":"2024-09-14T06:18:34.521933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:58:18.826180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030371690000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1562,"num_read_kb":9726,"num_write":495,"num_write_kb":9169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":253,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521995+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:18:34.521995+0000","last_peered":"2024-09-14T06:18:34.521995+0000","last_clean":"2024-09-14T06:18:34.521995+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:18:34.521995+0000","last_undegraded":"2024-09-14T06:18:34.521995+0000","last_fullsized":"2024-09-14T06:18:34.521995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"123'382","reported_seq":3317,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522136+0000","last_change":"2024-09-14T06:18:17.229810+0000","last_active":"2024-09-14T06:18:34.522136+0000","last_peered":"2024-09-14T06:18:34.522136+0000","last_clean":"2024-09-14T06:18:34.522136+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:18:34.522136+0000","last_undegraded":"2024-09-14T06:18:34.522136+0000","last_fullsized":"2024-09-14T06:18:34.522136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:45:12.946377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030864040000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3293,"num_read_kb":28583,"num_write":338,"num_write_kb":15237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":123,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522548+0000","last_change":"2024-09-14T06:17:34.196668+0000","last_active":"2024-09-14T06:18:34.522548+0000","last_peered":"2024-09-14T06:18:34.522548+0000","last_clean":"2024-09-14T06:18:34.522548+0000","last_became_active":"2024-09-14T06:17:34.196072+0000","last_became_peered":"2024-09-14T06:17:34.196072+0000","last_unstale":"2024-09-14T06:18:34.522548+0000","last_undegraded":"2024-09-14T06:18:34.522548+0000","last_fullsized":"2024-09-14T06:18:34.522548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:18:45.616834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"123'369","reported_seq":2035,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739803+0000","last_change":"2024-09-14T06:18:17.300971+0000","last_active":"2024-09-14T06:18:34.739803+0000","last_peered":"2024-09-14T06:18:34.739803+0000","last_clean":"2024-09-14T06:18:34.739803+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:18:34.739803+0000","last_undegraded":"2024-09-14T06:18:34.739803+0000","last_fullsized":"2024-09-14T06:18:34.739803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:05.035879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018818730000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1780,"num_read_kb":12934,"num_write":307,"num_write_kb":11582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":255,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740225+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:18:34.740225+0000","last_peered":"2024-09-14T06:18:34.740225+0000","last_clean":"2024-09-14T06:18:34.740225+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:18:34.740225+0000","last_undegraded":"2024-09-14T06:18:34.740225+0000","last_fullsized":"2024-09-14T06:18:34.740225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"123'272","reported_seq":1766,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521495+0000","last_change":"2024-09-14T06:18:25.223649+0000","last_active":"2024-09-14T06:18:34.521495+0000","last_peered":"2024-09-14T06:18:34.521495+0000","last_clean":"2024-09-14T06:18:34.521495+0000","last_became_active":"2024-09-14T06:18:25.221823+0000","last_became_peered":"2024-09-14T06:18:25.221823+0000","last_unstale":"2024-09-14T06:18:34.521495+0000","last_undegraded":"2024-09-14T06:18:34.521495+0000","last_fullsized":"2024-09-14T06:18:34.521495+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:35:13.386184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019482589999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1362,"num_read_kb":21261,"num_write":264,"num_write_kb":9954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"123'394","reported_seq":1248,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739512+0000","last_change":"2024-09-14T06:18:17.226968+0000","last_active":"2024-09-14T06:18:34.739512+0000","last_peered":"2024-09-14T06:18:34.739512+0000","last_clean":"2024-09-14T06:18:34.739512+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:18:34.739512+0000","last_undegraded":"2024-09-14T06:18:34.739512+0000","last_fullsized":"2024-09-14T06:18:34.739512+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:39:35.192039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001556469,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":547,"num_read_kb":4255,"num_write":315,"num_write_kb":7133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":127,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520864+0000","last_change":"2024-09-14T06:17:34.203977+0000","last_active":"2024-09-14T06:18:34.520864+0000","last_peered":"2024-09-14T06:18:34.520864+0000","last_clean":"2024-09-14T06:18:34.520864+0000","last_became_active":"2024-09-14T06:17:34.199244+0000","last_became_peered":"2024-09-14T06:17:34.199244+0000","last_unstale":"2024-09-14T06:18:34.520864+0000","last_undegraded":"2024-09-14T06:18:34.520864+0000","last_fullsized":"2024-09-14T06:18:34.520864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:21.934094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"123'407","reported_seq":2036,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521548+0000","last_change":"2024-09-14T06:18:17.238754+0000","last_active":"2024-09-14T06:18:34.521548+0000","last_peered":"2024-09-14T06:18:34.521548+0000","last_clean":"2024-09-14T06:18:34.521548+0000","last_became_active":"2024-09-14T06:18:17.230545+0000","last_became_peered":"2024-09-14T06:18:17.230545+0000","last_unstale":"2024-09-14T06:18:34.521548+0000","last_undegraded":"2024-09-14T06:18:34.521548+0000","last_fullsized":"2024-09-14T06:18:34.521548+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:17:11.336110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048632429999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2097,"num_read_kb":18969,"num_write":521,"num_write_kb":11051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":36437,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":254,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739790+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:18:34.739790+0000","last_peered":"2024-09-14T06:18:34.739790+0000","last_clean":"2024-09-14T06:18:34.739790+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:18:34.739790+0000","last_undegraded":"2024-09-14T06:18:34.739790+0000","last_fullsized":"2024-09-14T06:18:34.739790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"117'540","reported_seq":2241,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739370+0000","last_change":"2024-09-14T06:18:17.300145+0000","last_active":"2024-09-14T06:18:34.739370+0000","last_peered":"2024-09-14T06:18:34.739370+0000","last_clean":"2024-09-14T06:18:34.739370+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:18:34.739370+0000","last_undegraded":"2024-09-14T06:18:34.739370+0000","last_fullsized":"2024-09-14T06:18:34.739370+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:25.627665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001640601,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2885,"num_read_kb":8812,"num_write":721,"num_write_kb":8784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":253,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520702+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:18:34.520702+0000","last_peered":"2024-09-14T06:18:34.520702+0000","last_clean":"2024-09-14T06:18:34.520702+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:18:34.520702+0000","last_undegraded":"2024-09-14T06:18:34.520702+0000","last_fullsized":"2024-09-14T06:18:34.520702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"122'471","reported_seq":2078,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521629+0000","last_change":"2024-09-14T06:18:17.227668+0000","last_active":"2024-09-14T06:18:34.521629+0000","last_peered":"2024-09-14T06:18:34.521629+0000","last_clean":"2024-09-14T06:18:34.521629+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:18:34.521629+0000","last_undegraded":"2024-09-14T06:18:34.521629+0000","last_fullsized":"2024-09-14T06:18:34.521629+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:19:40.041408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001487162,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2220,"num_read_kb":23013,"num_write":375,"num_write_kb":14523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":274,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522087+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:18:34.522087+0000","last_peered":"2024-09-14T06:18:34.522087+0000","last_clean":"2024-09-14T06:18:34.522087+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:18:34.522087+0000","last_undegraded":"2024-09-14T06:18:34.522087+0000","last_fullsized":"2024-09-14T06:18:34.522087+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":345,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738574+0000","last_change":"2024-09-14T06:16:04.635772+0000","last_active":"2024-09-14T06:18:34.738574+0000","last_peered":"2024-09-14T06:18:34.738574+0000","last_clean":"2024-09-14T06:18:34.738574+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:18:34.738574+0000","last_undegraded":"2024-09-14T06:18:34.738574+0000","last_fullsized":"2024-09-14T06:18:34.738574+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:03.019247+0000","last_clean_scrub_stamp":"2024-09-14T06:16:03.019247+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:01.627344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"122'442","reported_seq":1709,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521572+0000","last_change":"2024-09-14T06:18:17.230742+0000","last_active":"2024-09-14T06:18:34.521572+0000","last_peered":"2024-09-14T06:18:34.521572+0000","last_clean":"2024-09-14T06:18:34.521572+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:18:34.521572+0000","last_undegraded":"2024-09-14T06:18:34.521572+0000","last_fullsized":"2024-09-14T06:18:34.521572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:58:35.960390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00348241,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2133,"num_read_kb":6715,"num_write":589,"num_write_kb":6849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"122'398","reported_seq":1904,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520702+0000","last_change":"2024-09-14T06:18:17.336435+0000","last_active":"2024-09-14T06:18:34.520702+0000","last_peered":"2024-09-14T06:18:34.520702+0000","last_clean":"2024-09-14T06:18:34.520702+0000","last_became_active":"2024-09-14T06:17:38.224238+0000","last_became_peered":"2024-09-14T06:17:38.224238+0000","last_unstale":"2024-09-14T06:18:34.520702+0000","last_undegraded":"2024-09-14T06:18:34.520702+0000","last_fullsized":"2024-09-14T06:18:34.520702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:37.628302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064732729999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2223,"num_read_kb":13651,"num_write":549,"num_write_kb":14353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8477378,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"123'384","reported_seq":2444,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521434+0000","last_change":"2024-09-14T06:18:17.336082+0000","last_active":"2024-09-14T06:18:34.521434+0000","last_peered":"2024-09-14T06:18:34.521434+0000","last_clean":"2024-09-14T06:18:34.521434+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:18:34.521434+0000","last_undegraded":"2024-09-14T06:18:34.521434+0000","last_fullsized":"2024-09-14T06:18:34.521434+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:58.737457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005800637,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":2425,"num_read_kb":16285,"num_write":322,"num_write_kb":13807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"123'318","reported_seq":2937,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739738+0000","last_change":"2024-09-14T06:18:17.236905+0000","last_active":"2024-09-14T06:18:34.739738+0000","last_peered":"2024-09-14T06:18:34.739738+0000","last_clean":"2024-09-14T06:18:34.739738+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:18:34.739738+0000","last_undegraded":"2024-09-14T06:18:34.739738+0000","last_fullsized":"2024-09-14T06:18:34.739738+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:57:26.637373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013408000000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2870,"num_read_kb":16793,"num_write":272,"num_write_kb":9901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"123'189","reported_seq":1698,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739303+0000","last_change":"2024-09-14T06:18:17.226333+0000","last_active":"2024-09-14T06:18:34.739303+0000","last_peered":"2024-09-14T06:18:34.739303+0000","last_clean":"2024-09-14T06:18:34.739303+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:18:34.739303+0000","last_undegraded":"2024-09-14T06:18:34.739303+0000","last_fullsized":"2024-09-14T06:18:34.739303+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:57:04.903011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010333339999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1287,"num_read_kb":29081,"num_write":248,"num_write_kb":11577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"122'687","reported_seq":2640,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522423+0000","last_change":"2024-09-14T06:18:17.337571+0000","last_active":"2024-09-14T06:18:34.522423+0000","last_peered":"2024-09-14T06:18:34.522423+0000","last_clean":"2024-09-14T06:18:34.522423+0000","last_became_active":"2024-09-14T06:17:38.227106+0000","last_became_peered":"2024-09-14T06:17:38.227106+0000","last_unstale":"2024-09-14T06:18:34.522423+0000","last_undegraded":"2024-09-14T06:18:34.522423+0000","last_fullsized":"2024-09-14T06:18:34.522423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:27:58.009450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006423425,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3768,"num_read_kb":26949,"num_write":994,"num_write_kb":20276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":29390214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"123'762","reported_seq":5455,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521683+0000","last_change":"2024-09-14T06:18:25.224296+0000","last_active":"2024-09-14T06:18:34.521683+0000","last_peered":"2024-09-14T06:18:34.521683+0000","last_clean":"2024-09-14T06:18:34.521683+0000","last_became_active":"2024-09-14T06:18:25.223198+0000","last_became_peered":"2024-09-14T06:18:25.223198+0000","last_unstale":"2024-09-14T06:18:34.521683+0000","last_undegraded":"2024-09-14T06:18:34.521683+0000","last_fullsized":"2024-09-14T06:18:34.521683+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:33:35.589616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021565500000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10255,"num_read_kb":51706,"num_write":697,"num_write_kb":16134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"123'661","reported_seq":2988,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739487+0000","last_change":"2024-09-14T06:18:17.226919+0000","last_active":"2024-09-14T06:18:34.739487+0000","last_peered":"2024-09-14T06:18:34.739487+0000","last_clean":"2024-09-14T06:18:34.739487+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:18:34.739487+0000","last_undegraded":"2024-09-14T06:18:34.739487+0000","last_fullsized":"2024-09-14T06:18:34.739487+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:55:17.915614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001432902,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4425,"num_read_kb":16166,"num_write":768,"num_write_kb":12749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"122'360","reported_seq":2605,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738517+0000","last_change":"2024-09-14T06:18:17.221410+0000","last_active":"2024-09-14T06:18:34.738517+0000","last_peered":"2024-09-14T06:18:34.738517+0000","last_clean":"2024-09-14T06:18:34.738517+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:18:34.738517+0000","last_undegraded":"2024-09-14T06:18:34.738517+0000","last_fullsized":"2024-09-14T06:18:34.738517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:24:43.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068876799999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2476,"num_read_kb":47684,"num_write":341,"num_write_kb":14643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"123'556","reported_seq":3001,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521007+0000","last_change":"2024-09-14T06:18:17.332968+0000","last_active":"2024-09-14T06:18:34.521007+0000","last_peered":"2024-09-14T06:18:34.521007+0000","last_clean":"2024-09-14T06:18:34.521007+0000","last_became_active":"2024-09-14T06:17:38.224251+0000","last_became_peered":"2024-09-14T06:17:38.224251+0000","last_unstale":"2024-09-14T06:18:34.521007+0000","last_undegraded":"2024-09-14T06:18:34.521007+0000","last_fullsized":"2024-09-14T06:18:34.521007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:32:17.880108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033873589999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4434,"num_read_kb":14632,"num_write":613,"num_write_kb":7670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":12239008,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"122'518","reported_seq":3082,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739545+0000","last_change":"2024-09-14T06:18:17.300159+0000","last_active":"2024-09-14T06:18:34.739545+0000","last_peered":"2024-09-14T06:18:34.739545+0000","last_clean":"2024-09-14T06:18:34.739545+0000","last_became_active":"2024-09-14T06:17:37.232145+0000","last_became_peered":"2024-09-14T06:17:37.232145+0000","last_unstale":"2024-09-14T06:18:34.739545+0000","last_undegraded":"2024-09-14T06:18:34.739545+0000","last_fullsized":"2024-09-14T06:18:34.739545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:12:47.470446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016513039999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3345,"num_read_kb":11048,"num_write":606,"num_write_kb":8456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":10176554,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"122'505","reported_seq":4220,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521410+0000","last_change":"2024-09-14T06:18:17.333610+0000","last_active":"2024-09-14T06:18:34.521410+0000","last_peered":"2024-09-14T06:18:34.521410+0000","last_clean":"2024-09-14T06:18:34.521410+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:18:34.521410+0000","last_undegraded":"2024-09-14T06:18:34.521410+0000","last_fullsized":"2024-09-14T06:18:34.521410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:51:31.054844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039616110000000003,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6037,"num_read_kb":16850,"num_write":431,"num_write_kb":5588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":210,"num_object_clones":0,"num_object_copies":420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210,"num_whiteouts":206,"num_read":107256,"num_read_kb":751154,"num_write":17465,"num_write_kb":429271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1267,"num_bytes_recovered":144403203,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15570,"ondisk_log_size":15570,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509475,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55028,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316812,"statfs":{"total":96636764160,"available":96580415488,"internally_reserved":0,"allocated":1662976,"data_stored":789529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36972,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334868,"statfs":{"total":96636764160,"available":96598904832,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509477,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73132,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298708,"statfs":{"total":96636764160,"available":96561876992,"internally_reserved":0,"allocated":1724416,"data_stored":1379897,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509476,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59184,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312656,"statfs":{"total":96636764160,"available":96576159744,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542179,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36972,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334868,"statfs":{"total":96636764160,"available":96598904832,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28652,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343188,"statfs":{"total":96636764160,"available":96607424512,"internally_reserved":0,"allocated":1662976,"data_stored":789529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542178,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28716,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343124,"statfs":{"total":96636764160,"available":96607358976,"internally_reserved":0,"allocated":1724416,"data_stored":1379897,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542180,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28652,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343188,"statfs":{"total":96636764160,"available":96607424512,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:18:39.676 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T06:18:39.969 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:18:39.969 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":139,"fsid":"038563b1-268f-461f-b6a6-4b4b41ad0236","created":"2024-09-14T06:15:49.493035+0000","modified":"2024-09-14T06:18:34.496136+0000","last_up_change":"2024-09-14T06:16:02.014989+0000","last_in_change":"2024-09-14T06:16:26.221200+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:16:02.429225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T06:16:08.129454+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T06:16:18.928556+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":123,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c669ea76-ed19-4c12-9e56-d79d62846f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6809","nonce":1157559201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6811","nonce":1157559201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6815","nonce":1157559201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":1157559201},{"type":"v1","addr":"172.21.15.39:6813","nonce":1157559201}]},"public_addr":"172.21.15.39:6809/1157559201","cluster_addr":"172.21.15.39:6811/1157559201","heartbeat_back_addr":"172.21.15.39:6815/1157559201","heartbeat_front_addr":"172.21.15.39:6813/1157559201","state":["exists","up"]},{"osd":1,"uuid":"7d7b73c4-cb71-4f13-b13d-fcfb91bcce0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6817","nonce":4062089212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6819","nonce":4062089212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6823","nonce":4062089212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":4062089212},{"type":"v1","addr":"172.21.15.39:6821","nonce":4062089212}]},"public_addr":"172.21.15.39:6817/4062089212","cluster_addr":"172.21.15.39:6819/4062089212","heartbeat_back_addr":"172.21.15.39:6823/4062089212","heartbeat_front_addr":"172.21.15.39:6821/4062089212","state":["exists","up"]},{"osd":2,"uuid":"9bda988f-e615-4545-849b-318c408c4c42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6825","nonce":1724773882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6827","nonce":1724773882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6831","nonce":1724773882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1724773882},{"type":"v1","addr":"172.21.15.39:6829","nonce":1724773882}]},"public_addr":"172.21.15.39:6825/1724773882","cluster_addr":"172.21.15.39:6827/1724773882","heartbeat_back_addr":"172.21.15.39:6831/1724773882","heartbeat_front_addr":"172.21.15.39:6829/1724773882","state":["exists","up"]},{"osd":3,"uuid":"8ea6cd69-2ef1-42ea-b339-7394bf8f4a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6801","nonce":3775031293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6803","nonce":3775031293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6807","nonce":3775031293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3775031293},{"type":"v1","addr":"172.21.15.39:6805","nonce":3775031293}]},"public_addr":"172.21.15.39:6801/3775031293","cluster_addr":"172.21.15.39:6803/3775031293","heartbeat_back_addr":"172.21.15.39:6807/3775031293","heartbeat_front_addr":"172.21.15.39:6805/3775031293","state":["exists","up"]},{"osd":4,"uuid":"fea45e8a-e8ea-4963-b93f-0651b84c9e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6825","nonce":2632317380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6827","nonce":2632317380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6831","nonce":2632317380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2632317380},{"type":"v1","addr":"172.21.15.87:6829","nonce":2632317380}]},"public_addr":"172.21.15.87:6825/2632317380","cluster_addr":"172.21.15.87:6827/2632317380","heartbeat_back_addr":"172.21.15.87:6831/2632317380","heartbeat_front_addr":"172.21.15.87:6829/2632317380","state":["exists","up"]},{"osd":5,"uuid":"b601e852-a180-477f-86fc-6a213fa866f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6817","nonce":3501664149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6819","nonce":3501664149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6823","nonce":3501664149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":3501664149},{"type":"v1","addr":"172.21.15.87:6821","nonce":3501664149}]},"public_addr":"172.21.15.87:6817/3501664149","cluster_addr":"172.21.15.87:6819/3501664149","heartbeat_back_addr":"172.21.15.87:6823/3501664149","heartbeat_front_addr":"172.21.15.87:6821/3501664149","state":["exists","up"]},{"osd":6,"uuid":"1113afe0-f524-41b1-8e51-eb5164602d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6801","nonce":1395936437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6803","nonce":1395936437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6807","nonce":1395936437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1395936437},{"type":"v1","addr":"172.21.15.87:6805","nonce":1395936437}]},"public_addr":"172.21.15.87:6801/1395936437","cluster_addr":"172.21.15.87:6803/1395936437","heartbeat_back_addr":"172.21.15.87:6807/1395936437","heartbeat_front_addr":"172.21.15.87:6805/1395936437","state":["exists","up"]},{"osd":7,"uuid":"c9162545-318b-4591-be64-cc8ace9097fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6809","nonce":711112216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6811","nonce":711112216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6815","nonce":711112216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":711112216},{"type":"v1","addr":"172.21.15.87:6813","nonce":711112216}]},"public_addr":"172.21.15.87:6809/711112216","cluster_addr":"172.21.15.87:6811/711112216","heartbeat_back_addr":"172.21.15.87:6815/711112216","heartbeat_front_addr":"172.21.15.87:6813/711112216","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.705575+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.806946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.853831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.690315+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.494526+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.546122+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.173691+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:15:58.148509+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.0","osds":[4,5]},{"pgid":"3.1c","osds":[4,6]}],"pg_upmap_items":[{"pgid":"3.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:18:40.987 INFO:tasks.ceph:Scrubbing osd.0 2024-09-14T06:18:40.988 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-14T06:18:41.151 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:41.152 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T06:18:41.152 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:41.170 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-14T06:18:41.466 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 0 to deep-scrub 2024-09-14T06:18:41.485 INFO:tasks.ceph:Scrubbing osd.1 2024-09-14T06:18:41.485 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-14T06:18:41.638 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T06:18:41.638 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:41.655 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-14T06:18:41.949 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 1 to deep-scrub 2024-09-14T06:18:41.967 INFO:tasks.ceph:Scrubbing osd.2 2024-09-14T06:18:41.968 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-14T06:18:42.113 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:42.113 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T06:18:42.113 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:42.129 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-14T06:18:42.432 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 2 to deep-scrub 2024-09-14T06:18:42.450 INFO:tasks.ceph:Scrubbing osd.3 2024-09-14T06:18:42.451 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-14T06:18:42.597 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:42.597 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T06:18:42.597 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:42.612 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-14T06:18:42.905 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 3 to deep-scrub 2024-09-14T06:18:42.926 INFO:tasks.ceph:Scrubbing osd.4 2024-09-14T06:18:42.926 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-14T06:18:43.074 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:43.074 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T06:18:43.074 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:43.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-14T06:18:43.393 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 4 to deep-scrub 2024-09-14T06:18:43.413 INFO:tasks.ceph:Scrubbing osd.5 2024-09-14T06:18:43.413 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-14T06:18:43.571 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:43.572 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T06:18:43.572 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:43.590 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-14T06:18:43.881 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 5 to deep-scrub 2024-09-14T06:18:43.899 INFO:tasks.ceph:Scrubbing osd.6 2024-09-14T06:18:43.899 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-14T06:18:44.056 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:44.056 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T06:18:44.056 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:44.073 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-14T06:18:44.379 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 6 to deep-scrub 2024-09-14T06:18:44.399 INFO:tasks.ceph:Scrubbing osd.7 2024-09-14T06:18:44.399 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-14T06:18:44.557 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-14T06:18:44.557 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T06:18:44.557 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-14T06:18:44.573 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-14T06:18:44.878 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 7 to deep-scrub 2024-09-14T06:18:44.899 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:18:45.212 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:18:45.213 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:18:45.229 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-09-14T06:18:44.421798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":214,"num_object_clones":0,"num_object_copies":428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":214,"num_whiteouts":206,"num_read":107322,"num_read_kb":751209,"num_write":17568,"num_write_kb":430901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1270,"num_bytes_recovered":144403222,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15653,"ondisk_log_size":15653,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":347296,"kb_used_data":13096,"kb_used_omap":12,"kb_used_meta":334003,"kb_avail":754627424,"statfs":{"total":773094113280,"available":772738482176,"internally_reserved":0,"allocated":13410304,"data_stored":7496892,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":342019632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002842"},"pg_stats":[{"pgid":"3.1f","version":"123'604","reported_seq":3721,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740380+0000","last_change":"2024-09-14T06:18:17.225123+0000","last_active":"2024-09-14T06:18:34.740380+0000","last_peered":"2024-09-14T06:18:34.740380+0000","last_clean":"2024-09-14T06:18:34.740380+0000","last_became_active":"2024-09-14T06:17:02.193313+0000","last_became_peered":"2024-09-14T06:17:02.193313+0000","last_unstale":"2024-09-14T06:18:34.740380+0000","last_undegraded":"2024-09-14T06:18:34.740380+0000","last_fullsized":"2024-09-14T06:18:34.740380+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:43:51.113692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002155396,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4617,"num_read_kb":53605,"num_write":714,"num_write_kb":19355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"122'407","reported_seq":4554,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739967+0000","last_change":"2024-09-14T06:18:17.224253+0000","last_active":"2024-09-14T06:18:34.739967+0000","last_peered":"2024-09-14T06:18:34.739967+0000","last_clean":"2024-09-14T06:18:34.739967+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:18:34.739967+0000","last_undegraded":"2024-09-14T06:18:34.739967+0000","last_fullsized":"2024-09-14T06:18:34.739967+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:46:26.694964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019300890000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5022,"num_read_kb":43064,"num_write":350,"num_write_kb":15485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"123'1132","reported_seq":2556,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522291+0000","last_change":"2024-09-14T06:18:17.337001+0000","last_active":"2024-09-14T06:18:34.522291+0000","last_peered":"2024-09-14T06:18:34.522291+0000","last_clean":"2024-09-14T06:18:34.522291+0000","last_became_active":"2024-09-14T06:17:38.226838+0000","last_became_peered":"2024-09-14T06:17:38.226838+0000","last_unstale":"2024-09-14T06:18:34.522291+0000","last_undegraded":"2024-09-14T06:18:34.522291+0000","last_fullsized":"2024-09-14T06:18:34.522291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":1132,"log_dups_size":0,"ondisk_log_size":1132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:01.176538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062318570000000004,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5125,"num_read_kb":14378,"num_write":1577,"num_write_kb":13994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10785266,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"123'934","reported_seq":3133,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521839+0000","last_change":"2024-09-14T06:18:17.228603+0000","last_active":"2024-09-14T06:18:34.521839+0000","last_peered":"2024-09-14T06:18:34.521839+0000","last_clean":"2024-09-14T06:18:34.521839+0000","last_became_active":"2024-09-14T06:17:48.737020+0000","last_became_peered":"2024-09-14T06:17:48.737020+0000","last_unstale":"2024-09-14T06:18:34.521839+0000","last_undegraded":"2024-09-14T06:18:34.521839+0000","last_fullsized":"2024-09-14T06:18:34.521839+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:04:33.297278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024240709999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":6011,"num_read_kb":10015,"num_write":1345,"num_write_kb":6524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":3869232,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"123'349","reported_seq":3834,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521767+0000","last_change":"2024-09-14T06:18:17.228692+0000","last_active":"2024-09-14T06:18:34.521767+0000","last_peered":"2024-09-14T06:18:34.521767+0000","last_clean":"2024-09-14T06:18:34.521767+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:18:34.521767+0000","last_undegraded":"2024-09-14T06:18:34.521767+0000","last_fullsized":"2024-09-14T06:18:34.521767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:40:01.686387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024977549999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3342,"num_read_kb":39056,"num_write":366,"num_write_kb":25723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"122'539","reported_seq":3387,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738796+0000","last_change":"2024-09-14T06:18:17.221967+0000","last_active":"2024-09-14T06:18:34.738796+0000","last_peered":"2024-09-14T06:18:34.738796+0000","last_clean":"2024-09-14T06:18:34.738796+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:18:34.738796+0000","last_undegraded":"2024-09-14T06:18:34.738796+0000","last_fullsized":"2024-09-14T06:18:34.738796+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:05:11.528071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087057599999999997,"stat_sum":{"num_bytes":19,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":4088,"num_read_kb":13128,"num_write":658,"num_write_kb":7511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"122'532","reported_seq":2187,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521641+0000","last_change":"2024-09-14T06:18:17.227798+0000","last_active":"2024-09-14T06:18:34.521641+0000","last_peered":"2024-09-14T06:18:34.521641+0000","last_clean":"2024-09-14T06:18:34.521641+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:18:34.521641+0000","last_undegraded":"2024-09-14T06:18:34.521641+0000","last_fullsized":"2024-09-14T06:18:34.521641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:09:43.212330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001716539,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2883,"num_read_kb":19234,"num_write":704,"num_write_kb":12873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"123'400","reported_seq":3103,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740079+0000","last_change":"2024-09-14T06:18:17.225653+0000","last_active":"2024-09-14T06:18:34.740079+0000","last_peered":"2024-09-14T06:18:34.740079+0000","last_clean":"2024-09-14T06:18:34.740079+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:18:34.740079+0000","last_undegraded":"2024-09-14T06:18:34.740079+0000","last_fullsized":"2024-09-14T06:18:34.740079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:38:39.952754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002524938,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":3795,"num_read_kb":37772,"num_write":360,"num_write_kb":32941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"119'395","reported_seq":1793,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740431+0000","last_change":"2024-09-14T06:18:17.224547+0000","last_active":"2024-09-14T06:18:34.740431+0000","last_peered":"2024-09-14T06:18:34.740431+0000","last_clean":"2024-09-14T06:18:34.740431+0000","last_became_active":"2024-09-14T06:17:37.233410+0000","last_became_peered":"2024-09-14T06:17:37.233410+0000","last_unstale":"2024-09-14T06:18:34.740431+0000","last_undegraded":"2024-09-14T06:18:34.740431+0000","last_fullsized":"2024-09-14T06:18:34.740431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:33.122557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019943040000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1416,"num_read_kb":17477,"num_write":336,"num_write_kb":17193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":23467377,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"122'436","reported_seq":2420,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521785+0000","last_change":"2024-09-14T06:18:25.223056+0000","last_active":"2024-09-14T06:18:34.521785+0000","last_peered":"2024-09-14T06:18:34.521785+0000","last_clean":"2024-09-14T06:18:34.521785+0000","last_became_active":"2024-09-14T06:18:25.219200+0000","last_became_peered":"2024-09-14T06:18:25.219200+0000","last_unstale":"2024-09-14T06:18:34.521785+0000","last_undegraded":"2024-09-14T06:18:34.521785+0000","last_fullsized":"2024-09-14T06:18:34.521785+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:26:30.150015+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.047023011000000003,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2483,"num_read_kb":16488,"num_write":510,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":6785399,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"139'435","reported_seq":3834,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:42.620098+0000","last_change":"2024-09-14T06:18:42.620098+0000","last_active":"2024-09-14T06:18:42.620098+0000","last_peered":"2024-09-14T06:18:42.620098+0000","last_clean":"2024-09-14T06:18:42.620098+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:18:42.620098+0000","last_undegraded":"2024-09-14T06:18:42.620098+0000","last_fullsized":"2024-09-14T06:18:42.620098+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'435","last_scrub_stamp":"2024-09-14T06:18:42.620035+0000","last_deep_scrub":"139'435","last_deep_scrub_stamp":"2024-09-14T06:18:42.620035+0000","last_clean_scrub_stamp":"2024-09-14T06:18:42.620035+0000","objects_scrubbed":3,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:42:20.494172+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00094053699999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4134,"num_read_kb":64629,"num_write":506,"num_write_kb":22219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"123'308","reported_seq":2112,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522107+0000","last_change":"2024-09-14T06:18:17.333655+0000","last_active":"2024-09-14T06:18:34.522107+0000","last_peered":"2024-09-14T06:18:34.522107+0000","last_clean":"2024-09-14T06:18:34.522107+0000","last_became_active":"2024-09-14T06:17:37.231244+0000","last_became_peered":"2024-09-14T06:17:37.231244+0000","last_unstale":"2024-09-14T06:18:34.522107+0000","last_undegraded":"2024-09-14T06:18:34.522107+0000","last_fullsized":"2024-09-14T06:18:34.522107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:10.356294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044577879999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2916,"num_read_kb":27195,"num_write":273,"num_write_kb":15734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":32506492,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":123,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522222+0000","last_change":"2024-09-14T06:17:34.200709+0000","last_active":"2024-09-14T06:18:34.522222+0000","last_peered":"2024-09-14T06:18:34.522222+0000","last_clean":"2024-09-14T06:18:34.522222+0000","last_became_active":"2024-09-14T06:17:34.200238+0000","last_became_peered":"2024-09-14T06:17:34.200238+0000","last_unstale":"2024-09-14T06:18:34.522222+0000","last_undegraded":"2024-09-14T06:18:34.522222+0000","last_fullsized":"2024-09-14T06:18:34.522222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:34.864695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"123'487","reported_seq":1380,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521933+0000","last_change":"2024-09-14T06:18:17.229782+0000","last_active":"2024-09-14T06:18:34.521933+0000","last_peered":"2024-09-14T06:18:34.521933+0000","last_clean":"2024-09-14T06:18:34.521933+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:18:34.521933+0000","last_undegraded":"2024-09-14T06:18:34.521933+0000","last_fullsized":"2024-09-14T06:18:34.521933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:58:18.826180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030371690000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1562,"num_read_kb":9726,"num_write":495,"num_write_kb":9169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":253,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521995+0000","last_change":"2024-09-14T06:16:12.383385+0000","last_active":"2024-09-14T06:18:34.521995+0000","last_peered":"2024-09-14T06:18:34.521995+0000","last_clean":"2024-09-14T06:18:34.521995+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:18:34.521995+0000","last_undegraded":"2024-09-14T06:18:34.521995+0000","last_fullsized":"2024-09-14T06:18:34.521995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:42.314158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"123'382","reported_seq":3317,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522136+0000","last_change":"2024-09-14T06:18:17.229810+0000","last_active":"2024-09-14T06:18:34.522136+0000","last_peered":"2024-09-14T06:18:34.522136+0000","last_clean":"2024-09-14T06:18:34.522136+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:18:34.522136+0000","last_undegraded":"2024-09-14T06:18:34.522136+0000","last_fullsized":"2024-09-14T06:18:34.522136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:45:12.946377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030864040000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3293,"num_read_kb":28583,"num_write":338,"num_write_kb":15237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":123,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522548+0000","last_change":"2024-09-14T06:17:34.196668+0000","last_active":"2024-09-14T06:18:34.522548+0000","last_peered":"2024-09-14T06:18:34.522548+0000","last_clean":"2024-09-14T06:18:34.522548+0000","last_became_active":"2024-09-14T06:17:34.196072+0000","last_became_peered":"2024-09-14T06:17:34.196072+0000","last_unstale":"2024-09-14T06:18:34.522548+0000","last_undegraded":"2024-09-14T06:18:34.522548+0000","last_fullsized":"2024-09-14T06:18:34.522548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:18:45.616834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"123'369","reported_seq":2035,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739803+0000","last_change":"2024-09-14T06:18:17.300971+0000","last_active":"2024-09-14T06:18:34.739803+0000","last_peered":"2024-09-14T06:18:34.739803+0000","last_clean":"2024-09-14T06:18:34.739803+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:18:34.739803+0000","last_undegraded":"2024-09-14T06:18:34.739803+0000","last_fullsized":"2024-09-14T06:18:34.739803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:05.035879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018818730000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1780,"num_read_kb":12934,"num_write":307,"num_write_kb":11582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":255,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.740225+0000","last_change":"2024-09-14T06:16:12.163023+0000","last_active":"2024-09-14T06:18:34.740225+0000","last_peered":"2024-09-14T06:18:34.740225+0000","last_clean":"2024-09-14T06:18:34.740225+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:18:34.740225+0000","last_undegraded":"2024-09-14T06:18:34.740225+0000","last_fullsized":"2024-09-14T06:18:34.740225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:16.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"123'272","reported_seq":1766,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521495+0000","last_change":"2024-09-14T06:18:25.223649+0000","last_active":"2024-09-14T06:18:34.521495+0000","last_peered":"2024-09-14T06:18:34.521495+0000","last_clean":"2024-09-14T06:18:34.521495+0000","last_became_active":"2024-09-14T06:18:25.221823+0000","last_became_peered":"2024-09-14T06:18:25.221823+0000","last_unstale":"2024-09-14T06:18:34.521495+0000","last_undegraded":"2024-09-14T06:18:34.521495+0000","last_fullsized":"2024-09-14T06:18:34.521495+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:35:13.386184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019482589999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1362,"num_read_kb":21261,"num_write":264,"num_write_kb":9954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"123'394","reported_seq":1248,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739512+0000","last_change":"2024-09-14T06:18:17.226968+0000","last_active":"2024-09-14T06:18:34.739512+0000","last_peered":"2024-09-14T06:18:34.739512+0000","last_clean":"2024-09-14T06:18:34.739512+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:18:34.739512+0000","last_undegraded":"2024-09-14T06:18:34.739512+0000","last_fullsized":"2024-09-14T06:18:34.739512+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:39:35.192039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001556469,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":547,"num_read_kb":4255,"num_write":315,"num_write_kb":7133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":127,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520864+0000","last_change":"2024-09-14T06:17:34.203977+0000","last_active":"2024-09-14T06:18:34.520864+0000","last_peered":"2024-09-14T06:18:34.520864+0000","last_clean":"2024-09-14T06:18:34.520864+0000","last_became_active":"2024-09-14T06:17:34.199244+0000","last_became_peered":"2024-09-14T06:17:34.199244+0000","last_unstale":"2024-09-14T06:18:34.520864+0000","last_undegraded":"2024-09-14T06:18:34.520864+0000","last_fullsized":"2024-09-14T06:18:34.520864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:21.934094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"123'407","reported_seq":2036,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521548+0000","last_change":"2024-09-14T06:18:17.238754+0000","last_active":"2024-09-14T06:18:34.521548+0000","last_peered":"2024-09-14T06:18:34.521548+0000","last_clean":"2024-09-14T06:18:34.521548+0000","last_became_active":"2024-09-14T06:18:17.230545+0000","last_became_peered":"2024-09-14T06:18:17.230545+0000","last_unstale":"2024-09-14T06:18:34.521548+0000","last_undegraded":"2024-09-14T06:18:34.521548+0000","last_fullsized":"2024-09-14T06:18:34.521548+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:17:11.336110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048632429999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2097,"num_read_kb":18969,"num_write":521,"num_write_kb":11051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":36437,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":254,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739790+0000","last_change":"2024-09-14T06:16:12.920120+0000","last_active":"2024-09-14T06:18:34.739790+0000","last_peered":"2024-09-14T06:18:34.739790+0000","last_clean":"2024-09-14T06:18:34.739790+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:18:34.739790+0000","last_undegraded":"2024-09-14T06:18:34.739790+0000","last_fullsized":"2024-09-14T06:18:34.739790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:05.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"117'540","reported_seq":2241,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739370+0000","last_change":"2024-09-14T06:18:17.300145+0000","last_active":"2024-09-14T06:18:34.739370+0000","last_peered":"2024-09-14T06:18:34.739370+0000","last_clean":"2024-09-14T06:18:34.739370+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:18:34.739370+0000","last_undegraded":"2024-09-14T06:18:34.739370+0000","last_fullsized":"2024-09-14T06:18:34.739370+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:25.627665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001640601,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2885,"num_read_kb":8812,"num_write":721,"num_write_kb":8784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":253,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520702+0000","last_change":"2024-09-14T06:16:12.179657+0000","last_active":"2024-09-14T06:18:34.520702+0000","last_peered":"2024-09-14T06:18:34.520702+0000","last_clean":"2024-09-14T06:18:34.520702+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:18:34.520702+0000","last_undegraded":"2024-09-14T06:18:34.520702+0000","last_fullsized":"2024-09-14T06:18:34.520702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:24.126824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"122'471","reported_seq":2078,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521629+0000","last_change":"2024-09-14T06:18:17.227668+0000","last_active":"2024-09-14T06:18:34.521629+0000","last_peered":"2024-09-14T06:18:34.521629+0000","last_clean":"2024-09-14T06:18:34.521629+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:18:34.521629+0000","last_undegraded":"2024-09-14T06:18:34.521629+0000","last_fullsized":"2024-09-14T06:18:34.521629+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:19:40.041408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001487162,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2220,"num_read_kb":23013,"num_write":375,"num_write_kb":14523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":274,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522087+0000","last_change":"2024-09-14T06:16:27.317849+0000","last_active":"2024-09-14T06:18:34.522087+0000","last_peered":"2024-09-14T06:18:34.522087+0000","last_clean":"2024-09-14T06:18:34.522087+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:18:34.522087+0000","last_undegraded":"2024-09-14T06:18:34.522087+0000","last_fullsized":"2024-09-14T06:18:34.522087+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:09.097727+0000","last_clean_scrub_stamp":"2024-09-14T06:16:09.097727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:20:57.958636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"139'74","reported_seq":355,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:42.235742+0000","last_change":"2024-09-14T06:18:42.235742+0000","last_active":"2024-09-14T06:18:42.235742+0000","last_peered":"2024-09-14T06:18:42.235742+0000","last_clean":"2024-09-14T06:18:42.235742+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:18:42.235742+0000","last_undegraded":"2024-09-14T06:18:42.235742+0000","last_fullsized":"2024-09-14T06:18:42.235742+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'74","last_scrub_stamp":"2024-09-14T06:18:42.235682+0000","last_deep_scrub":"139'74","last_deep_scrub_stamp":"2024-09-14T06:18:42.235682+0000","last_clean_scrub_stamp":"2024-09-14T06:18:42.235682+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:11:24.823765+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"122'442","reported_seq":1709,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521572+0000","last_change":"2024-09-14T06:18:17.230742+0000","last_active":"2024-09-14T06:18:34.521572+0000","last_peered":"2024-09-14T06:18:34.521572+0000","last_clean":"2024-09-14T06:18:34.521572+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:18:34.521572+0000","last_undegraded":"2024-09-14T06:18:34.521572+0000","last_fullsized":"2024-09-14T06:18:34.521572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:58:35.960390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00348241,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2133,"num_read_kb":6715,"num_write":589,"num_write_kb":6849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"122'398","reported_seq":1904,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.520702+0000","last_change":"2024-09-14T06:18:17.336435+0000","last_active":"2024-09-14T06:18:34.520702+0000","last_peered":"2024-09-14T06:18:34.520702+0000","last_clean":"2024-09-14T06:18:34.520702+0000","last_became_active":"2024-09-14T06:17:38.224238+0000","last_became_peered":"2024-09-14T06:17:38.224238+0000","last_unstale":"2024-09-14T06:18:34.520702+0000","last_undegraded":"2024-09-14T06:18:34.520702+0000","last_fullsized":"2024-09-14T06:18:34.520702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:45:37.628302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064732729999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2223,"num_read_kb":13651,"num_write":549,"num_write_kb":14353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8477378,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"123'384","reported_seq":2444,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521434+0000","last_change":"2024-09-14T06:18:17.336082+0000","last_active":"2024-09-14T06:18:34.521434+0000","last_peered":"2024-09-14T06:18:34.521434+0000","last_clean":"2024-09-14T06:18:34.521434+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:18:34.521434+0000","last_undegraded":"2024-09-14T06:18:34.521434+0000","last_fullsized":"2024-09-14T06:18:34.521434+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:58.737457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005800637,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":2425,"num_read_kb":16285,"num_write":322,"num_write_kb":13807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"123'318","reported_seq":2937,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739738+0000","last_change":"2024-09-14T06:18:17.236905+0000","last_active":"2024-09-14T06:18:34.739738+0000","last_peered":"2024-09-14T06:18:34.739738+0000","last_clean":"2024-09-14T06:18:34.739738+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:18:34.739738+0000","last_undegraded":"2024-09-14T06:18:34.739738+0000","last_fullsized":"2024-09-14T06:18:34.739738+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:57:26.637373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013408000000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2870,"num_read_kb":16793,"num_write":272,"num_write_kb":9901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"139'192","reported_seq":1709,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:41.625281+0000","last_change":"2024-09-14T06:18:41.625281+0000","last_active":"2024-09-14T06:18:41.625281+0000","last_peered":"2024-09-14T06:18:41.625281+0000","last_clean":"2024-09-14T06:18:41.625281+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:18:41.625281+0000","last_undegraded":"2024-09-14T06:18:41.625281+0000","last_fullsized":"2024-09-14T06:18:41.625281+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'192","last_scrub_stamp":"2024-09-14T06:18:41.625180+0000","last_deep_scrub":"139'192","last_deep_scrub_stamp":"2024-09-14T06:18:41.625180+0000","last_clean_scrub_stamp":"2024-09-14T06:18:41.625180+0000","objects_scrubbed":3,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:47:55.302855+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0010333339999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1287,"num_read_kb":29081,"num_write":248,"num_write_kb":11577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"122'687","reported_seq":2640,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.522423+0000","last_change":"2024-09-14T06:18:17.337571+0000","last_active":"2024-09-14T06:18:34.522423+0000","last_peered":"2024-09-14T06:18:34.522423+0000","last_clean":"2024-09-14T06:18:34.522423+0000","last_became_active":"2024-09-14T06:17:38.227106+0000","last_became_peered":"2024-09-14T06:17:38.227106+0000","last_unstale":"2024-09-14T06:18:34.522423+0000","last_undegraded":"2024-09-14T06:18:34.522423+0000","last_fullsized":"2024-09-14T06:18:34.522423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:27:58.009450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006423425,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3768,"num_read_kb":26949,"num_write":994,"num_write_kb":20276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":29390214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"123'762","reported_seq":5456,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521727+0000","last_change":"2024-09-14T06:18:25.224296+0000","last_active":"2024-09-14T06:18:34.521727+0000","last_peered":"2024-09-14T06:18:34.521727+0000","last_clean":"2024-09-14T06:18:34.521727+0000","last_became_active":"2024-09-14T06:18:25.223198+0000","last_became_peered":"2024-09-14T06:18:25.223198+0000","last_unstale":"2024-09-14T06:18:34.521727+0000","last_undegraded":"2024-09-14T06:18:34.521727+0000","last_fullsized":"2024-09-14T06:18:34.521727+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021565500000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10255,"num_read_kb":51706,"num_write":697,"num_write_kb":16134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"123'661","reported_seq":2988,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739487+0000","last_change":"2024-09-14T06:18:17.226919+0000","last_active":"2024-09-14T06:18:34.739487+0000","last_peered":"2024-09-14T06:18:34.739487+0000","last_clean":"2024-09-14T06:18:34.739487+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:18:34.739487+0000","last_undegraded":"2024-09-14T06:18:34.739487+0000","last_fullsized":"2024-09-14T06:18:34.739487+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:55:17.915614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001432902,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4425,"num_read_kb":16166,"num_write":768,"num_write_kb":12749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"122'360","reported_seq":2606,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.738588+0000","last_change":"2024-09-14T06:18:17.221410+0000","last_active":"2024-09-14T06:18:34.738588+0000","last_peered":"2024-09-14T06:18:34.738588+0000","last_clean":"2024-09-14T06:18:34.738588+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:18:34.738588+0000","last_undegraded":"2024-09-14T06:18:34.738588+0000","last_fullsized":"2024-09-14T06:18:34.738588+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068876799999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2476,"num_read_kb":47684,"num_write":341,"num_write_kb":14643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"123'556","reported_seq":3001,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521007+0000","last_change":"2024-09-14T06:18:17.332968+0000","last_active":"2024-09-14T06:18:34.521007+0000","last_peered":"2024-09-14T06:18:34.521007+0000","last_clean":"2024-09-14T06:18:34.521007+0000","last_became_active":"2024-09-14T06:17:38.224251+0000","last_became_peered":"2024-09-14T06:17:38.224251+0000","last_unstale":"2024-09-14T06:18:34.521007+0000","last_undegraded":"2024-09-14T06:18:34.521007+0000","last_fullsized":"2024-09-14T06:18:34.521007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:32:17.880108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033873589999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4434,"num_read_kb":14632,"num_write":613,"num_write_kb":7670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":12239008,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"122'518","reported_seq":3083,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.739671+0000","last_change":"2024-09-14T06:18:17.300159+0000","last_active":"2024-09-14T06:18:34.739671+0000","last_peered":"2024-09-14T06:18:34.739671+0000","last_clean":"2024-09-14T06:18:34.739671+0000","last_became_active":"2024-09-14T06:17:37.232145+0000","last_became_peered":"2024-09-14T06:17:37.232145+0000","last_unstale":"2024-09-14T06:18:34.739671+0000","last_undegraded":"2024-09-14T06:18:34.739671+0000","last_fullsized":"2024-09-14T06:18:34.739671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:12:47.470446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016513039999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3345,"num_read_kb":11048,"num_write":606,"num_write_kb":8456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":10176554,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"122'505","reported_seq":4221,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:34.521539+0000","last_change":"2024-09-14T06:18:17.333610+0000","last_active":"2024-09-14T06:18:34.521539+0000","last_peered":"2024-09-14T06:18:34.521539+0000","last_clean":"2024-09-14T06:18:34.521539+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:18:34.521539+0000","last_undegraded":"2024-09-14T06:18:34.521539+0000","last_fullsized":"2024-09-14T06:18:34.521539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:16:19.154013+0000","last_clean_scrub_stamp":"2024-09-14T06:16:19.154013+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:51:31.054844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039616110000000003,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6037,"num_read_kb":16850,"num_write":431,"num_write_kb":5588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":210,"num_object_clones":0,"num_object_copies":420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210,"num_whiteouts":206,"num_read":107256,"num_read_kb":751154,"num_write":17465,"num_write_kb":429271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1267,"num_bytes_recovered":144403203,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15576,"ondisk_log_size":15576,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509476,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55028,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316812,"statfs":{"total":96636764160,"available":96580415488,"internally_reserved":0,"allocated":1662976,"data_stored":789529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509476,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36972,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334868,"statfs":{"total":96636764160,"available":96598904832,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509478,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73132,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298708,"statfs":{"total":96636764160,"available":96561876992,"internally_reserved":0,"allocated":1724416,"data_stored":1379897,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509477,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59172,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312668,"statfs":{"total":96636764160,"available":96576172032,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542180,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36972,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334868,"statfs":{"total":96636764160,"available":96598904832,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542180,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28652,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343188,"statfs":{"total":96636764160,"available":96607424512,"internally_reserved":0,"allocated":1662976,"data_stored":789529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28716,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343124,"statfs":{"total":96636764160,"available":96607358976,"internally_reserved":0,"allocated":1724416,"data_stored":1379897,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28652,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343188,"statfs":{"total":96636764160,"available":96607424512,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:18:45.235 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.235 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.235 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.235 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.235 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.235 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.235 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.235 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.235 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.236 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.236 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.236 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-14T06:16:09.097727+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=9, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.236 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.236 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-14T06:16:09.097727+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=9, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.236 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.236 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-14T06:16:09.097727+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=9, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.236 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.237 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-14T06:16:09.097727+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=9, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.237 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.237 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.237 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-14T06:16:09.097727+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=9, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.237 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.237 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-14T06:16:09.097727+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=9, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.237 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.237 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-14T06:16:09.097727+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=9, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.238 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.238 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-14T06:16:09.097727+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=9, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.238 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.238 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.238 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.238 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.238 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.238 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.239 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.239 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.239 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.239 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.239 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-14T06:16:19.154013+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=16, tm_sec=19, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=18, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:18:45.239 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-14T06:19:05.240 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:19:05.565 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-14T06:19:05.566 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-14T06:19:05.583 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-09-14T06:19:04.427831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":214,"num_object_clones":0,"num_object_copies":428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":214,"num_whiteouts":206,"num_read":107322,"num_read_kb":751209,"num_write":17568,"num_write_kb":430901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1270,"num_bytes_recovered":144403222,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15858,"ondisk_log_size":15858,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":351400,"kb_used_data":13096,"kb_used_omap":12,"kb_used_meta":338163,"kb_avail":754623320,"statfs":{"total":773094113280,"available":772734279680,"internally_reserved":0,"allocated":13410304,"data_stored":7496892,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":346279472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003706"},"pg_stats":[{"pgid":"3.1f","version":"139'610","reported_seq":3735,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:47.944483+0000","last_change":"2024-09-14T06:18:47.944483+0000","last_active":"2024-09-14T06:18:47.944483+0000","last_peered":"2024-09-14T06:18:47.944483+0000","last_clean":"2024-09-14T06:18:47.944483+0000","last_became_active":"2024-09-14T06:17:02.193313+0000","last_became_peered":"2024-09-14T06:17:02.193313+0000","last_unstale":"2024-09-14T06:18:47.944483+0000","last_undegraded":"2024-09-14T06:18:47.944483+0000","last_fullsized":"2024-09-14T06:18:47.944483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'610","last_scrub_stamp":"2024-09-14T06:18:47.944254+0000","last_deep_scrub":"139'610","last_deep_scrub_stamp":"2024-09-14T06:18:47.944254+0000","last_clean_scrub_stamp":"2024-09-14T06:18:47.944254+0000","objects_scrubbed":6,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:42:15.923035+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.002155396,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4617,"num_read_kb":53605,"num_write":714,"num_write_kb":19355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":5467237,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"139'415","reported_seq":4570,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:46.930357+0000","last_change":"2024-09-14T06:18:46.930357+0000","last_active":"2024-09-14T06:18:46.930357+0000","last_peered":"2024-09-14T06:18:46.930357+0000","last_clean":"2024-09-14T06:18:46.930357+0000","last_became_active":"2024-09-14T06:16:27.252395+0000","last_became_peered":"2024-09-14T06:16:27.252395+0000","last_unstale":"2024-09-14T06:18:46.930357+0000","last_undegraded":"2024-09-14T06:18:46.930357+0000","last_fullsized":"2024-09-14T06:18:46.930357+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'415","last_scrub_stamp":"2024-09-14T06:18:46.930136+0000","last_deep_scrub":"139'415","last_deep_scrub_stamp":"2024-09-14T06:18:46.930136+0000","last_clean_scrub_stamp":"2024-09-14T06:18:46.930136+0000","objects_scrubbed":8,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:29:11.720297+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0019300890000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5022,"num_read_kb":43064,"num_write":350,"num_write_kb":15485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"139'1136","reported_seq":2568,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:52.104072+0000","last_change":"2024-09-14T06:18:52.104072+0000","last_active":"2024-09-14T06:18:52.104072+0000","last_peered":"2024-09-14T06:18:52.104072+0000","last_clean":"2024-09-14T06:18:52.104072+0000","last_became_active":"2024-09-14T06:17:38.226838+0000","last_became_peered":"2024-09-14T06:17:38.226838+0000","last_unstale":"2024-09-14T06:18:52.104072+0000","last_undegraded":"2024-09-14T06:18:52.104072+0000","last_fullsized":"2024-09-14T06:18:52.104072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'1136","last_scrub_stamp":"2024-09-14T06:18:52.103882+0000","last_deep_scrub":"139'1136","last_deep_scrub_stamp":"2024-09-14T06:18:52.103882+0000","last_clean_scrub_stamp":"2024-09-14T06:18:52.103882+0000","objects_scrubbed":4,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:44:37.336043+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0062318570000000004,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5125,"num_read_kb":14378,"num_write":1577,"num_write_kb":13994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10785266,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"139'940","reported_seq":3147,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:51.600073+0000","last_change":"2024-09-14T06:18:51.600073+0000","last_active":"2024-09-14T06:18:51.600073+0000","last_peered":"2024-09-14T06:18:51.600073+0000","last_clean":"2024-09-14T06:18:51.600073+0000","last_became_active":"2024-09-14T06:17:48.737020+0000","last_became_peered":"2024-09-14T06:17:48.737020+0000","last_unstale":"2024-09-14T06:18:51.600073+0000","last_undegraded":"2024-09-14T06:18:51.600073+0000","last_fullsized":"2024-09-14T06:18:51.600073+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'940","last_scrub_stamp":"2024-09-14T06:18:51.599862+0000","last_deep_scrub":"139'940","last_deep_scrub_stamp":"2024-09-14T06:18:51.599862+0000","last_clean_scrub_stamp":"2024-09-14T06:18:51.599862+0000","objects_scrubbed":6,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:57:40.424363+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0024240709999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":6011,"num_read_kb":10015,"num_write":1345,"num_write_kb":6524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":3869232,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"139'357","reported_seq":3851,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:47.637613+0000","last_change":"2024-09-14T06:18:47.637467+0000","last_active":"2024-09-14T06:18:47.637613+0000","last_peered":"2024-09-14T06:18:47.637613+0000","last_clean":"2024-09-14T06:18:47.637613+0000","last_became_active":"2024-09-14T06:16:20.188450+0000","last_became_peered":"2024-09-14T06:16:20.188450+0000","last_unstale":"2024-09-14T06:18:47.637613+0000","last_undegraded":"2024-09-14T06:18:47.637613+0000","last_fullsized":"2024-09-14T06:18:47.637613+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'357","last_scrub_stamp":"2024-09-14T06:18:47.637243+0000","last_deep_scrub":"139'357","last_deep_scrub_stamp":"2024-09-14T06:18:47.637243+0000","last_clean_scrub_stamp":"2024-09-14T06:18:47.637243+0000","objects_scrubbed":8,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:26.521732+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0024977549999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3342,"num_read_kb":39056,"num_write":366,"num_write_kb":25723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"139'555","reported_seq":3412,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:43.246942+0000","last_change":"2024-09-14T06:18:43.246879+0000","last_active":"2024-09-14T06:18:43.246942+0000","last_peered":"2024-09-14T06:18:43.246942+0000","last_clean":"2024-09-14T06:18:43.246942+0000","last_became_active":"2024-09-14T06:16:20.191431+0000","last_became_peered":"2024-09-14T06:16:20.191431+0000","last_unstale":"2024-09-14T06:18:43.246942+0000","last_undegraded":"2024-09-14T06:18:43.246942+0000","last_fullsized":"2024-09-14T06:18:43.246942+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'555","last_scrub_stamp":"2024-09-14T06:18:43.246795+0000","last_deep_scrub":"139'555","last_deep_scrub_stamp":"2024-09-14T06:18:43.246795+0000","last_clean_scrub_stamp":"2024-09-14T06:18:43.246795+0000","objects_scrubbed":16,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:40:06.109603+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.00087057599999999997,"stat_sum":{"num_bytes":19,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":4088,"num_read_kb":13128,"num_write":658,"num_write_kb":7511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"139'539","reported_seq":2203,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:48.599322+0000","last_change":"2024-09-14T06:18:48.599322+0000","last_active":"2024-09-14T06:18:48.599322+0000","last_peered":"2024-09-14T06:18:48.599322+0000","last_clean":"2024-09-14T06:18:48.599322+0000","last_became_active":"2024-09-14T06:16:20.190503+0000","last_became_peered":"2024-09-14T06:16:20.190503+0000","last_unstale":"2024-09-14T06:18:48.599322+0000","last_undegraded":"2024-09-14T06:18:48.599322+0000","last_fullsized":"2024-09-14T06:18:48.599322+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'539","last_scrub_stamp":"2024-09-14T06:18:48.599128+0000","last_deep_scrub":"139'539","last_deep_scrub_stamp":"2024-09-14T06:18:48.599128+0000","last_clean_scrub_stamp":"2024-09-14T06:18:48.599128+0000","objects_scrubbed":7,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:47:08.402560+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001716539,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2883,"num_read_kb":19234,"num_write":704,"num_write_kb":12873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"139'405","reported_seq":3117,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:44.940846+0000","last_change":"2024-09-14T06:18:44.940689+0000","last_active":"2024-09-14T06:18:44.940846+0000","last_peered":"2024-09-14T06:18:44.940846+0000","last_clean":"2024-09-14T06:18:44.940846+0000","last_became_active":"2024-09-14T06:16:20.195807+0000","last_became_peered":"2024-09-14T06:16:20.195807+0000","last_unstale":"2024-09-14T06:18:44.940846+0000","last_undegraded":"2024-09-14T06:18:44.940846+0000","last_fullsized":"2024-09-14T06:18:44.940846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'405","last_scrub_stamp":"2024-09-14T06:18:44.940472+0000","last_deep_scrub":"139'405","last_deep_scrub_stamp":"2024-09-14T06:18:44.940472+0000","last_clean_scrub_stamp":"2024-09-14T06:18:44.940472+0000","objects_scrubbed":5,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:47:48.976641+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.002524938,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":3795,"num_read_kb":37772,"num_write":360,"num_write_kb":32941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"139'402","reported_seq":1809,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:45.973443+0000","last_change":"2024-09-14T06:18:45.973267+0000","last_active":"2024-09-14T06:18:45.973443+0000","last_peered":"2024-09-14T06:18:45.973443+0000","last_clean":"2024-09-14T06:18:45.973443+0000","last_became_active":"2024-09-14T06:17:37.233410+0000","last_became_peered":"2024-09-14T06:17:37.233410+0000","last_unstale":"2024-09-14T06:18:45.973443+0000","last_undegraded":"2024-09-14T06:18:45.973443+0000","last_fullsized":"2024-09-14T06:18:45.973443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'402","last_scrub_stamp":"2024-09-14T06:18:45.973069+0000","last_deep_scrub":"139'402","last_deep_scrub_stamp":"2024-09-14T06:18:45.973069+0000","last_clean_scrub_stamp":"2024-09-14T06:18:45.973069+0000","objects_scrubbed":7,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:14:09.804859+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0019943040000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1416,"num_read_kb":17477,"num_write":336,"num_write_kb":17193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":23467377,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"139'439","reported_seq":2432,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:44.943566+0000","last_change":"2024-09-14T06:18:44.943510+0000","last_active":"2024-09-14T06:18:44.943566+0000","last_peered":"2024-09-14T06:18:44.943566+0000","last_clean":"2024-09-14T06:18:44.943566+0000","last_became_active":"2024-09-14T06:18:25.219200+0000","last_became_peered":"2024-09-14T06:18:25.219200+0000","last_unstale":"2024-09-14T06:18:44.943566+0000","last_undegraded":"2024-09-14T06:18:44.943566+0000","last_fullsized":"2024-09-14T06:18:44.943566+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'439","last_scrub_stamp":"2024-09-14T06:18:44.943427+0000","last_deep_scrub":"139'439","last_deep_scrub_stamp":"2024-09-14T06:18:44.943427+0000","last_clean_scrub_stamp":"2024-09-14T06:18:44.943427+0000","objects_scrubbed":3,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:59:48.269665+0000","scrub_duration":3,"objects_trimmed":5,"snaptrim_duration":0.047023011000000003,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2483,"num_read_kb":16488,"num_write":510,"num_write_kb":10283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":6785399,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"139'435","reported_seq":3835,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:42.620146+0000","last_change":"2024-09-14T06:18:42.620098+0000","last_active":"2024-09-14T06:18:42.620146+0000","last_peered":"2024-09-14T06:18:42.620146+0000","last_clean":"2024-09-14T06:18:42.620146+0000","last_became_active":"2024-09-14T06:16:20.194222+0000","last_became_peered":"2024-09-14T06:16:20.194222+0000","last_unstale":"2024-09-14T06:18:42.620146+0000","last_undegraded":"2024-09-14T06:18:42.620146+0000","last_fullsized":"2024-09-14T06:18:42.620146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'435","last_scrub_stamp":"2024-09-14T06:18:42.620035+0000","last_deep_scrub":"139'435","last_deep_scrub_stamp":"2024-09-14T06:18:42.620035+0000","last_clean_scrub_stamp":"2024-09-14T06:18:42.620035+0000","objects_scrubbed":3,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:42:20.494172+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00094053699999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4134,"num_read_kb":64629,"num_write":506,"num_write_kb":22219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"139'313","reported_seq":2125,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:51.127315+0000","last_change":"2024-09-14T06:18:51.127315+0000","last_active":"2024-09-14T06:18:51.127315+0000","last_peered":"2024-09-14T06:18:51.127315+0000","last_clean":"2024-09-14T06:18:51.127315+0000","last_became_active":"2024-09-14T06:17:37.231244+0000","last_became_peered":"2024-09-14T06:17:37.231244+0000","last_unstale":"2024-09-14T06:18:51.127315+0000","last_undegraded":"2024-09-14T06:18:51.127315+0000","last_fullsized":"2024-09-14T06:18:51.127315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'313","last_scrub_stamp":"2024-09-14T06:18:51.127072+0000","last_deep_scrub":"139'313","last_deep_scrub_stamp":"2024-09-14T06:18:51.127072+0000","last_clean_scrub_stamp":"2024-09-14T06:18:51.127072+0000","objects_scrubbed":5,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:05:38.353797+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0044577879999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2916,"num_read_kb":27195,"num_write":273,"num_write_kb":15734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":32506492,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":131,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:54.077571+0000","last_change":"2024-09-14T06:18:54.077571+0000","last_active":"2024-09-14T06:18:54.077571+0000","last_peered":"2024-09-14T06:18:54.077571+0000","last_clean":"2024-09-14T06:18:54.077571+0000","last_became_active":"2024-09-14T06:17:34.200238+0000","last_became_peered":"2024-09-14T06:17:34.200238+0000","last_unstale":"2024-09-14T06:18:54.077571+0000","last_undegraded":"2024-09-14T06:18:54.077571+0000","last_fullsized":"2024-09-14T06:18:54.077571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:18:54.077372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:18:54.077372+0000","last_clean_scrub_stamp":"2024-09-14T06:18:54.077372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:31:06.269989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"139'491","reported_seq":1392,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:49.571980+0000","last_change":"2024-09-14T06:18:49.571980+0000","last_active":"2024-09-14T06:18:49.571980+0000","last_peered":"2024-09-14T06:18:49.571980+0000","last_clean":"2024-09-14T06:18:49.571980+0000","last_became_active":"2024-09-14T06:16:20.186033+0000","last_became_peered":"2024-09-14T06:16:20.186033+0000","last_unstale":"2024-09-14T06:18:49.571980+0000","last_undegraded":"2024-09-14T06:18:49.571980+0000","last_fullsized":"2024-09-14T06:18:49.571980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'491","last_scrub_stamp":"2024-09-14T06:18:49.571782+0000","last_deep_scrub":"139'491","last_deep_scrub_stamp":"2024-09-14T06:18:49.571782+0000","last_clean_scrub_stamp":"2024-09-14T06:18:49.571782+0000","objects_scrubbed":4,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:39:06.658237+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0030371690000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1562,"num_read_kb":9726,"num_write":495,"num_write_kb":9169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":261,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:44.680237+0000","last_change":"2024-09-14T06:18:44.680237+0000","last_active":"2024-09-14T06:18:44.680237+0000","last_peered":"2024-09-14T06:18:44.680237+0000","last_clean":"2024-09-14T06:18:44.680237+0000","last_became_active":"2024-09-14T06:16:10.145597+0000","last_became_peered":"2024-09-14T06:16:10.145597+0000","last_unstale":"2024-09-14T06:18:44.680237+0000","last_undegraded":"2024-09-14T06:18:44.680237+0000","last_fullsized":"2024-09-14T06:18:44.680237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:18:44.680021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:18:44.680021+0000","last_clean_scrub_stamp":"2024-09-14T06:18:44.680021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:22:01.142660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090627000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"139'388","reported_seq":3331,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:45.704099+0000","last_change":"2024-09-14T06:18:45.704099+0000","last_active":"2024-09-14T06:18:45.704099+0000","last_peered":"2024-09-14T06:18:45.704099+0000","last_clean":"2024-09-14T06:18:45.704099+0000","last_became_active":"2024-09-14T06:16:20.193557+0000","last_became_peered":"2024-09-14T06:16:20.193557+0000","last_unstale":"2024-09-14T06:18:45.704099+0000","last_undegraded":"2024-09-14T06:18:45.704099+0000","last_fullsized":"2024-09-14T06:18:45.704099+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'388","last_scrub_stamp":"2024-09-14T06:18:45.703912+0000","last_deep_scrub":"139'388","last_deep_scrub_stamp":"2024-09-14T06:18:45.703912+0000","last_clean_scrub_stamp":"2024-09-14T06:18:45.703912+0000","objects_scrubbed":6,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:34:02.690617+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0030864040000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3293,"num_read_kb":28583,"num_write":338,"num_write_kb":15237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":131,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:46.043148+0000","last_change":"2024-09-14T06:18:46.043148+0000","last_active":"2024-09-14T06:18:46.043148+0000","last_peered":"2024-09-14T06:18:46.043148+0000","last_clean":"2024-09-14T06:18:46.043148+0000","last_became_active":"2024-09-14T06:17:34.196072+0000","last_became_peered":"2024-09-14T06:17:34.196072+0000","last_unstale":"2024-09-14T06:18:46.043148+0000","last_undegraded":"2024-09-14T06:18:46.043148+0000","last_fullsized":"2024-09-14T06:18:46.043148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:18:46.042970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:18:46.042970+0000","last_clean_scrub_stamp":"2024-09-14T06:18:46.042970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:21:15.513792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"139'372","reported_seq":2046,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:46.058906+0000","last_change":"2024-09-14T06:18:46.058906+0000","last_active":"2024-09-14T06:18:46.058906+0000","last_peered":"2024-09-14T06:18:46.058906+0000","last_clean":"2024-09-14T06:18:46.058906+0000","last_became_active":"2024-09-14T06:16:20.188238+0000","last_became_peered":"2024-09-14T06:16:20.188238+0000","last_unstale":"2024-09-14T06:18:46.058906+0000","last_undegraded":"2024-09-14T06:18:46.058906+0000","last_fullsized":"2024-09-14T06:18:46.058906+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'372","last_scrub_stamp":"2024-09-14T06:18:46.058725+0000","last_deep_scrub":"139'372","last_deep_scrub_stamp":"2024-09-14T06:18:46.058725+0000","last_clean_scrub_stamp":"2024-09-14T06:18:46.058725+0000","objects_scrubbed":3,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:03.370811+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0018818730000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1780,"num_read_kb":12934,"num_write":307,"num_write_kb":11582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":263,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:43.933359+0000","last_change":"2024-09-14T06:18:43.933359+0000","last_active":"2024-09-14T06:18:43.933359+0000","last_peered":"2024-09-14T06:18:43.933359+0000","last_clean":"2024-09-14T06:18:43.933359+0000","last_became_active":"2024-09-14T06:16:10.197746+0000","last_became_peered":"2024-09-14T06:16:10.197746+0000","last_unstale":"2024-09-14T06:18:43.933359+0000","last_undegraded":"2024-09-14T06:18:43.933359+0000","last_fullsized":"2024-09-14T06:18:43.933359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:18:43.933156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:18:43.933156+0000","last_clean_scrub_stamp":"2024-09-14T06:18:43.933156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:59:52.387944+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0013436800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"139'282","reported_seq":1784,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:43.963408+0000","last_change":"2024-09-14T06:18:43.963408+0000","last_active":"2024-09-14T06:18:43.963408+0000","last_peered":"2024-09-14T06:18:43.963408+0000","last_clean":"2024-09-14T06:18:43.963408+0000","last_became_active":"2024-09-14T06:18:25.221823+0000","last_became_peered":"2024-09-14T06:18:25.221823+0000","last_unstale":"2024-09-14T06:18:43.963408+0000","last_undegraded":"2024-09-14T06:18:43.963408+0000","last_fullsized":"2024-09-14T06:18:43.963408+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'282","last_scrub_stamp":"2024-09-14T06:18:43.963311+0000","last_deep_scrub":"139'282","last_deep_scrub_stamp":"2024-09-14T06:18:43.963311+0000","last_clean_scrub_stamp":"2024-09-14T06:18:43.963311+0000","objects_scrubbed":10,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:24:01.791281+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0019482589999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1362,"num_read_kb":21261,"num_write":264,"num_write_kb":9954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":606696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"139'400","reported_seq":1262,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:43.588818+0000","last_change":"2024-09-14T06:18:43.588818+0000","last_active":"2024-09-14T06:18:43.588818+0000","last_peered":"2024-09-14T06:18:43.588818+0000","last_clean":"2024-09-14T06:18:43.588818+0000","last_became_active":"2024-09-14T06:16:27.249635+0000","last_became_peered":"2024-09-14T06:16:27.249635+0000","last_unstale":"2024-09-14T06:18:43.588818+0000","last_undegraded":"2024-09-14T06:18:43.588818+0000","last_fullsized":"2024-09-14T06:18:43.588818+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'400","last_scrub_stamp":"2024-09-14T06:18:43.588720+0000","last_deep_scrub":"139'400","last_deep_scrub_stamp":"2024-09-14T06:18:43.588720+0000","last_clean_scrub_stamp":"2024-09-14T06:18:43.588720+0000","objects_scrubbed":7,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:13:03.192045+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001556469,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":547,"num_read_kb":4255,"num_write":315,"num_write_kb":7133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":42510,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"139'3","reported_seq":136,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:44.074148+0000","last_change":"2024-09-14T06:18:44.074148+0000","last_active":"2024-09-14T06:18:44.074148+0000","last_peered":"2024-09-14T06:18:44.074148+0000","last_clean":"2024-09-14T06:18:44.074148+0000","last_became_active":"2024-09-14T06:17:34.199244+0000","last_became_peered":"2024-09-14T06:17:34.199244+0000","last_unstale":"2024-09-14T06:18:44.074148+0000","last_undegraded":"2024-09-14T06:18:44.074148+0000","last_fullsized":"2024-09-14T06:18:44.074148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'3","last_scrub_stamp":"2024-09-14T06:18:44.074057+0000","last_deep_scrub":"139'3","last_deep_scrub_stamp":"2024-09-14T06:18:44.074057+0000","last_clean_scrub_stamp":"2024-09-14T06:18:44.074057+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:12:11.462952+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"139'415","reported_seq":2052,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:46.662393+0000","last_change":"2024-09-14T06:18:46.662393+0000","last_active":"2024-09-14T06:18:46.662393+0000","last_peered":"2024-09-14T06:18:46.662393+0000","last_clean":"2024-09-14T06:18:46.662393+0000","last_became_active":"2024-09-14T06:18:17.230545+0000","last_became_peered":"2024-09-14T06:18:17.230545+0000","last_unstale":"2024-09-14T06:18:46.662393+0000","last_undegraded":"2024-09-14T06:18:46.662393+0000","last_fullsized":"2024-09-14T06:18:46.662393+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'415","last_scrub_stamp":"2024-09-14T06:18:46.662180+0000","last_deep_scrub":"139'415","last_deep_scrub_stamp":"2024-09-14T06:18:46.662180+0000","last_clean_scrub_stamp":"2024-09-14T06:18:46.662180+0000","objects_scrubbed":8,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:46:30.054803+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0048632429999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2097,"num_read_kb":18969,"num_write":521,"num_write_kb":11051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":36437,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":262,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:45.522960+0000","last_change":"2024-09-14T06:18:45.522960+0000","last_active":"2024-09-14T06:18:45.522960+0000","last_peered":"2024-09-14T06:18:45.522960+0000","last_clean":"2024-09-14T06:18:45.522960+0000","last_became_active":"2024-09-14T06:16:10.145549+0000","last_became_peered":"2024-09-14T06:16:10.145549+0000","last_unstale":"2024-09-14T06:18:45.522960+0000","last_undegraded":"2024-09-14T06:18:45.522960+0000","last_fullsized":"2024-09-14T06:18:45.522960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:18:45.522675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:18:45.522675+0000","last_clean_scrub_stamp":"2024-09-14T06:18:45.522675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:59:55.455080+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00077148099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"139'545","reported_seq":2254,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:45.071463+0000","last_change":"2024-09-14T06:18:45.071463+0000","last_active":"2024-09-14T06:18:45.071463+0000","last_peered":"2024-09-14T06:18:45.071463+0000","last_clean":"2024-09-14T06:18:45.071463+0000","last_became_active":"2024-09-14T06:16:27.252340+0000","last_became_peered":"2024-09-14T06:16:27.252340+0000","last_unstale":"2024-09-14T06:18:45.071463+0000","last_undegraded":"2024-09-14T06:18:45.071463+0000","last_fullsized":"2024-09-14T06:18:45.071463+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'545","last_scrub_stamp":"2024-09-14T06:18:45.071265+0000","last_deep_scrub":"139'545","last_deep_scrub_stamp":"2024-09-14T06:18:45.071265+0000","last_clean_scrub_stamp":"2024-09-14T06:18:45.071265+0000","objects_scrubbed":5,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:33:38.313148+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001640601,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2885,"num_read_kb":8812,"num_write":721,"num_write_kb":8784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":261,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:43.117792+0000","last_change":"2024-09-14T06:18:43.117792+0000","last_active":"2024-09-14T06:18:43.117792+0000","last_peered":"2024-09-14T06:18:43.117792+0000","last_clean":"2024-09-14T06:18:43.117792+0000","last_became_active":"2024-09-14T06:16:10.149935+0000","last_became_peered":"2024-09-14T06:16:10.149935+0000","last_unstale":"2024-09-14T06:18:43.117792+0000","last_undegraded":"2024-09-14T06:18:43.117792+0000","last_fullsized":"2024-09-14T06:18:43.117792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:18:43.117611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:18:43.117611+0000","last_clean_scrub_stamp":"2024-09-14T06:18:43.117611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:51:44.908599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011423550000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"139'477","reported_seq":2092,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:43.708729+0000","last_change":"2024-09-14T06:18:43.708729+0000","last_active":"2024-09-14T06:18:43.708729+0000","last_peered":"2024-09-14T06:18:43.708729+0000","last_clean":"2024-09-14T06:18:43.708729+0000","last_became_active":"2024-09-14T06:16:27.253951+0000","last_became_peered":"2024-09-14T06:16:27.253951+0000","last_unstale":"2024-09-14T06:18:43.708729+0000","last_undegraded":"2024-09-14T06:18:43.708729+0000","last_fullsized":"2024-09-14T06:18:43.708729+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'477","last_scrub_stamp":"2024-09-14T06:18:43.708550+0000","last_deep_scrub":"139'477","last_deep_scrub_stamp":"2024-09-14T06:18:43.708550+0000","last_clean_scrub_stamp":"2024-09-14T06:18:43.708550+0000","objects_scrubbed":6,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:05:48.419393+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001487162,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2220,"num_read_kb":23013,"num_write":375,"num_write_kb":14523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"139'2","reported_seq":283,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:45.038817+0000","last_change":"2024-09-14T06:18:45.038817+0000","last_active":"2024-09-14T06:18:45.038817+0000","last_peered":"2024-09-14T06:18:45.038817+0000","last_clean":"2024-09-14T06:18:45.038817+0000","last_became_active":"2024-09-14T06:16:27.251108+0000","last_became_peered":"2024-09-14T06:16:27.251108+0000","last_unstale":"2024-09-14T06:18:45.038817+0000","last_undegraded":"2024-09-14T06:18:45.038817+0000","last_fullsized":"2024-09-14T06:18:45.038817+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'2","last_scrub_stamp":"2024-09-14T06:18:45.038746+0000","last_deep_scrub":"139'2","last_deep_scrub_stamp":"2024-09-14T06:18:45.038746+0000","last_clean_scrub_stamp":"2024-09-14T06:18:45.038746+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:55:38.194429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034029939999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"139'74","reported_seq":355,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:42.235742+0000","last_change":"2024-09-14T06:18:42.235742+0000","last_active":"2024-09-14T06:18:42.235742+0000","last_peered":"2024-09-14T06:18:42.235742+0000","last_clean":"2024-09-14T06:18:42.235742+0000","last_became_active":"2024-09-14T06:16:04.633789+0000","last_became_peered":"2024-09-14T06:16:04.633789+0000","last_unstale":"2024-09-14T06:18:42.235742+0000","last_undegraded":"2024-09-14T06:18:42.235742+0000","last_fullsized":"2024-09-14T06:18:42.235742+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'74","last_scrub_stamp":"2024-09-14T06:18:42.235682+0000","last_deep_scrub":"139'74","last_deep_scrub_stamp":"2024-09-14T06:18:42.235682+0000","last_clean_scrub_stamp":"2024-09-14T06:18:42.235682+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:11:24.823765+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"139'451","reported_seq":1726,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:50.570789+0000","last_change":"2024-09-14T06:18:50.570789+0000","last_active":"2024-09-14T06:18:50.570789+0000","last_peered":"2024-09-14T06:18:50.570789+0000","last_clean":"2024-09-14T06:18:50.570789+0000","last_became_active":"2024-09-14T06:16:20.186278+0000","last_became_peered":"2024-09-14T06:16:20.186278+0000","last_unstale":"2024-09-14T06:18:50.570789+0000","last_undegraded":"2024-09-14T06:18:50.570789+0000","last_fullsized":"2024-09-14T06:18:50.570789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'451","last_scrub_stamp":"2024-09-14T06:18:50.570584+0000","last_deep_scrub":"139'451","last_deep_scrub_stamp":"2024-09-14T06:18:50.570584+0000","last_clean_scrub_stamp":"2024-09-14T06:18:50.570584+0000","objects_scrubbed":9,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:04:03.079929+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.00348241,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2133,"num_read_kb":6715,"num_write":589,"num_write_kb":6849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"139'400","reported_seq":1914,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:47.095134+0000","last_change":"2024-09-14T06:18:47.095134+0000","last_active":"2024-09-14T06:18:47.095134+0000","last_peered":"2024-09-14T06:18:47.095134+0000","last_clean":"2024-09-14T06:18:47.095134+0000","last_became_active":"2024-09-14T06:17:38.224238+0000","last_became_peered":"2024-09-14T06:17:38.224238+0000","last_unstale":"2024-09-14T06:18:47.095134+0000","last_undegraded":"2024-09-14T06:18:47.095134+0000","last_fullsized":"2024-09-14T06:18:47.095134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'400","last_scrub_stamp":"2024-09-14T06:18:47.094910+0000","last_deep_scrub":"139'400","last_deep_scrub_stamp":"2024-09-14T06:18:47.094910+0000","last_clean_scrub_stamp":"2024-09-14T06:18:47.094910+0000","objects_scrubbed":2,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:54:48.293340+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0064732729999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2223,"num_read_kb":13651,"num_write":549,"num_write_kb":14353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8477378,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"139'389","reported_seq":2457,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:49.120104+0000","last_change":"2024-09-14T06:18:49.120104+0000","last_active":"2024-09-14T06:18:49.120104+0000","last_peered":"2024-09-14T06:18:49.120104+0000","last_clean":"2024-09-14T06:18:49.120104+0000","last_became_active":"2024-09-14T06:16:20.190619+0000","last_became_peered":"2024-09-14T06:16:20.190619+0000","last_unstale":"2024-09-14T06:18:49.120104+0000","last_undegraded":"2024-09-14T06:18:49.120104+0000","last_fullsized":"2024-09-14T06:18:49.120104+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'389","last_scrub_stamp":"2024-09-14T06:18:49.119895+0000","last_deep_scrub":"139'389","last_deep_scrub_stamp":"2024-09-14T06:18:49.119895+0000","last_clean_scrub_stamp":"2024-09-14T06:18:49.119895+0000","objects_scrubbed":5,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:54:56.858916+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.005800637,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":2425,"num_read_kb":16285,"num_write":322,"num_write_kb":13807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"139'326","reported_seq":2953,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:46.550492+0000","last_change":"2024-09-14T06:18:46.550492+0000","last_active":"2024-09-14T06:18:46.550492+0000","last_peered":"2024-09-14T06:18:46.550492+0000","last_clean":"2024-09-14T06:18:46.550492+0000","last_became_active":"2024-09-14T06:16:20.194232+0000","last_became_peered":"2024-09-14T06:16:20.194232+0000","last_unstale":"2024-09-14T06:18:46.550492+0000","last_undegraded":"2024-09-14T06:18:46.550492+0000","last_fullsized":"2024-09-14T06:18:46.550492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'326","last_scrub_stamp":"2024-09-14T06:18:46.550236+0000","last_deep_scrub":"139'326","last_deep_scrub_stamp":"2024-09-14T06:18:46.550236+0000","last_clean_scrub_stamp":"2024-09-14T06:18:46.550236+0000","objects_scrubbed":8,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:02:47.814755+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0013408000000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2870,"num_read_kb":16793,"num_write":272,"num_write_kb":9901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"139'192","reported_seq":1709,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:41.625281+0000","last_change":"2024-09-14T06:18:41.625281+0000","last_active":"2024-09-14T06:18:41.625281+0000","last_peered":"2024-09-14T06:18:41.625281+0000","last_clean":"2024-09-14T06:18:41.625281+0000","last_became_active":"2024-09-14T06:16:20.194382+0000","last_became_peered":"2024-09-14T06:16:20.194382+0000","last_unstale":"2024-09-14T06:18:41.625281+0000","last_undegraded":"2024-09-14T06:18:41.625281+0000","last_fullsized":"2024-09-14T06:18:41.625281+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'192","last_scrub_stamp":"2024-09-14T06:18:41.625180+0000","last_deep_scrub":"139'192","last_deep_scrub_stamp":"2024-09-14T06:18:41.625180+0000","last_clean_scrub_stamp":"2024-09-14T06:18:41.625180+0000","objects_scrubbed":3,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:47:55.302855+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0010333339999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1287,"num_read_kb":29081,"num_write":248,"num_write_kb":11577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"139'695","reported_seq":2656,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:50.115029+0000","last_change":"2024-09-14T06:18:50.115029+0000","last_active":"2024-09-14T06:18:50.115029+0000","last_peered":"2024-09-14T06:18:50.115029+0000","last_clean":"2024-09-14T06:18:50.115029+0000","last_became_active":"2024-09-14T06:17:38.227106+0000","last_became_peered":"2024-09-14T06:17:38.227106+0000","last_unstale":"2024-09-14T06:18:50.115029+0000","last_undegraded":"2024-09-14T06:18:50.115029+0000","last_fullsized":"2024-09-14T06:18:50.115029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'695","last_scrub_stamp":"2024-09-14T06:18:50.114829+0000","last_deep_scrub":"139'695","last_deep_scrub_stamp":"2024-09-14T06:18:50.114829+0000","last_clean_scrub_stamp":"2024-09-14T06:18:50.114829+0000","objects_scrubbed":8,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:17:14.676427+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.006423425,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3768,"num_read_kb":26949,"num_write":994,"num_write_kb":20276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":29390214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"139'770","reported_seq":5472,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:42.985991+0000","last_change":"2024-09-14T06:18:42.985991+0000","last_active":"2024-09-14T06:18:42.985991+0000","last_peered":"2024-09-14T06:18:42.985991+0000","last_clean":"2024-09-14T06:18:42.985991+0000","last_became_active":"2024-09-14T06:18:25.223198+0000","last_became_peered":"2024-09-14T06:18:25.223198+0000","last_unstale":"2024-09-14T06:18:42.985991+0000","last_undegraded":"2024-09-14T06:18:42.985991+0000","last_fullsized":"2024-09-14T06:18:42.985991+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'770","last_scrub_stamp":"2024-09-14T06:18:42.985891+0000","last_deep_scrub":"139'770","last_deep_scrub_stamp":"2024-09-14T06:18:42.985891+0000","last_clean_scrub_stamp":"2024-09-14T06:18:42.985891+0000","objects_scrubbed":8,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:39:59.160089+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0021565500000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10255,"num_read_kb":51706,"num_write":697,"num_write_kb":16134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10530,"num_keys_recovered":9,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'669","reported_seq":3004,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:44.601535+0000","last_change":"2024-09-14T06:18:44.601535+0000","last_active":"2024-09-14T06:18:44.601535+0000","last_peered":"2024-09-14T06:18:44.601535+0000","last_clean":"2024-09-14T06:18:44.601535+0000","last_became_active":"2024-09-14T06:16:20.191921+0000","last_became_peered":"2024-09-14T06:16:20.191921+0000","last_unstale":"2024-09-14T06:18:44.601535+0000","last_undegraded":"2024-09-14T06:18:44.601535+0000","last_fullsized":"2024-09-14T06:18:44.601535+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'669","last_scrub_stamp":"2024-09-14T06:18:44.601390+0000","last_deep_scrub":"139'669","last_deep_scrub_stamp":"2024-09-14T06:18:44.601390+0000","last_clean_scrub_stamp":"2024-09-14T06:18:44.601390+0000","objects_scrubbed":8,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:51:33.705621+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001432902,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4425,"num_read_kb":16166,"num_write":768,"num_write_kb":12749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"139'368","reported_seq":2622,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:44.246234+0000","last_change":"2024-09-14T06:18:44.246234+0000","last_active":"2024-09-14T06:18:44.246234+0000","last_peered":"2024-09-14T06:18:44.246234+0000","last_clean":"2024-09-14T06:18:44.246234+0000","last_became_active":"2024-09-14T06:16:20.199077+0000","last_became_peered":"2024-09-14T06:16:20.199077+0000","last_unstale":"2024-09-14T06:18:44.246234+0000","last_undegraded":"2024-09-14T06:18:44.246234+0000","last_fullsized":"2024-09-14T06:18:44.246234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'368","last_scrub_stamp":"2024-09-14T06:18:44.246113+0000","last_deep_scrub":"139'368","last_deep_scrub_stamp":"2024-09-14T06:18:44.246113+0000","last_clean_scrub_stamp":"2024-09-14T06:18:44.246113+0000","objects_scrubbed":8,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:57:39.380498+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00068876799999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2476,"num_read_kb":47684,"num_write":341,"num_write_kb":14643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"139'561","reported_seq":3014,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:48.110296+0000","last_change":"2024-09-14T06:18:48.110296+0000","last_active":"2024-09-14T06:18:48.110296+0000","last_peered":"2024-09-14T06:18:48.110296+0000","last_clean":"2024-09-14T06:18:48.110296+0000","last_became_active":"2024-09-14T06:17:38.224251+0000","last_became_peered":"2024-09-14T06:17:38.224251+0000","last_unstale":"2024-09-14T06:18:48.110296+0000","last_undegraded":"2024-09-14T06:18:48.110296+0000","last_fullsized":"2024-09-14T06:18:48.110296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'561","last_scrub_stamp":"2024-09-14T06:18:48.110102+0000","last_deep_scrub":"139'561","last_deep_scrub_stamp":"2024-09-14T06:18:48.110102+0000","last_clean_scrub_stamp":"2024-09-14T06:18:48.110102+0000","objects_scrubbed":5,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:17:00.375702+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0033873589999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4434,"num_read_kb":14632,"num_write":613,"num_write_kb":7670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":12239008,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"139'527","reported_seq":3100,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:47.075019+0000","last_change":"2024-09-14T06:18:47.075019+0000","last_active":"2024-09-14T06:18:47.075019+0000","last_peered":"2024-09-14T06:18:47.075019+0000","last_clean":"2024-09-14T06:18:47.075019+0000","last_became_active":"2024-09-14T06:17:37.232145+0000","last_became_peered":"2024-09-14T06:17:37.232145+0000","last_unstale":"2024-09-14T06:18:47.075019+0000","last_undegraded":"2024-09-14T06:18:47.075019+0000","last_fullsized":"2024-09-14T06:18:47.075019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'527","last_scrub_stamp":"2024-09-14T06:18:47.074787+0000","last_deep_scrub":"139'527","last_deep_scrub_stamp":"2024-09-14T06:18:47.074787+0000","last_clean_scrub_stamp":"2024-09-14T06:18:47.074787+0000","objects_scrubbed":9,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:12:39.682577+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0016513039999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3345,"num_read_kb":11048,"num_write":606,"num_write_kb":8456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":10176554,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"139'515","reported_seq":4239,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:18:53.065207+0000","last_change":"2024-09-14T06:18:53.065207+0000","last_active":"2024-09-14T06:18:53.065207+0000","last_peered":"2024-09-14T06:18:53.065207+0000","last_clean":"2024-09-14T06:18:53.065207+0000","last_became_active":"2024-09-14T06:16:20.190500+0000","last_became_peered":"2024-09-14T06:16:20.190500+0000","last_unstale":"2024-09-14T06:18:53.065207+0000","last_undegraded":"2024-09-14T06:18:53.065207+0000","last_fullsized":"2024-09-14T06:18:53.065207+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'515","last_scrub_stamp":"2024-09-14T06:18:53.065004+0000","last_deep_scrub":"139'515","last_deep_scrub_stamp":"2024-09-14T06:18:53.065004+0000","last_clean_scrub_stamp":"2024-09-14T06:18:53.065004+0000","objects_scrubbed":10,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:10:21.976129+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0039616110000000003,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6037,"num_read_kb":16850,"num_write":431,"num_write_kb":5588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":210,"num_object_clones":0,"num_object_copies":420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210,"num_whiteouts":206,"num_read":107256,"num_read_kb":751154,"num_write":17465,"num_write_kb":429271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1267,"num_bytes_recovered":144403203,"num_keys_recovered":219,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15779,"ondisk_log_size":15779,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":20380,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55016,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316824,"statfs":{"total":96636764160,"available":96580427776,"internally_reserved":0,"allocated":1662976,"data_stored":789529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36964,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334876,"statfs":{"total":96636764160,"available":96598913024,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77284,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294556,"statfs":{"total":96636764160,"available":96557625344,"internally_reserved":0,"allocated":1724416,"data_stored":1379897,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59172,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312668,"statfs":{"total":96636764160,"available":96576172032,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542184,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36964,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334876,"statfs":{"total":96636764160,"available":96598913024,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542184,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343192,"statfs":{"total":96636764160,"available":96607428608,"internally_reserved":0,"allocated":1662976,"data_stored":789529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542183,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28708,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343132,"statfs":{"total":96636764160,"available":96607367168,"internally_reserved":0,"allocated":1724416,"data_stored":1379897,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343196,"statfs":{"total":96636764160,"available":96607432704,"internally_reserved":0,"allocated":1658880,"data_stored":789510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":10190,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:19:05.586 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-14T06:19:05.930 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-14T06:19:05.930 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-14T06:19:05.931 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-14T06:19:05.931 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:19:11.895 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-14T06:19:12.033 INFO:tasks.ceph.osd.0:Stopped 2024-09-14T06:19:12.033 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-14T06:19:12.033 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:19:18.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-14T06:19:18.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-14T06:19:18.135 INFO:tasks.ceph.osd.1:Stopped 2024-09-14T06:19:18.135 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-14T06:19:18.135 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:19:18.203 INFO:tasks.ceph.osd.2:Stopped 2024-09-14T06:19:18.203 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-14T06:19:18.204 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:19:18.290 INFO:tasks.ceph.osd.3:Stopped 2024-09-14T06:19:18.290 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-14T06:19:18.291 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:19:18.369 INFO:tasks.ceph.osd.4:Stopped 2024-09-14T06:19:18.369 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-14T06:19:18.369 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:19:18.447 INFO:tasks.ceph.osd.5:Stopped 2024-09-14T06:19:18.447 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-14T06:19:18.447 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:19:23.609 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-14T06:19:23.609 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-14T06:19:24.549 INFO:tasks.ceph.osd.6:Stopped 2024-09-14T06:19:24.550 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-14T06:19:24.550 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:19:24.614 INFO:tasks.ceph.osd.7:Stopped 2024-09-14T06:19:24.615 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-14T06:19:24.615 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-14T06:19:24.615 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:19:24.670 INFO:tasks.ceph.mgr.y:Stopped 2024-09-14T06:19:24.671 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-14T06:19:24.671 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:19:24.699 INFO:tasks.ceph.mgr.x:Stopped 2024-09-14T06:19:24.699 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-14T06:19:24.699 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-14T06:19:24.699 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:19:24.717 INFO:tasks.ceph.mon.a:Stopped 2024-09-14T06:19:24.717 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-14T06:19:24.717 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:19:24.782 INFO:tasks.ceph.mon.c:Stopped 2024-09-14T06:19:24.782 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-14T06:19:24.783 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:19:24.807 INFO:tasks.ceph.mon.b:Stopped 2024-09-14T06:19:24.807 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-14T06:19:24.807 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-14T06:19:24.841 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-14T06:19:24.841 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-14T06:19:25.228 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-14T06:19:25.228 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-14T06:19:25.396 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-14T06:19:25.396 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-14T06:19:25.510 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-14T06:19:25.510 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-14T06:19:25.647 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi087.front.sepia.ceph.com 2024-09-14T06:19:25.647 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-14T06:19:25.835 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi087.front.sepia.ceph.com 2024-09-14T06:19:25.835 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-14T06:19:26.036 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi087.front.sepia.ceph.com 2024-09-14T06:19:26.036 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-14T06:19:26.209 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi087.front.sepia.ceph.com 2024-09-14T06:19:26.209 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-14T06:19:26.369 INFO:tasks.ceph:Archiving mon data... 2024-09-14T06:19:26.370 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900364/data/mon.a.tgz 2024-09-14T06:19:26.370 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-09-14T06:19:26.376 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.qBUFbzcuLj 2024-09-14T06:19:26.376 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.qBUFbzcuLj 2024-09-14T06:19:26.525 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.qBUFbzcuLj 2024-09-14T06:19:26.588 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.qBUFbzcuLj is 382KB 2024-09-14T06:19:26.690 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.qBUFbzcuLj 2024-09-14T06:19:26.696 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900364/data/mon.c.tgz 2024-09-14T06:19:26.696 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-09-14T06:19:26.744 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.aH0EhTFhBP 2024-09-14T06:19:26.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.aH0EhTFhBP 2024-09-14T06:19:26.921 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.aH0EhTFhBP 2024-09-14T06:19:26.984 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.aH0EhTFhBP is 514KB 2024-09-14T06:19:27.045 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.aH0EhTFhBP 2024-09-14T06:19:27.050 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900364/data/mon.b.tgz 2024-09-14T06:19:27.051 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2024-09-14T06:19:27.056 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.4B4GaFkQec 2024-09-14T06:19:27.057 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.4B4GaFkQec 2024-09-14T06:19:27.226 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.4B4GaFkQec 2024-09-14T06:19:27.289 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.4B4GaFkQec is 513KB 2024-09-14T06:19:27.354 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.4B4GaFkQec 2024-09-14T06:19:27.359 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-14T06:19:27.360 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T06:19:27.362 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T06:19:27.422 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T06:19:27.465 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T06:19:27.465 INFO:tasks.ceph:Archiving crash dumps... 2024-09-14T06:19:27.466 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900364/remote/smithi039/crash 2024-09-14T06:19:27.467 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T06:19:27.481 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900364/remote/smithi087/crash 2024-09-14T06:19:27.482 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T06:19:27.520 INFO:tasks.ceph:Compressing logs... 2024-09-14T06:19:27.521 DEBUG:teuthology.orchestra.run.smithi039:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T06:19:27.529 DEBUG:teuthology.orchestra.run.smithi087:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T06:19:27.560 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31434.log 2024-09-14T06:19:27.560 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39054.log 2024-09-14T06:19:27.561 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31434.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31434.log.gz 2024-09-14T06:19:27.561 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35035.log 2024-09-14T06:19:27.562 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39054.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39054.log.gz 2024-09-14T06:19:27.563 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34864.log 2024-09-14T06:19:27.563 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34090.log 2024-09-14T06:19:27.564 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35035.log: /var/log/ceph/ceph-client.admin.34864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38280.log 2024-09-14T06:19:27.564 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34864.log.gz 2024-09-14T06:19:27.564 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35035.log.gz 2024-09-14T06:19:27.564 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34090.log.gz 2024-09-14T06:19:27.564 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38950.log 2024-09-14T06:19:27.564 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33983.log 2024-09-14T06:19:27.564 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38280.log.gz 2024-09-14T06:19:27.568 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34747.log 2024-09-14T06:19:27.568 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38950.log.gz 2024-09-14T06:19:27.568 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36937.log 2024-09-14T06:19:27.568 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33983.log.gz 2024-09-14T06:19:27.568 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34747.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34747.log.gz 2024-09-14T06:19:27.568 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37515.log 2024-09-14T06:19:27.568 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36937.log.gz 2024-09-14T06:19:27.568 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-14T06:19:27.568 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39184.log 2024-09-14T06:19:27.568 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37515.log.gz 2024-09-14T06:19:27.569 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33565.log 2024-09-14T06:19:27.569 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.39184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39132.log 2024-09-14T06:19:27.569 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39184.log.gz 2024-09-14T06:19:27.569 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33565.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33565.log.gz 2024-09-14T06:19:27.569 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31075.log 2024-09-14T06:19:27.569 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30247.log 2024-09-14T06:19:27.569 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39132.log.gz 2024-09-14T06:19:27.569 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31075.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31075.log.gz 2024-09-14T06:19:27.569 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34409.log 2024-09-14T06:19:27.573 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32042.log 2024-09-14T06:19:27.573 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30247.log: /var/log/ceph/ceph-client.admin.34409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36234.log 2024-09-14T06:19:27.573 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34409.log.gz 2024-09-14T06:19:27.574 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30247.log.gz 2024-09-14T06:19:27.574 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32042.log.gz 2024-09-14T06:19:27.574 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38287.log 2024-09-14T06:19:27.574 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38088.log 2024-09-14T06:19:27.574 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36234.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36234.log.gz 2024-09-14T06:19:27.574 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35138.log 2024-09-14T06:19:27.574 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38287.log.gz 2024-09-14T06:19:27.574 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36568.log 2024-09-14T06:19:27.574 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38088.log.gz 2024-09-14T06:19:27.574 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35138.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35138.log.gz -5 2024-09-14T06:19:27.574 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-14T06:19:27.574 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37728.log 2024-09-14T06:19:27.574 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36568.log.gz 2024-09-14T06:19:27.575 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33158.log 2024-09-14T06:19:27.575 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31774.log 2024-09-14T06:19:27.575 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37728.log.gz 2024-09-14T06:19:27.575 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33158.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33158.log.gz -5 2024-09-14T06:19:27.575 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32194.log 2024-09-14T06:19:27.575 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31774.log.gz 2024-09-14T06:19:27.575 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31266.log 2024-09-14T06:19:27.575 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38508.log 2024-09-14T06:19:27.575 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32194.log.gz 2024-09-14T06:19:27.575 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31266.log.gz 2024-09-14T06:19:27.580 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32731.log 2024-09-14T06:19:27.580 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38014.log 2024-09-14T06:19:27.580 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38508.log.gz 2024-09-14T06:19:27.580 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32731.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32731.log.gz 2024-09-14T06:19:27.580 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30659.log 2024-09-14T06:19:27.580 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37576.log 2024-09-14T06:19:27.581 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38014.log.gz 2024-09-14T06:19:27.581 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30659.log.gz 2024-09-14T06:19:27.581 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32705.log 2024-09-14T06:19:27.581 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33513.log 2024-09-14T06:19:27.581 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37576.log.gz 2024-09-14T06:19:27.581 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32705.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32705.log.gz 2024-09-14T06:19:27.581 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36037.log 2024-09-14T06:19:27.582 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36869.log 2024-09-14T06:19:27.582 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33513.log.gz 2024-09-14T06:19:27.582 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36037.log.gz 2024-09-14T06:19:27.582 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34350.log 2024-09-14T06:19:27.582 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35349.log 2024-09-14T06:19:27.582 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36869.log.gz 2024-09-14T06:19:27.582 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34350.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34350.log.gz 2024-09-14T06:19:27.582 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32809.log 2024-09-14T06:19:27.582 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38794.log 2024-09-14T06:19:27.582 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35349.log.gz 2024-09-14T06:19:27.582 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32809.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32809.log.gz 2024-09-14T06:19:27.583 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35484.log 2024-09-14T06:19:27.583 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32238.log 2024-09-14T06:19:27.583 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38794.log.gz 2024-09-14T06:19:27.583 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35484.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35484.log.gz 2024-09-14T06:19:27.583 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35664.log 2024-09-14T06:19:27.583 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29975.log 2024-09-14T06:19:27.583 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32238.log.gz 2024-09-14T06:19:27.583 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35664.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35664.log.gz 2024-09-14T06:19:27.583 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33466.log 2024-09-14T06:19:27.589 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.29975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30686.log 2024-09-14T06:19:27.590 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29975.log.gz 2024-09-14T06:19:27.590 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33466.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33466.log.gz 2024-09-14T06:19:27.590 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35442.log 2024-09-14T06:19:27.590 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37832.log 2024-09-14T06:19:27.590 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30686.log.gz 2024-09-14T06:19:27.590 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35442.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35442.log.gz 2024-09-14T06:19:27.590 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33043.log 2024-09-14T06:19:27.590 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30248.log 2024-09-14T06:19:27.590 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37832.log.gz 2024-09-14T06:19:27.590 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33043.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33043.log.gz 2024-09-14T06:19:27.590 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31225.log 2024-09-14T06:19:27.591 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34197.log 2024-09-14T06:19:27.591 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30248.log.gz 2024-09-14T06:19:27.591 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31225.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31225.log.gz 2024-09-14T06:19:27.591 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37754.log 2024-09-14T06:19:27.591 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35172.log 2024-09-14T06:19:27.591 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34197.log.gz 2024-09-14T06:19:27.591 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37754.log.gz 2024-09-14T06:19:27.591 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39158.log 2024-09-14T06:19:27.591 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35172.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-14T06:19:27.591 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35172.log.gz 2024-09-14T06:19:27.591 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39158.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39158.log.gz 2024-09-14T06:19:27.591 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37086.log 2024-09-14T06:19:27.591 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26466.log 2024-09-14T06:19:27.591 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37086.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.37086.log.gz --verbose 2024-09-14T06:19:27.592 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.37884.log 2024-09-14T06:19:27.592 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.26466.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26466.log.gz 2024-09-14T06:19:27.592 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31889.log 2024-09-14T06:19:27.592 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36651.log 2024-09-14T06:19:27.592 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37884.log: /var/log/ceph/ceph-client.admin.31889.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37884.log.gz 2024-09-14T06:19:27.592 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31889.log.gz 2024-09-14T06:19:27.592 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32522.log 2024-09-14T06:19:27.592 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30435.log 2024-09-14T06:19:27.592 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36651.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36651.log.gz 2024-09-14T06:19:27.592 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35082.log 2024-09-14T06:19:27.592 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32522.log.gz 2024-09-14T06:19:27.592 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30435.log.gz 2024-09-14T06:19:27.592 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-14T06:19:27.593 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37572.log 2024-09-14T06:19:27.593 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35082.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35082.log.gz 2024-09-14T06:19:27.593 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32902.log 2024-09-14T06:19:27.593 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37988.log 2024-09-14T06:19:27.593 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37572.log.gz 2024-09-14T06:19:27.600 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26499.log 2024-09-14T06:19:27.601 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26451.log 2024-09-14T06:19:27.601 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.26499.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26499.log.gz 2024-09-14T06:19:27.601 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-14T06:19:27.601 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.26451.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26451.log.gz 2024-09-14T06:19:27.601 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-14T06:19:27.601 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26427.log 2024-09-14T06:19:27.601 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-14T06:19:27.601 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26355.log 2024-09-14T06:19:27.601 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.26427.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26427.log.gz 2024-09-14T06:19:27.601 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26403.log 2024-09-14T06:19:27.601 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-14T06:19:27.601 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.26355.log: /var/log/ceph/ceph-client.admin.26403.log: 28.5% -- replaced with /var/log/ceph/ceph-client.admin.26355.log.gz 2024-09-14T06:19:27.601 INFO:teuthology.orchestra.run.smithi087.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26403.log.gz 2024-09-14T06:19:27.602 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-14T06:19:27.602 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26475.log/var/log/ceph/ceph-mgr.x.log: 2024-09-14T06:19:27.602 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26379.log 2024-09-14T06:19:27.602 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.26475.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.26475.log.gz -5 2024-09-14T06:19:27.602 INFO:teuthology.orchestra.run.smithi087.stderr: --verbose -- /var/log/ceph/ceph-client.admin.26315.log 2024-09-14T06:19:27.602 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.26379.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-14T06:19:27.602 INFO:teuthology.orchestra.run.smithi087.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26379.log.gz 2024-09-14T06:19:27.602 INFO:teuthology.orchestra.run.smithi087.stderr: 85.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-14T06:19:27.602 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.26315.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-14T06:19:27.602 INFO:teuthology.orchestra.run.smithi087.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26315.log.gz 2024-09-14T06:19:27.602 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-14T06:19:27.602 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.4.log: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-14T06:19:27.602 INFO:teuthology.orchestra.run.smithi087.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-14T06:19:27.618 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35869.log 2024-09-14T06:19:27.618 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32902.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32902.log.gz 2024-09-14T06:19:27.618 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30455.log 2024-09-14T06:19:27.618 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37988.log.gz 2024-09-14T06:19:27.618 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38482.log 2024-09-14T06:19:27.618 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35869.log.gz 2024-09-14T06:19:27.618 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37114.log 2024-09-14T06:19:27.618 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30455.log.gz 2024-09-14T06:19:27.618 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30249.log 2024-09-14T06:19:27.618 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38482.log.gz 2024-09-14T06:19:27.618 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32457.log 2024-09-14T06:19:27.619 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37114.log.gz 2024-09-14T06:19:27.619 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34455.log 2024-09-14T06:19:27.619 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30249.log.gz 2024-09-14T06:19:27.619 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37144.log 2024-09-14T06:19:27.619 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32457.log.gz 2024-09-14T06:19:27.619 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31592.log 2024-09-14T06:19:27.619 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34455.log.gz 2024-09-14T06:19:27.619 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38742.log 2024-09-14T06:19:27.619 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37144.log.gz 2024-09-14T06:19:27.619 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31592.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36290.log 2024-09-14T06:19:27.619 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31592.log.gz 2024-09-14T06:19:27.619 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37858.log 2024-09-14T06:19:27.620 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38742.log.gz 2024-09-14T06:19:27.620 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32149.log 2024-09-14T06:19:27.620 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36290.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36290.log.gz 2024-09-14T06:19:27.620 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37858.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.34115.log 2024-09-14T06:19:27.620 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37858.log.gz 2024-09-14T06:19:27.620 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34703.log 2024-09-14T06:19:27.620 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32149.log.gz 2024-09-14T06:19:27.620 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30196.log 2024-09-14T06:19:27.620 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34115.log.gz 2024-09-14T06:19:27.620 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37936.log 2024-09-14T06:19:27.620 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34703.log.gz 2024-09-14T06:19:27.620 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37307.log 2024-09-14T06:19:27.620 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30196.log.gz 2024-09-14T06:19:27.621 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38040.log 2024-09-14T06:19:27.621 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37936.log.gz 2024-09-14T06:19:27.621 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37962.log 2024-09-14T06:19:27.621 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37307.log.gz 2024-09-14T06:19:27.621 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30872.log 2024-09-14T06:19:27.621 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38040.log.gz 2024-09-14T06:19:27.621 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38846.log 2024-09-14T06:19:27.621 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37962.log.gz 2024-09-14T06:19:27.621 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30458.log 2024-09-14T06:19:27.621 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30872.log.gz 2024-09-14T06:19:27.621 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33540.log 2024-09-14T06:19:27.621 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38846.log.gz 2024-09-14T06:19:27.622 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37404.log 2024-09-14T06:19:27.622 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30458.log.gz 2024-09-14T06:19:27.622 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33202.log 2024-09-14T06:19:27.622 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33540.log.gz 2024-09-14T06:19:27.622 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32102.log 2024-09-14T06:19:27.622 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37404.log.gz 2024-09-14T06:19:27.622 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38084.log 2024-09-14T06:19:27.622 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33202.log.gz 2024-09-14T06:19:27.622 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37140.log 2024-09-14T06:19:27.622 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32102.log.gz 2024-09-14T06:19:27.622 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34041.log 2024-09-14T06:19:27.622 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38084.log.gz 2024-09-14T06:19:27.622 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37400.log 2024-09-14T06:19:27.623 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37140.log.gz 2024-09-14T06:19:27.623 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31510.log 2024-09-14T06:19:27.623 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34041.log.gz 2024-09-14T06:19:27.623 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30924.log 2024-09-14T06:19:27.623 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37400.log.gz 2024-09-14T06:19:27.623 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35205.log 2024-09-14T06:19:27.623 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31510.log.gz 2024-09-14T06:19:27.623 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35324.log 2024-09-14T06:19:27.623 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30924.log.gz 2024-09-14T06:19:27.623 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34977.log 2024-09-14T06:19:27.623 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35205.log.gz 2024-09-14T06:19:27.623 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35078.log 2024-09-14T06:19:27.623 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35324.log.gz 2024-09-14T06:19:27.624 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31888.log 2024-09-14T06:19:27.624 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34977.log.gz 2024-09-14T06:19:27.624 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38509.log 2024-09-14T06:19:27.624 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35078.log.gz 2024-09-14T06:19:27.624 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32328.log 2024-09-14T06:19:27.624 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31888.log.gz 2024-09-14T06:19:27.624 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32757.log 2024-09-14T06:19:27.624 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38509.log.gz 2024-09-14T06:19:27.624 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.31065.log 2024-09-14T06:19:27.624 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32328.log.gz 2024-09-14T06:19:27.625 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38274.log 2024-09-14T06:19:27.625 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32757.log.gz 2024-09-14T06:19:27.625 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.31065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30250.log 2024-09-14T06:19:27.625 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38274.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-14T06:19:27.625 INFO:teuthology.orchestra.run.smithi039.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.0.31065.log.gz 2024-09-14T06:19:27.625 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38274.log.gz 2024-09-14T06:19:27.625 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30250.log.gz 2024-09-14T06:19:27.625 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30638.log 2024-09-14T06:19:27.625 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31513.log 2024-09-14T06:19:27.625 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32584.log 2024-09-14T06:19:27.625 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30638.log.gz 2024-09-14T06:19:27.625 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39236.log 2024-09-14T06:19:27.626 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31513.log.gz 2024-09-14T06:19:27.626 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34861.log 2024-09-14T06:19:27.626 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32584.log.gz 2024-09-14T06:19:27.626 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35771.log 2024-09-14T06:19:27.626 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39236.log: /var/log/ceph/ceph-client.admin.34861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30241.log 2024-09-14T06:19:27.626 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39236.log.gz 2024-09-14T06:19:27.626 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34861.log.gz/var/log/ceph/ceph-client.admin.35771.log: 2024-09-14T06:19:27.626 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35771.log.gz 2024-09-14T06:19:27.626 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36087.log 2024-09-14T06:19:27.626 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30446.log 2024-09-14T06:19:27.627 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30241.log.gz 2024-09-14T06:19:27.627 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36672.log 2024-09-14T06:19:27.627 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36087.log.gz 2024-09-14T06:19:27.627 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36427.log 2024-09-14T06:19:27.627 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30446.log: /var/log/ceph/ceph-client.admin.36672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30240.log 2024-09-14T06:19:27.627 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36672.log.gz 2024-09-14T06:19:27.627 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30446.log.gz 2024-09-14T06:19:27.627 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36427.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36427.log.gz 2024-09-14T06:19:27.627 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38089.log 2024-09-14T06:19:27.627 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30430.log 2024-09-14T06:19:27.627 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30240.log: /var/log/ceph/ceph-client.admin.38089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34822.log 2024-09-14T06:19:27.627 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30240.log.gz 2024-09-14T06:19:27.627 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38089.log.gz/var/log/ceph/ceph-client.admin.30430.log: 2024-09-14T06:19:27.628 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30430.log.gz 2024-09-14T06:19:27.628 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37350.log 2024-09-14T06:19:27.628 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36397.log 2024-09-14T06:19:27.628 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34822.log: /var/log/ceph/ceph-client.admin.37350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33110.log 2024-09-14T06:19:27.628 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34822.log.gz 2024-09-14T06:19:27.628 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37350.log.gz 2024-09-14T06:19:27.628 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36397.log.gzgzip 2024-09-14T06:19:27.628 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33465.log 2024-09-14T06:19:27.628 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38498.log 2024-09-14T06:19:27.640 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33110.log: /var/log/ceph/ceph-client.admin.33465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33788.log 2024-09-14T06:19:27.640 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33110.log.gz 2024-09-14T06:19:27.640 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33465.log.gz 2024-09-14T06:19:27.640 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38498.log.gz 2024-09-14T06:19:27.640 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-14T06:19:27.640 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33788.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33788.log.gz 2024-09-14T06:19:27.640 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38976.log 2024-09-14T06:19:27.641 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36314.log 2024-09-14T06:19:27.641 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.38976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34588.log 2024-09-14T06:19:27.641 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38976.log.gz 2024-09-14T06:19:27.641 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36314.log.gz 2024-09-14T06:19:27.641 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37702.log 2024-09-14T06:19:27.641 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35817.log 2024-09-14T06:19:27.641 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34588.log.gz 2024-09-14T06:19:27.641 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37702.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37910.log 2024-09-14T06:19:27.641 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37702.log.gz 2024-09-14T06:19:27.641 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35817.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30656.log 2024-09-14T06:19:27.641 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35817.log.gz 2024-09-14T06:19:27.641 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32332.log 2024-09-14T06:19:27.641 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37910.log.gz/var/log/ceph/ceph-client.admin.30656.log: 2024-09-14T06:19:27.642 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30656.log.gz 2024-09-14T06:19:27.642 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36342.log 2024-09-14T06:19:27.642 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35482.log 2024-09-14T06:19:27.642 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32332.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32332.log.gz 2024-09-14T06:19:27.642 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36342.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36772.log 2024-09-14T06:19:27.642 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36342.log.gz 2024-09-14T06:19:27.642 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35482.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35482.log.gz 2024-09-14T06:19:27.642 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39338.log 2024-09-14T06:19:27.642 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32266.log 2024-09-14T06:19:27.642 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36772.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36772.log.gz 2024-09-14T06:19:27.642 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39338.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30443.log 2024-09-14T06:19:27.642 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39338.log.gz 2024-09-14T06:19:27.642 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32266.log.gz 2024-09-14T06:19:27.642 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30144.log 2024-09-14T06:19:27.643 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38820.log 2024-09-14T06:19:27.643 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30443.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30443.log.gz 2024-09-14T06:19:27.643 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30144.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30144.log.gz 2024-09-14T06:19:27.643 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33930.log 2024-09-14T06:19:27.643 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31636.log 2024-09-14T06:19:27.643 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38820.log.gz 2024-09-14T06:19:27.643 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33930.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35011.log 2024-09-14T06:19:27.643 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33930.log.gz 2024-09-14T06:19:27.643 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31636.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31636.log.gz 2024-09-14T06:19:27.643 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38716.log 2024-09-14T06:19:27.643 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38291.log 2024-09-14T06:19:27.643 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35011.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35011.log.gz 2024-09-14T06:19:27.643 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38716.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34236.log 2024-09-14T06:19:27.643 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38716.log.gz 2024-09-14T06:19:27.644 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38291.log.gz 2024-09-14T06:19:27.644 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33303.log 2024-09-14T06:19:27.644 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39080.log 2024-09-14T06:19:27.644 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34236.log.gz 2024-09-14T06:19:27.644 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33303.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33303.log.gz 2024-09-14T06:19:27.644 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32399.log 2024-09-14T06:19:27.644 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38092.log 2024-09-14T06:19:27.644 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39080.log.gz 2024-09-14T06:19:27.644 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32399.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31799.log 2024-09-14T06:19:27.644 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32399.log.gz 2024-09-14T06:19:27.644 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38092.log.gz 2024-09-14T06:19:27.644 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35576.log 2024-09-14T06:19:27.644 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-14T06:19:27.645 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31799.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31799.log.gz 2024-09-14T06:19:27.645 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35576.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35407.log 2024-09-14T06:19:27.645 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35576.log.gz 2024-09-14T06:19:27.652 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph.log: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-14T06:19:27.652 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39288.log 2024-09-14T06:19:27.652 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36473.log 2024-09-14T06:19:27.652 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35407.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35407.log.gz 2024-09-14T06:19:27.652 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31315.log 2024-09-14T06:19:27.652 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39288.log.gz 2024-09-14T06:19:27.652 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36473.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36473.log.gz 2024-09-14T06:19:27.652 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32982.log 2024-09-14T06:19:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36873.log 2024-09-14T06:19:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31315.log: /var/log/ceph/ceph-client.admin.32982.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31315.log.gz 2024-09-14T06:19:27.653 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32982.log.gz 2024-09-14T06:19:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37624.log 2024-09-14T06:19:27.653 INFO:teuthology.orchestra.run.smithi039.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-14T06:19:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36873.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36873.log.gz 2024-09-14T06:19:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31078.log 2024-09-14T06:19:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37546.log 2024-09-14T06:19:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37624.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37624.log.gz 2024-09-14T06:19:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38301.log 2024-09-14T06:19:27.653 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31078.log.gz 2024-09-14T06:19:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37780.log 2024-09-14T06:19:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37546.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37546.log.gz 2024-09-14T06:19:27.654 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26374.log 2024-09-14T06:19:27.654 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38301.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31977.log 2024-09-14T06:19:27.654 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38301.log.gz 2024-09-14T06:19:27.654 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37780.log: 0.0%/var/log/ceph/ceph-client.admin.26374.log: -- replaced with /var/log/ceph/ceph-client.admin.37780.log.gz 2024-09-14T06:19:27.654 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26374.log.gz 2024-09-14T06:19:27.654 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37676.log 2024-09-14T06:19:27.654 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33661.log 2024-09-14T06:19:27.654 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31977.log.gz 2024-09-14T06:19:27.654 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38690.log 2024-09-14T06:19:27.654 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37676.log.gz 2024-09-14T06:19:27.654 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30662.log 2024-09-14T06:19:27.654 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33661.log.gz 2024-09-14T06:19:27.654 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36000.log 2024-09-14T06:19:27.654 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38690.log: /var/log/ceph/ceph-client.admin.30662.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38690.log.gz 2024-09-14T06:19:27.655 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30662.log.gz 2024-09-14T06:19:27.655 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32928.log 2024-09-14T06:19:27.655 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38497.log 2024-09-14T06:19:27.655 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36000.log.gz 2024-09-14T06:19:27.655 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32928.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-14T06:19:27.655 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32928.log.gz 2024-09-14T06:19:27.658 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38497.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38497.log.gz 2024-09-14T06:19:27.658 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35722.log 2024-09-14T06:19:27.658 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37062.log 2024-09-14T06:19:27.659 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.35722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30098.log 2024-09-14T06:19:27.659 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35722.log.gz 2024-09-14T06:19:27.659 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37062.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37062.log.gz 2024-09-14T06:19:27.659 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38924.log 2024-09-14T06:19:27.659 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33615.log 2024-09-14T06:19:27.659 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30098.log.gz 2024-09-14T06:19:27.659 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38924.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34374.log 2024-09-14T06:19:27.659 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38924.log.gz 2024-09-14T06:19:27.659 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33615.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33615.log.gz 2024-09-14T06:19:27.659 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35483.log 2024-09-14T06:19:27.659 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34298.log 2024-09-14T06:19:27.660 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34374.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34374.log.gz 2024-09-14T06:19:27.660 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35483.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35483.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36115.log 2024-09-14T06:19:27.660 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-14T06:19:27.662 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34298.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34298.log.gz 2024-09-14T06:19:27.662 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30246.log 2024-09-14T06:19:27.662 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36115.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36115.log.gz 2024-09-14T06:19:27.663 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37235.log 2024-09-14T06:19:27.663 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34649.log 2024-09-14T06:19:27.663 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30246.log: /var/log/ceph/ceph-client.admin.37235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30246.log.gz 2024-09-14T06:19:27.663 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37235.log.gz 2024-09-14T06:19:27.663 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34792.log 2024-09-14T06:19:27.663 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38083.log 2024-09-14T06:19:27.663 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34649.log.gz 2024-09-14T06:19:27.663 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34792.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30118.log 2024-09-14T06:19:27.663 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34792.log.gz 2024-09-14T06:19:27.663 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38083.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38083.log.gz 2024-09-14T06:19:27.665 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30898.log 2024-09-14T06:19:27.665 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34558.log 2024-09-14T06:19:27.665 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30118.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30118.log.gz 2024-09-14T06:19:27.665 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30898.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.33747.log 2024-09-14T06:19:27.665 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30898.log.gz 2024-09-14T06:19:27.665 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34558.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34558.log.gz 2024-09-14T06:19:27.665 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32190.log 2024-09-14T06:19:27.665 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34224.log 2024-09-14T06:19:27.667 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33747.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33747.log.gz 2024-09-14T06:19:27.667 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32190.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26220.log 2024-09-14T06:19:27.667 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-14T06:19:27.667 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34224.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34224.log.gz 2024-09-14T06:19:27.667 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35283.log 2024-09-14T06:19:27.668 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33722.log 2024-09-14T06:19:27.669 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35283.log: /var/log/ceph/ceph-client.admin.26220.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26220.log.gz 2024-09-14T06:19:27.669 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35283.log.gz 2024-09-14T06:19:27.669 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35950.log 2024-09-14T06:19:27.669 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-14T06:19:27.669 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33722.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33722.log.gz 2024-09-14T06:19:27.670 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35950.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35950.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32497.log 2024-09-14T06:19:27.670 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-14T06:19:27.671 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30170.log 2024-09-14T06:19:27.672 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32497.log: 93.9% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32497.log.gz 2024-09-14T06:19:27.672 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-14T06:19:27.673 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32429.log 2024-09-14T06:19:27.673 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37207.log 2024-09-14T06:19:27.673 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30170.log.gz 2024-09-14T06:19:27.673 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32429.log.gz 2024-09-14T06:19:27.675 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32007.log 2024-09-14T06:19:27.675 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36802.log 2024-09-14T06:19:27.675 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37207.log.gz 2024-09-14T06:19:27.675 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32007.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36206.log 2024-09-14T06:19:27.675 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32007.log.gz 2024-09-14T06:19:27.686 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36802.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36802.log.gz 2024-09-14T06:19:27.686 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31002.log 2024-09-14T06:19:27.686 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36987.log 2024-09-14T06:19:27.686 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36206.log: /var/log/ceph/ceph-client.admin.31002.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36206.log.gz 2024-09-14T06:19:27.686 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31002.log.gz 2024-09-14T06:19:27.686 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35767.log 2024-09-14T06:19:27.687 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38283.log 2024-09-14T06:19:27.687 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36987.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36987.log.gz 2024-09-14T06:19:27.687 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35767.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36979.log 2024-09-14T06:19:27.687 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-14T06:19:27.687 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38283.log.gz 2024-09-14T06:19:27.687 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31641.log 2024-09-14T06:19:27.687 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30073.log 2024-09-14T06:19:27.687 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36979.log: /var/log/ceph/ceph-client.admin.31641.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36979.log.gz 2024-09-14T06:19:27.687 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31641.log.gz 2024-09-14T06:19:27.687 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30976.log 2024-09-14T06:19:27.687 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32675.log 2024-09-14T06:19:27.687 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30073.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30073.log.gz 2024-09-14T06:19:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30976.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30976.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38302.log 2024-09-14T06:19:27.688 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-14T06:19:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32675.log.gz 2024-09-14T06:19:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39262.log 2024-09-14T06:19:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39210.log 2024-09-14T06:19:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38302.log.gz 2024-09-14T06:19:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39262.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.39262.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.36713.log 2024-09-14T06:19:27.688 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-14T06:19:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39210.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39210.log.gz 2024-09-14T06:19:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38898.log 2024-09-14T06:19:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31388.log 2024-09-14T06:19:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36713.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36713.log.gz 2024-09-14T06:19:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38898.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39314.log 2024-09-14T06:19:27.689 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-14T06:19:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31388.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31388.log.gz 2024-09-14T06:19:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30454.log 2024-09-14T06:19:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39002.log 2024-09-14T06:19:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39314.log.gz 2024-09-14T06:19:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30454.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.30454.log.gz 2024-09-14T06:19:27.689 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39106.log 2024-09-14T06:19:27.694 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39002.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39002.log.gz 2024-09-14T06:19:27.694 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34933.log 2024-09-14T06:19:27.694 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30950.log 2024-09-14T06:19:27.694 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39106.log: /var/log/ceph/ceph-client.admin.34933.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39106.log.gz 2024-09-14T06:19:27.694 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34933.log.gz 2024-09-14T06:19:27.694 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35211.log 2024-09-14T06:19:27.694 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33896.log 2024-09-14T06:19:27.695 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30950.log: /var/log/ceph/ceph-client.admin.35211.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30950.log.gz 2024-09-14T06:19:27.695 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35211.log.gz 2024-09-14T06:19:27.695 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30660.log 2024-09-14T06:19:27.695 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30661.log 2024-09-14T06:19:27.695 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33896.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33896.log.gz 2024-09-14T06:19:27.695 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30660.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.30660.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.31724.log 2024-09-14T06:19:27.695 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-14T06:19:27.695 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30661.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30661.log.gz 2024-09-14T06:19:27.695 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33069.log 2024-09-14T06:19:27.695 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32862.log 2024-09-14T06:19:27.695 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31724.log: /var/log/ceph/ceph-client.admin.33069.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31724.log.gz 2024-09-14T06:19:27.695 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33069.log.gz 2024-09-14T06:19:27.695 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34699.log 2024-09-14T06:19:27.695 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38294.log 2024-09-14T06:19:27.696 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32862.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32862.log.gz 2024-09-14T06:19:27.696 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34699.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34699.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38491.log 2024-09-14T06:19:27.696 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-14T06:19:27.696 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38294.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38294.log.gz 2024-09-14T06:19:27.696 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33409.log 2024-09-14T06:19:27.700 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34517.log 2024-09-14T06:19:27.700 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38491.log: /var/log/ceph/ceph-client.admin.33409.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38491.log.gz 2024-09-14T06:19:27.700 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33409.log.gz 2024-09-14T06:19:27.700 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32783.log 2024-09-14T06:19:27.700 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33307.log 2024-09-14T06:19:27.700 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34517.log: /var/log/ceph/ceph-client.admin.32783.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34517.log.gz 2024-09-14T06:19:27.700 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32783.log.gz 2024-09-14T06:19:27.700 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34324.log 2024-09-14T06:19:27.700 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38492.log 2024-09-14T06:19:27.700 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33307.log.gz 2024-09-14T06:19:27.700 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34324.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34324.log.gz 2024-09-14T06:19:27.700 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38510.log 2024-09-14T06:19:27.701 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38492.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38492.log.gz 2024-09-14T06:19:27.701 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35902.log 2024-09-14T06:19:27.701 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31937.log 2024-09-14T06:19:27.701 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38510.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38510.log.gz 2024-09-14T06:19:27.701 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39028.log 2024-09-14T06:19:27.701 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35902.log.gz 2024-09-14T06:19:27.701 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31937.log.gz 2024-09-14T06:19:27.704 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30048.log 2024-09-14T06:19:27.704 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38075.log 2024-09-14T06:19:27.704 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39028.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39028.log.gz 2024-09-14T06:19:27.704 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30048.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30048.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21426.log 2024-09-14T06:19:27.704 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-14T06:19:27.704 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38075.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38075.log.gz 2024-09-14T06:19:27.704 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35611.log 2024-09-14T06:19:27.704 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36609.log 2024-09-14T06:19:27.704 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph.tmp-client.admin.21426.log: /var/log/ceph/ceph-client.admin.35611.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21426.log.gz 2024-09-14T06:19:27.705 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35611.log.gz 2024-09-14T06:19:27.705 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30642.log 2024-09-14T06:19:27.705 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32588.log 2024-09-14T06:19:27.705 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36609.log.gz 2024-09-14T06:19:27.705 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30642.log.gz 2024-09-14T06:19:27.706 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35845.log 2024-09-14T06:19:27.706 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32296.log 2024-09-14T06:19:27.706 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32588.log.gz 2024-09-14T06:19:27.707 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35845.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35845.log.gz 2024-09-14T06:19:27.707 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37628.log 2024-09-14T06:19:27.707 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33144.log 2024-09-14T06:19:27.707 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32296.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32296.log.gz 2024-09-14T06:19:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37628.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30234.log 2024-09-14T06:19:27.708 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37628.log.gz 2024-09-14T06:19:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33144.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33144.log.gz 2024-09-14T06:19:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30846.log 2024-09-14T06:19:27.709 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36178.log 2024-09-14T06:19:27.709 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30234.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30234.log.gz 2024-09-14T06:19:27.710 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30846.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30846.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37271.log 2024-09-14T06:19:27.710 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-14T06:19:27.710 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36178.log.gz 2024-09-14T06:19:27.710 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38872.log 2024-09-14T06:19:27.710 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36338.log 2024-09-14T06:19:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37271.log: /var/log/ceph/ceph-client.admin.38872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37271.log.gz 2024-09-14T06:19:27.715 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38872.log.gz 2024-09-14T06:19:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33240.log 2024-09-14T06:19:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38093.log 2024-09-14T06:19:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36338.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36338.log.gz 2024-09-14T06:19:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33240.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31077.log 2024-09-14T06:19:27.715 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33240.log.gz 2024-09-14T06:19:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38093.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38093.log.gz 2024-09-14T06:19:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30439.log 2024-09-14T06:19:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33847.log 2024-09-14T06:19:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31077.log.gz 2024-09-14T06:19:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.30439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30439.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31384.log 2024-09-14T06:19:27.716 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-14T06:19:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.33847.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33847.log.gz 2024-09-14T06:19:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38094.log 2024-09-14T06:19:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36827.log 2024-09-14T06:19:27.721 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31384.log: /var/log/ceph/ceph-client.admin.38094.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31384.log.gz 2024-09-14T06:19:27.722 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38094.log.gz 2024-09-14T06:19:27.722 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36262.log 2024-09-14T06:19:27.722 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36827.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36827.log.gz 2024-09-14T06:19:27.722 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32813.log 2024-09-14T06:19:27.722 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38768.log 2024-09-14T06:19:27.722 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36262.log: /var/log/ceph/ceph-client.admin.32813.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36262.log.gz 2024-09-14T06:19:27.722 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32813.log.gz 2024-09-14T06:19:27.723 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36538.log 2024-09-14T06:19:27.723 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31032.log 2024-09-14T06:19:27.723 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38768.log: /var/log/ceph/ceph-client.admin.36538.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38768.log.gz 2024-09-14T06:19:27.723 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36538.log.gz 2024-09-14T06:19:27.723 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37806.log 2024-09-14T06:19:27.723 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31032.log.gz/var/log/ceph/ceph-client.admin.37806.log: 2024-09-14T06:19:27.723 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37806.log.gz 2024-09-14T06:19:27.744 INFO:teuthology.orchestra.run.smithi087.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-14T06:19:27.810 INFO:teuthology.orchestra.run.smithi039.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-14T06:19:27.867 INFO:teuthology.orchestra.run.smithi039.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-14T06:19:28.105 INFO:teuthology.orchestra.run.smithi087.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-14T06:19:28.297 INFO:teuthology.orchestra.run.smithi039.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-14T06:19:28.610 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-14T06:19:28.727 INFO:teuthology.orchestra.run.smithi087.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-14T06:19:29.197 INFO:teuthology.orchestra.run.smithi087.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-14T06:19:29.243 INFO:teuthology.orchestra.run.smithi039.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-14T06:19:29.296 INFO:teuthology.orchestra.run.smithi087.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-14T06:19:29.298 INFO:teuthology.orchestra.run.smithi087.stderr: 2024-09-14T06:19:29.298 INFO:teuthology.orchestra.run.smithi087.stderr:real 0m1.729s 2024-09-14T06:19:29.298 INFO:teuthology.orchestra.run.smithi087.stderr:user 0m4.905s 2024-09-14T06:19:29.298 INFO:teuthology.orchestra.run.smithi087.stderr:sys 0m0.259s 2024-09-14T06:19:29.770 INFO:teuthology.orchestra.run.smithi039.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-14T06:19:29.807 INFO:teuthology.orchestra.run.smithi039.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-14T06:19:29.809 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-14T06:19:29.809 INFO:teuthology.orchestra.run.smithi039.stderr:real 0m2.278s 2024-09-14T06:19:29.809 INFO:teuthology.orchestra.run.smithi039.stderr:user 0m7.212s 2024-09-14T06:19:29.809 INFO:teuthology.orchestra.run.smithi039.stderr:sys 0m0.344s 2024-09-14T06:19:29.809 INFO:tasks.ceph:Archiving logs... 2024-09-14T06:19:29.809 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900364/remote/smithi039/log 2024-09-14T06:19:29.810 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T06:19:30.804 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900364/remote/smithi087/log 2024-09-14T06:19:30.805 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T06:19:31.353 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-14T06:19:31.364 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-14T06:19:31.365 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T06:19:31.367 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T06:19:31.445 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-14T06:19:31.456 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-14T06:19:31.456 DEBUG:teuthology.orchestra.run.smithi039:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T06:19:31.459 DEBUG:teuthology.orchestra.run.smithi087:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T06:19:31.470 INFO:teuthology.orchestra.run.smithi087.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T06:19:31.470 INFO:teuthology.orchestra.run.smithi087.stdout:============================================================================== 2024-09-14T06:19:31.470 INFO:teuthology.orchestra.run.smithi087.stdout:+hv01.front.sepi 67.205.162.81 3 u 45 64 77 0.104 +0.050 1.445 2024-09-14T06:19:31.470 INFO:teuthology.orchestra.run.smithi087.stdout:+hv02.front.sepi 63.231.80.2 3 u 43 64 77 0.142 -0.949 1.416 2024-09-14T06:19:31.470 INFO:teuthology.orchestra.run.smithi087.stdout:*hv03.front.sepi 74.6.168.72 3 u 47 64 77 0.133 -0.382 1.417 2024-09-14T06:19:31.470 INFO:teuthology.orchestra.run.smithi087.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T06:19:31.472 INFO:teuthology.orchestra.run.smithi039.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T06:19:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:============================================================================== 2024-09-14T06:19:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:+hv01.front.sepi 67.205.162.81 3 u 45 64 77 0.118 -0.248 0.631 2024-09-14T06:19:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:+hv02.front.sepi 63.231.80.2 3 u 52 64 77 0.137 -1.187 0.695 2024-09-14T06:19:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:*hv03.front.sepi 74.6.168.72 3 u 46 64 77 0.109 -0.659 0.706 2024-09-14T06:19:31.472 INFO:teuthology.orchestra.run.smithi039.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T06:19:31.472 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-14T06:19:31.535 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-14T06:19:31.535 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-14T06:19:31.601 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-14T06:19:31.681 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-14T06:19:31.774 INFO:teuthology.task.internal:Duration was 834.073864 seconds 2024-09-14T06:19:31.775 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-14T06:19:31.876 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-14T06:19:31.876 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T06:19:31.879 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T06:19:31.917 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-14T06:19:31.917 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi039.front.sepia.ceph.com 2024-09-14T06:19:31.918 DEBUG:teuthology.orchestra.run.smithi039:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T06:19:31.969 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi087.front.sepia.ceph.com 2024-09-14T06:19:31.969 DEBUG:teuthology.orchestra.run.smithi087:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T06:19:31.979 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-14T06:19:31.979 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T06:19:32.013 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T06:19:32.047 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-14T06:19:32.048 DEBUG:teuthology.orchestra.run.smithi039:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T06:19:32.084 DEBUG:teuthology.orchestra.run.smithi087:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T06:19:32.155 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-14T06:19:32.167 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-14T06:19:32.167 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T06:19:32.178 DEBUG:teuthology.orchestra.run.smithi087:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T06:19:32.189 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-14T06:19:32.214 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T06:19:32.220 DEBUG:teuthology.orchestra.run.smithi087:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T06:19:32.229 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = core 2024-09-14T06:19:32.242 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = core 2024-09-14T06:19:32.273 DEBUG:teuthology.orchestra.run.smithi039:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T06:19:32.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:19:32.311 DEBUG:teuthology.orchestra.run.smithi087:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T06:19:32.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:19:32.319 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-14T06:19:32.378 INFO:teuthology.task.internal:Transferring archived files... 2024-09-14T06:19:32.379 DEBUG:teuthology.misc:Transferring archived files from smithi039:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900364/remote/smithi039 2024-09-14T06:19:32.379 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T06:19:32.406 DEBUG:teuthology.misc:Transferring archived files from smithi087:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900364/remote/smithi087 2024-09-14T06:19:32.406 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T06:19:32.429 INFO:teuthology.task.internal:Removing archive directory... 2024-09-14T06:19:32.429 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T06:19:32.445 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T06:19:32.472 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-14T06:19:32.486 INFO:teuthology.task.internal:Not uploading archives. 2024-09-14T06:19:32.486 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-14T06:19:32.600 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-14T06:19:32.600 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T06:19:32.603 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T06:19:32.607 INFO:teuthology.orchestra.run.smithi039.stdout: 658642 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 06:19 /home/ubuntu/cephtest 2024-09-14T06:19:32.609 INFO:teuthology.orchestra.run.smithi087.stdout: 658637 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 06:19 /home/ubuntu/cephtest 2024-09-14T06:19:32.610 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-14T06:19:32.693 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-14T06:19:32.864 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_copy_on_read} duration: 834.0738642215729 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-14T06:19:32.864 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T06:19:32.952 INFO:teuthology.run:pass